browsertrix/backend/btrixcloud
Ilya Kreymer dce1ae6129
better resources scaling by number of browsers per crawler container (#1103)
- set crawler cpu / memory with fixed base + incremental bumps based on number of browsers
- allow parsing k8s quantities with parse_quantity, compute in operator
- set 'crawler_cpu = crawler_cpu_base + crawler_extra_cpu_per_browser * (num_browsers - 1)'
and same for memory
2023-09-06 01:42:44 -04:00
..
migrations feat: implement 'collections' array with {name, id} for archived item details (#1098) 2023-08-25 00:26:46 -07:00
templates better resources scaling by number of browsers per crawler container (#1103) 2023-09-06 01:42:44 -04:00
__init__.py
basecrawls.py null check: check before accessing config in 'get_all_crawl_search_values' (#1144) 2023-09-05 23:57:05 -04:00
colls.py Add event webhook notifications system to backend (#1061) 2023-08-31 19:52:37 -07:00
crawlconfigs.py Add event webhook notifications system to backend (#1061) 2023-08-31 19:52:37 -07:00
crawlmanager.py Add max crawl size option to backend and frontend (#1045) 2023-08-26 22:00:37 -07:00
crawls.py Add event webhook notifications system to backend (#1061) 2023-08-31 19:52:37 -07:00
db.py feat: implement 'collections' array with {name, id} for archived item details (#1098) 2023-08-25 00:26:46 -07:00
emailsender.py
invites.py
k8sapi.py Add max crawl size option to backend and frontend (#1045) 2023-08-26 22:00:37 -07:00
main_op.py Add event webhook notifications system to backend (#1061) 2023-08-31 19:52:37 -07:00
main_scheduled_job.py Add event webhook notifications system to backend (#1061) 2023-08-31 19:52:37 -07:00
main.py Add event webhook notifications system to backend (#1061) 2023-08-31 19:52:37 -07:00
models.py Add event webhook notifications system to backend (#1061) 2023-08-31 19:52:37 -07:00
operator.py better resources scaling by number of browsers per crawler container (#1103) 2023-09-06 01:42:44 -04:00
orgs.py Add event webhook notifications system to backend (#1061) 2023-08-31 19:52:37 -07:00
pagination.py
profiles.py
storages.py
uploads.py Add event webhook notifications system to backend (#1061) 2023-08-31 19:52:37 -07:00
users.py
utils.py
version.py
webhooks.py Add event webhook notifications system to backend (#1061) 2023-08-31 19:52:37 -07:00
zip.py