browsertrix/backend/btrixcloud
Tessa Walsh d38abbca7f
Standardize handling of storage and execution time quotas (#1969)
Fixes #1968 

Changes:
- `stopped_quota_reached` and `skipped_quota_reached` migrated to new
values that indicate which quota was reached
- Before crawls are run, the operator checks if storage or exec mins
quotas are reached and if so fails the crawl with the appropriate state
of `skipped_storage_quota_reached` or `skipped_time_quota_reached`
- While crawls are running, the operator checks if the exec mins quota
is reached or if the size of all running crawls will mean the storage
quota is reached once uploaded; if so, the crawl is stopped gracefully
and given `stopped_storage_quota_needed` or `stopped_time_quota_reached`
state as appropriate
- Adds new nightly tests for enforcing storage quota
2024-07-25 12:49:11 -07:00
..
migrations Standardize handling of storage and execution time quotas (#1969) 2024-07-25 12:49:11 -07:00
operator Standardize handling of storage and execution time quotas (#1969) 2024-07-25 12:49:11 -07:00
__init__.py
auth.py Add created date to Organization and fix datetimes across backend (#1921) 2024-07-15 19:46:32 -07:00
background_jobs.py api docs cleanup + readd webhooks: (#1949) 2024-07-22 09:00:59 -07:00
basecrawls.py Implement downloading archived item + QA runs as multi-WACZ (#1933) 2024-07-25 10:28:57 -07:00
colls.py Document all API endpoints with response models (#1928) 2024-07-16 12:48:38 -07:00
crawlconfigs.py Pydantic 2.x update + type fixes + python 3.12 (#1947) 2024-07-22 17:23:03 -07:00
crawlmanager.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
crawls.py Implement downloading archived item + QA runs as multi-WACZ (#1933) 2024-07-25 10:28:57 -07:00
db.py Standardize handling of storage and execution time quotas (#1969) 2024-07-25 12:49:11 -07:00
emailsender.py Additional typing cleanup (#1938) 2024-07-17 10:49:22 -07:00
invites.py Add created date to Organization and fix datetimes across backend (#1921) 2024-07-15 19:46:32 -07:00
k8sapi.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
main_op.py Add superuser API endpoints to export and import org data (#1394) 2024-07-02 17:14:34 -04:00
main.py Add support e-mail to settings (#1960) 2024-07-23 20:58:12 -04:00
models.py Standardize handling of storage and execution time quotas (#1969) 2024-07-25 12:49:11 -07:00
orgs.py Standardize handling of storage and execution time quotas (#1969) 2024-07-25 12:49:11 -07:00
pages.py Pydantic 2.x update + type fixes + python 3.12 (#1947) 2024-07-22 17:23:03 -07:00
pagination.py
profiles.py Pydantic 2.x update + type fixes + python 3.12 (#1947) 2024-07-22 17:23:03 -07:00
storages.py Implement downloading archived item + QA runs as multi-WACZ (#1933) 2024-07-25 10:28:57 -07:00
subs.py Add API endpoint to import subscription for existing org (#1930) 2024-07-16 16:17:02 -07:00
uploads.py Implement downloading archived item + QA runs as multi-WACZ (#1933) 2024-07-25 10:28:57 -07:00
users.py Document all API endpoints with response models (#1928) 2024-07-16 12:48:38 -07:00
utils.py Additional typing cleanup (#1938) 2024-07-17 10:49:22 -07:00
version.py version: bump to 1.11.0-beta.1 2024-07-17 10:06:49 -07:00
webhooks.py Document all API endpoints with response models (#1928) 2024-07-16 12:48:38 -07:00