browsertrix/backend/btrixcloud/migrations
Tessa Walsh d38abbca7f
Standardize handling of storage and execution time quotas (#1969)
Fixes #1968 

Changes:
- `stopped_quota_reached` and `skipped_quota_reached` migrated to new
values that indicate which quota was reached
- Before crawls are run, the operator checks if storage or exec mins
quotas are reached and if so fails the crawl with the appropriate state
of `skipped_storage_quota_reached` or `skipped_time_quota_reached`
- While crawls are running, the operator checks if the exec mins quota
is reached or if the size of all running crawls will mean the storage
quota is reached once uploaded; if so, the crawl is stopped gracefully
and given `stopped_storage_quota_needed` or `stopped_time_quota_reached`
state as appropriate
- Adds new nightly tests for enforcing storage quota
2024-07-25 12:49:11 -07:00
..
__init__.py
migration_0001_archives_to_orgs.py
migration_0002_crawlconfig_crawlstats.py
migration_0003_mutable_crawl_configs.py Add created date to Organization and fix datetimes across backend (#1921) 2024-07-15 19:46:32 -07:00
migration_0004_config_seeds.py Pydantic 2.x update + type fixes + python 3.12 (#1947) 2024-07-22 17:23:03 -07:00
migration_0005_operator_scheduled_jobs.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0006_precompute_crawl_stats.py
migration_0007_colls_and_config_update.py
migration_0008_precompute_crawl_file_stats.py
migration_0009_crawl_types.py Change crawl.reviewStatus to 1-5 scale int (#1664) 2024-04-09 17:51:06 -07:00
migration_0010_collection_total_size.py
migration_0011_crawl_timeout_configmap.py
migration_0012_notes_to_description.py
migration_0013_crawl_name.py
migration_0014_to_collection_ids.py
migration_0015_org_storage_usage.py
migration_0016_operator_scheduled_jobs_v2.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0017_storage_by_type.py
migration_0018_usernames.py
migration_0019_org_slug.py
migration_0020_org_storage_refs.py quickfix: fix for additional lint error in updated pylint (#1805) 2024-05-15 17:00:15 -07:00
migration_0021_profile_filenames.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0022_partial_complete.py
migration_0023_available_extra_exec_mins.py
migration_0024_crawlerchannel.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0025_workflow_db_configmap_fixes.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0026_crawl_review_status.py Change crawl.reviewStatus to 1-5 scale int (#1664) 2024-04-09 17:51:06 -07:00
migration_0027_profile_modified.py Add migration to set profile modified date (#1832) 2024-05-29 15:56:27 -04:00
migration_0028_page_files_errors.py Backend: Move page file and error counts to crawl replay.json endpoint (#1868) 2024-06-20 19:02:57 -07:00
migration_0029_remove_workflow_configmaps.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0030_user_invites_flatten.py Refactor Invites and Registration, Flatten Per-User Invites (#1902) 2024-07-02 15:13:27 -07:00
migration_0031_org_created.py Add created date to Organization and fix datetimes across backend (#1921) 2024-07-15 19:46:32 -07:00
migration_0032_dupe_org_names.py Ensure org name and slug uniqueness is case-insensitive (#1929) 2024-07-18 15:30:12 -07:00
migration_0033_crawl_quota_states.py Standardize handling of storage and execution time quotas (#1969) 2024-07-25 12:49:11 -07:00