tea/browsertrix
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
551660bb62
browsertrix/backend/btrixcloud/migrations
History
Tessa Walsh d38abbca7f
Standardize handling of storage and execution time quotas (#1969) 
Fixes #1968 

Changes:
- `stopped_quota_reached` and `skipped_quota_reached` migrated to new
values that indicate which quota was reached
- Before crawls are run, the operator checks if storage or exec mins
quotas are reached and if so fails the crawl with the appropriate state
of `skipped_storage_quota_reached` or `skipped_time_quota_reached`
- While crawls are running, the operator checks if the exec mins quota
is reached or if the size of all running crawls will mean the storage
quota is reached once uploaded; if so, the crawl is stopped gracefully
and given `stopped_storage_quota_needed` or `stopped_time_quota_reached`
state as appropriate
- Adds new nightly tests for enforcing storage quota
2024-07-25 12:49:11 -07:00
..
__init__.py Format backend with Black 24 (#1507) 2024-02-07 11:35:34 -08:00
migration_0001_archives_to_orgs.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0002_crawlconfig_crawlstats.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0003_mutable_crawl_configs.py Add created date to Organization and fix datetimes across backend (#1921) 2024-07-15 19:46:32 -07:00
migration_0004_config_seeds.py Pydantic 2.x update + type fixes + python 3.12 (#1947) 2024-07-22 17:23:03 -07:00
migration_0005_operator_scheduled_jobs.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0006_precompute_crawl_stats.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0007_colls_and_config_update.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0008_precompute_crawl_file_stats.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0009_crawl_types.py Change crawl.reviewStatus to 1-5 scale int (#1664) 2024-04-09 17:51:06 -07:00
migration_0010_collection_total_size.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0011_crawl_timeout_configmap.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0012_notes_to_description.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0013_crawl_name.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0014_to_collection_ids.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0015_org_storage_usage.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0016_operator_scheduled_jobs_v2.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0017_storage_by_type.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0018_usernames.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0019_org_slug.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0020_org_storage_refs.py quickfix: fix for additional lint error in updated pylint (#1805) 2024-05-15 17:00:15 -07:00
migration_0021_profile_filenames.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0022_partial_complete.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0023_available_extra_exec_mins.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0024_crawlerchannel.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0025_workflow_db_configmap_fixes.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0026_crawl_review_status.py Change crawl.reviewStatus to 1-5 scale int (#1664) 2024-04-09 17:51:06 -07:00
migration_0027_profile_modified.py Add migration to set profile modified date (#1832) 2024-05-29 15:56:27 -04:00
migration_0028_page_files_errors.py Backend: Move page file and error counts to crawl replay.json endpoint (#1868) 2024-06-20 19:02:57 -07:00
migration_0029_remove_workflow_configmaps.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0030_user_invites_flatten.py Refactor Invites and Registration, Flatten Per-User Invites (#1902) 2024-07-02 15:13:27 -07:00
migration_0031_org_created.py Add created date to Organization and fix datetimes across backend (#1921) 2024-07-15 19:46:32 -07:00
migration_0032_dupe_org_names.py Ensure org name and slug uniqueness is case-insensitive (#1929) 2024-07-18 15:30:12 -07:00
migration_0033_crawl_quota_states.py Standardize handling of storage and execution time quotas (#1969) 2024-07-25 12:49:11 -07:00