browsertrix/backend/btrixcloud/migrations
Ilya Kreymer 3cd52342a7
Remove Crawl Workflow Configmaps (#1894)
Fixes #1893 

- Removes crawl workflow-scoped configmaps, and replaces with operator-controlled
per-crawl configmaps that only contain the json config passed to Browsertrix
Crawler (as a volume).
- Other configmap settings replaced are replaced the custom CrawlJob options
(mostly already were, just added profile_filename and storage_filename)
- Cron jobs also updated to create CrawlJob without relying on configmaps,
querying the db for additional settings.
- The `userid` associated with cron jobs is set to the user that last modified
 the schedule of the crawl, rather than whomever last modified the workflow
- Various functions that deal with updating configmaps have been removed,
including in migrations.
- New migration 0029 added to remove all crawl workflow configmaps
2024-06-28 15:25:23 -07:00
..
__init__.py
migration_0001_archives_to_orgs.py
migration_0002_crawlconfig_crawlstats.py
migration_0003_mutable_crawl_configs.py
migration_0004_config_seeds.py
migration_0005_operator_scheduled_jobs.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0006_precompute_crawl_stats.py
migration_0007_colls_and_config_update.py
migration_0008_precompute_crawl_file_stats.py
migration_0009_crawl_types.py
migration_0010_collection_total_size.py
migration_0011_crawl_timeout_configmap.py
migration_0012_notes_to_description.py
migration_0013_crawl_name.py
migration_0014_to_collection_ids.py
migration_0015_org_storage_usage.py
migration_0016_operator_scheduled_jobs_v2.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0017_storage_by_type.py
migration_0018_usernames.py
migration_0019_org_slug.py
migration_0020_org_storage_refs.py
migration_0021_profile_filenames.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0022_partial_complete.py
migration_0023_available_extra_exec_mins.py
migration_0024_crawlerchannel.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0025_workflow_db_configmap_fixes.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0026_crawl_review_status.py
migration_0027_profile_modified.py
migration_0028_page_files_errors.py
migration_0029_remove_workflow_configmaps.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00