tea/browsertrix
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
35469915a8
browsertrix/backend/btrixcloud/migrations
History
Ilya Kreymer 3cd52342a7
Remove Crawl Workflow Configmaps (#1894) 
Fixes #1893 

- Removes crawl workflow-scoped configmaps, and replaces with operator-controlled
per-crawl configmaps that only contain the json config passed to Browsertrix
Crawler (as a volume).
- Other configmap settings replaced are replaced the custom CrawlJob options
(mostly already were, just added profile_filename and storage_filename)
- Cron jobs also updated to create CrawlJob without relying on configmaps,
querying the db for additional settings.
- The `userid` associated with cron jobs is set to the user that last modified
 the schedule of the crawl, rather than whomever last modified the workflow
- Various functions that deal with updating configmaps have been removed,
including in migrations.
- New migration 0029 added to remove all crawl workflow configmaps
2024-06-28 15:25:23 -07:00
..
__init__.py Format backend with Black 24 (#1507) 2024-02-07 11:35:34 -08:00
migration_0001_archives_to_orgs.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0002_crawlconfig_crawlstats.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0003_mutable_crawl_configs.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0004_config_seeds.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
migration_0005_operator_scheduled_jobs.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0006_precompute_crawl_stats.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0007_colls_and_config_update.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0008_precompute_crawl_file_stats.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0009_crawl_types.py Change crawl.reviewStatus to 1-5 scale int (#1664) 2024-04-09 17:51:06 -07:00
migration_0010_collection_total_size.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0011_crawl_timeout_configmap.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0012_notes_to_description.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0013_crawl_name.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0014_to_collection_ids.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0015_org_storage_usage.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0016_operator_scheduled_jobs_v2.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0017_storage_by_type.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0018_usernames.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0019_org_slug.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0020_org_storage_refs.py quickfix: fix for additional lint error in updated pylint (#1805) 2024-05-15 17:00:15 -07:00
migration_0021_profile_filenames.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0022_partial_complete.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0023_available_extra_exec_mins.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
migration_0024_crawlerchannel.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0025_workflow_db_configmap_fixes.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00
migration_0026_crawl_review_status.py Change crawl.reviewStatus to 1-5 scale int (#1664) 2024-04-09 17:51:06 -07:00
migration_0027_profile_modified.py Add migration to set profile modified date (#1832) 2024-05-29 15:56:27 -04:00
migration_0028_page_files_errors.py Backend: Move page file and error counts to crawl replay.json endpoint (#1868) 2024-06-20 19:02:57 -07:00
migration_0029_remove_workflow_configmaps.py Remove Crawl Workflow Configmaps (#1894) 2024-06-28 15:25:23 -07:00