browsertrix/backend/btrixcloud
Tessa Walsh 032859f361
Support multiple crawler versions (#1420)
Fixes #1385 

## Changes
Supports multiple crawler 'channels' which can be configured to
different browsertrix-crawler versions
- Replaces `crawler_image` in helm chart with `crawler_channels` array
similar to how storages are handled
- The `default` crawler channel must always be provided and specifies
the default crawler image
- Adds backend `/orgs/{oid}/crawlconfigs/crawler-channels` API endpoint
to fetch information about available crawler versions (name, image, and
label) and test
- Adds crawler channel select to workflow creation/edit screens and
profile creation dialog, and updates related API endpoints and
configmaps accordingly. The select dropdown is shown only if more than
one channel is configured.
- Adds `crawlerChannel` to workflow and crawl details.
- Add `image` to crawler image, used to display actual image used as
part of the crawl.
- Modifies `crawler_crawl_id` backend test fixture to use `test` crawler
version to ensure crawler versions other than latest work
- Adds migration to add `crawlerChannel` set to `default` to existing
workflow and profile objects and workflow configmaps

---------
Co-authored-by: Ilya Kreymer <ikreymer@gmail.com>
Co-authored-by: Henry Wilkinson <henry@wilkinson.graphics>
2024-01-16 15:32:12 -08:00
..
migrations Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
__init__.py
auth.py
background_jobs.py Add API endpoint to retry all failed bg jobs (#1396) 2023-12-05 13:00:45 -08:00
basecrawls.py Add crawl, upload, and collection delete webhook event notifications (#1363) 2023-11-09 18:19:08 -08:00
colls.py Add crawl, upload, and collection delete webhook event notifications (#1363) 2023-11-09 18:19:08 -08:00
crawlconfigs.py Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
crawlmanager.py Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
crawls.py Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
db.py Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
emailsender.py Email Templates (#1375) 2023-11-15 15:22:12 -08:00
invites.py Email Templates (#1375) 2023-11-15 15:22:12 -08:00
k8sapi.py Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
main_op.py Send email to superuser when background job fails (#1355) 2023-11-08 19:55:59 -08:00
main.py Add API endpoint to retry all failed bg jobs (#1396) 2023-12-05 13:00:45 -08:00
models.py Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
operator.py Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
orgs.py Add API endpoints for crawl statistics (#1461) 2024-01-10 13:30:47 -08:00
pagination.py
profiles.py Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
storages.py charts cleanup: (#1360) 2023-11-08 19:24:00 -08:00
uploads.py
users.py Add API endpoints for crawl statistics (#1461) 2024-01-10 13:30:47 -08:00
utils.py Add API endpoints for crawl statistics (#1461) 2024-01-10 13:30:47 -08:00
version.py version: bump to 1.9.0-beta.0 2023-12-20 00:08:16 -08:00
webhooks.py Add crawl, upload, and collection delete webhook event notifications (#1363) 2023-11-09 18:19:08 -08:00
zip.py