browsertrix/backend
Ilya Kreymer 88a9f3baf7
ensure running crawl configmap is updated when exclusions are added/removed (#2409)
exclusions are already updated dynamically if crawler pod is running,
but when crawler pod is restarted, this ensures new exclusions are also
picked up:
- mount configmap in separate path, avoiding subPath, to allow dynamic
updates of mounted volume
- adds a lastConfigUpdate timestamp to CrawlJob - if lastConfigUpdate in
spec is different from current, the configmap is recreated by operator
- operator: also update image from channel avoid any issues with
updating crawler in channel
- only updates for exclusion add/remove so far, can later be expanded to
other crawler settings (see: #2355 for broader running crawl config
updates)
- fixes #2408
2025-02-19 11:42:19 -08:00
..
btrixcloud ensure running crawl configmap is updated when exclusions are added/removed (#2409) 2025-02-19 11:42:19 -08:00
test Upload Fixes: (#2397) 2025-02-17 13:05:33 -08:00
test_nightly Add superuser endpoint to get user emails with org info (#2211) 2024-12-09 16:38:01 -08:00
.pylintrc security: tweak get /invite endpoints / InviteOut to: (#2087) 2024-09-20 11:52:56 -07:00
dev-requirements.txt quickfix: pin mypy version to avoid issues with latest release 2024-07-19 18:30:57 -07:00
Dockerfile Pydantic 2.x update + type fixes + python 3.12 (#1947) 2024-07-22 17:23:03 -07:00
mypy.ini Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
requirements.txt switch to simpler streaming download + multiwacz metadata improvements: (#1982) 2024-10-03 16:13:31 -07:00
test-requirements.txt Fix nightly tests: Add boto3 as test requirement (#2116) 2024-10-23 13:41:22 -07:00