browsertrix/chart
Ilya Kreymer c9c39d47b7
Scheduled Crawl Refactor: Handle via Operator + Add Skipped Crawls on Quota Reached (#1162)
* use metacontroller's decoratorcontroller to create CrawlJob from Job
* scheduled job work:
- use existing job name for scheduled crawljob
- use suspended job, set startTime, completionTime and succeeded status on job when crawljob is done
- simplify cronjob template: remove job_image, cron_namespace, using same namespace as crawls,
placeholder job image for cronjobs

* move storage quota check to crawljob handler:
- add 'skipped_quota_reached' as new failed status type
- check for storage quota before checking if crawljob can be started, fail if not (check before any pods/pvcs created)

* frontend:
- show all crawls in crawl workflow, no need to filter by status
- add 'skipped_quota_reached' status, show as 'Skipped (Quota Reached)', render same as failed

* migration: make release namespace available as DEFAULT_NAMESPACE, delete old cronjobs in DEFAULT_NAMESPACE and recreate in crawlers namespace with new template
2023-09-12 13:05:43 -07:00
..
admin/logging Add lightweight logging mode (#668) 2023-03-10 14:34:37 -08:00
btrix-crds Add max crawl size option to backend and frontend (#1045) 2023-08-26 22:00:37 -07:00
charts Operator refactor to control pods + pvcs directly instead of statefulsets (#1149) 2023-09-11 10:38:04 -07:00
examples publish helm chart on release (fixes #1114) (#1117) (#1123) 2023-08-30 12:02:02 -07:00
templates Scheduled Crawl Refactor: Handle via Operator + Add Skipped Crawls on Quota Reached (#1162) 2023-09-12 13:05:43 -07:00
test Operator refactor to control pods + pvcs directly instead of statefulsets (#1149) 2023-09-11 10:38:04 -07:00
.helmignore Frontend + Backend Integrated Deployment (K8s only) (#45) 2021-12-03 10:17:22 -08:00
Chart.lock Operator refactor to control pods + pvcs directly instead of statefulsets (#1149) 2023-09-11 10:38:04 -07:00
Chart.yaml Operator refactor to control pods + pvcs directly instead of statefulsets (#1149) 2023-09-11 10:38:04 -07:00
README.md Refactor to use new operator on backend (#789) 2023-04-24 18:30:52 -07:00
values.yaml Operator refactor to control pods + pvcs directly instead of statefulsets (#1149) 2023-09-11 10:38:04 -07:00

Update Helm dependencies

  • It needs to update Helm charts after changing its dependencies (e.g. logging)
$ helm dependency update .

Update metacontroller

#!/bin/bash

# intall metacontroller
git clone --depth=1 https://github.com/metacontroller/metacontroller.git
cd metacontroller
helm package deploy/helm/metacontroller --destination deploy/helm
cd ..

# update dependency
helm dependency update
  • Bump up the metacontroller version in Chart.yaml