browsertrix/chart
Ilya Kreymer 887cb16146
Allow configurable max pages per crawl in deployment settings (#717)
* backend: max pages per crawl limit, part of fix for #716:
- set 'max_pages_crawl_limit' in values.yaml, default to 100,000
- if set/non-0, automatically set limit if none provided
- if set/non-0, return 400 if adding config with limit exceeding max limit
- return limit as 'maxPagesPerCrawl' in /api/settings
- api: /all/crawls - add runningOnly=0 to show all crawls, default to 1/true (for more reliable testing)

tests: add test for 'max_pages_per_crawl' setting
- ensure 'limit' can not be set higher than max_pages_per_crawl
- ensure pages crawled is at the limit
- set test limit to max 2 pages
- add settings test
- check for pages.jsonl and extraPages.jsonl when crawling 2 pages
2023-03-28 16:26:29 -07:00
..
admin/logging Add lightweight logging mode (#668) 2023-03-10 14:34:37 -08:00
charts [FIX] Add ingress class for admin logging (#532) 2023-01-31 12:52:57 -08:00
examples ansible: rocky firewall (#635) 2023-02-24 17:28:21 -08:00
templates Allow configurable max pages per crawl in deployment settings (#717) 2023-03-28 16:26:29 -07:00
test Allow configurable max pages per crawl in deployment settings (#717) 2023-03-28 16:26:29 -07:00
.helmignore Frontend + Backend Integrated Deployment (K8s only) (#45) 2021-12-03 10:17:22 -08:00
Chart.lock add logging 2023-01-04 00:15:23 -08:00
Chart.yaml add logging 2023-01-04 00:15:23 -08:00
README.md add logging 2023-01-04 00:15:23 -08:00
values.yaml Allow configurable max pages per crawl in deployment settings (#717) 2023-03-28 16:26:29 -07:00

Update Helm dependencies

  • It needs to update Helm charts after changing its dependencies (e.g. logging)
$ helm dependency update .