browsertrix/backend
Ilya Kreymer fa6627ce70
ensure QA configmap is updated for long running QA runs: (#1865)
- add a 'expire_at_duration_seconds' which is 75% of actual presign
duration time, or <25% remaining until presigned URL actually expires to
ensure presigned URLs are updated early than when they actually expire
- set cached expireAt time to the renew at time for more frequent
updates
- update QA configmap in place with updated presigned URLs when expireAt
time is reached
- mount qa config volume under /tmp/qa/ without subPath to get automatic
updates, which crawler will handle
- tests: fix qa test typo (from main)
- fixes #1864
2024-06-12 10:51:35 -07:00
..
btrixcloud ensure QA configmap is updated for long running QA runs: (#1865) 2024-06-12 10:51:35 -07:00
test ensure QA configmap is updated for long running QA runs: (#1865) 2024-06-12 10:51:35 -07:00
test_nightly Give test_crawl_timeout 10 mins to finish (#1627) 2024-03-26 18:33:30 -07:00
.pylintrc quickfix: pydantic / lint fix (#452) 2023-01-10 18:54:11 -08:00
Dockerfile Backend mem usage fix - use fixed MOTOR_MAX_WORKERS + switch to gunicorn (#1468) 2024-01-16 15:32:42 -08:00
mypy.ini Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
requirements.txt Add endpoints to read pages from older crawl WACZs into database (#1562) 2024-03-19 14:14:21 -07:00
test-requirements.txt Add slugs to org backend (#1250) 2023-10-10 18:30:09 -07:00