b4fd5e6e94
Fixes #1337 Crawl timeout is tracked via `elapsedCrawlTime` field on the crawl status, which is similar to regular crawl execution time, but only counts one pod if scale > 1. If scale == 1, this time is equivalent. Crawl is gracefully stopped when the elapsed execution time exceeds the timeout. For more responsiveness, also adding current crawl time since last update interval. Details: - handle crawl timeout via elapsed crawl time - longest running time of a single pod, instead of expire time. - include current running from last update for best precision - more accurately count elapsed time crawl is actually running - store elapsedCrawlTime in addition to crawlExecTime, storing the longest duration of each pod since last test interval --------- Co-authored-by: Tessa Walsh <tessa@bitarchivist.net> |
||
|---|---|---|
| .. | ||
| __init__.py | ||
| conftest.py | ||
| test_concurrent_crawl_limit.py | ||
| test_crawl_errors.py | ||
| test_crawl_logs.py | ||
| test_crawl_timeout.py | ||
| test_crawlconfig_crawl_stats.py | ||
| test_delete_crawls.py | ||
| test_execution_minutes_quota.py | ||
| test_invite_expiration.py | ||
| test_max_crawl_size_limit.py | ||
| test_upload_replicas.py | ||
| test_z_background_jobs.py | ||
| utils.py | ||