browsertrix/backend/test
Ilya Kreymer cb50c7c2c2
Pause / Resume Crawls Initial Implmentation. (#2572)
- add 'pause' crawl state (fixes #2567)
- gracefully shut down crawler pods, and then redis pod when paused
- crawler uploads WACZ before shutting down (dependent on
webrecorder/browsertrix-crawler#824, supported in 1.6.1+)
- add 'paused_at' on crawl spec to indicate when crawl is paused
- support max pause time limit, after which crawl becomes automatically
stopped.
- add 'stopped_pause_expired' when pause automatically expires and crawl
is stopped
- /crawl/<id>/{pause,resume} apis to toggle 'paused' on crawl spec
- ui: add pause/resume button, paused state (partially addresses #2568)
- ui: add pausing/resuming derivative states when crawl is running and
pausing, or paused and not pausing (partially addresses #2569)
- Designed to work with crawler 1.6.1+ which support pausing + uploading on pause

Work on #2566, Fixes #2576 

---------
Co-authored-by: sua yoo <sua@webrecorder.org>
Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
Co-authored-by: sua yoo <sua@suayoo.com>
2025-05-21 14:05:16 -07:00
..
data Backend work for public collections (#2198) 2025-01-13 15:15:48 -08:00
__init__.py
conftest.py Add behavior logs from Redis to database and add endpoint to serve (#2526) 2025-04-08 02:16:10 +02:00
echo_server.py
test_api.py Pause / Resume Crawls Initial Implmentation. (#2572) 2025-05-21 14:05:16 -07:00
test_collections.py remove deleted collections from crawlconfigs (#2615) 2025-05-20 18:38:40 -07:00
test_crawl_config_search_values.py
test_crawl_config_tags.py
test_crawlconfigs.py Add ISO-639-1 language code validation to backend (#2602) 2025-05-13 16:54:33 -04:00
test_filter_sort_results.py Add backend support for custom behaviors + validation endpoint (#2505) 2025-04-02 16:20:51 -07:00
test_login.py
test_org_subs.py Add API endpoint to check if subscription is activated (#2582) 2025-05-06 17:36:58 -07:00
test_org.py Add ISO-639-1 language code validation to backend (#2602) 2025-05-13 16:54:33 -04:00
test_permissions.py
test_profiles.py Serialize datetimes with Z suffix (#2058) 2024-09-12 16:16:13 -07:00
test_qa.py Calculate total for crawl QA page endpoint (#2435) 2025-02-27 11:46:35 -08:00
test_run_crawl.py Ensure crawl page counts are correct when re-adding pages (#2601) 2025-05-13 14:05:41 -04:00
test_stop_cancel_crawl.py
test_uploads.py Add behavior logs from Redis to database and add endpoint to serve (#2526) 2025-04-08 02:16:10 +02:00
test_users.py Fix user emails use userout (#2511) 2025-03-24 12:04:39 -07:00
test_utils.py
test_webhooks.py Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
test_workflow_auto_add_to_collection.py remove deleted collections from crawlconfigs (#2615) 2025-05-20 18:38:40 -07:00
test_y_org_import_export.py
test_z_delete_org.py Move org storage recalculation into background job (#2138) 2024-11-19 17:32:57 -05:00
utils.py