browsertrix/backend/test
Ilya Kreymer 702c9ab3b7
Better cacheing of presigned URLs + support for thumbnails (#2446)
Overhauls URL presigning by:
- cache the presigned urls in a flat, separate mongodb collection which
has an expiring index
- update presigned urls if not found / expired automatically in index
- remove logic on storing presignedUrl in files
- support cacheing presigned URL for thumbnails.
- add endpoints to clear presigned urls for org or for all files in all
orgs (superadmin only)
- supersedes #2438, fix for #2437
- removes previous presignedUrl and expireAt data from crawls and QA
runs

---------

Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
2025-03-03 12:05:23 -08:00
..
data Backend work for public collections (#2198) 2025-01-13 15:15:48 -08:00
__init__.py
conftest.py Add collection page list/search endpoint (#2354) 2025-02-10 16:44:37 -08:00
echo_server.py
test_api.py quickfix: update test_api.py to match all locales enabled by default (#2241) 2024-12-13 20:30:06 -08:00
test_collections.py Fix replay.json 400 response for empty collection (#2445) 2025-03-03 09:38:19 -08:00
test_crawl_config_search_values.py
test_crawl_config_tags.py
test_crawlconfigs.py Add support for custom link selectors to backend (#2346) 2025-02-13 22:22:27 -08:00
test_filter_sort_results.py
test_login.py
test_org_subs.py security: tweak get /invite endpoints / InviteOut to: (#2087) 2024-09-20 11:52:56 -07:00
test_org.py Rework crawl page migration + MongoDB Query Optimizations (#2412) 2025-02-20 15:26:11 -08:00
test_permissions.py
test_profiles.py Serialize datetimes with Z suffix (#2058) 2024-09-12 16:16:13 -07:00
test_qa.py Calculate total for crawl QA page endpoint (#2435) 2025-02-27 11:46:35 -08:00
test_run_crawl.py Consolidate list page endpoints + better QA sorting + optimize pages fix (#2417) 2025-02-21 13:47:20 -08:00
test_stop_cancel_crawl.py
test_uploads.py Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
test_users.py Add superuser endpoint to get user emails with org info (#2211) 2024-12-09 16:38:01 -08:00
test_utils.py
test_webhooks.py Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
test_workflow_auto_add_to_collection.py
test_y_org_import_export.py
test_z_delete_org.py Move org storage recalculation into background job (#2138) 2024-11-19 17:32:57 -05:00
utils.py