browsertrix/backend
Ilya Kreymer 702c9ab3b7
Better cacheing of presigned URLs + support for thumbnails (#2446)
Overhauls URL presigning by:
- cache the presigned urls in a flat, separate mongodb collection which
has an expiring index
- update presigned urls if not found / expired automatically in index
- remove logic on storing presignedUrl in files
- support cacheing presigned URL for thumbnails.
- add endpoints to clear presigned urls for org or for all files in all
orgs (superadmin only)
- supersedes #2438, fix for #2437
- removes previous presignedUrl and expireAt data from crawls and QA
runs

---------

Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
2025-03-03 12:05:23 -08:00
..
btrixcloud Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
test Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
test_nightly Add superuser endpoint to get user emails with org info (#2211) 2024-12-09 16:38:01 -08:00
.pylintrc security: tweak get /invite endpoints / InviteOut to: (#2087) 2024-09-20 11:52:56 -07:00
dev-requirements.txt quickfix: pin mypy version to avoid issues with latest release 2024-07-19 18:30:57 -07:00
Dockerfile Pydantic 2.x update + type fixes + python 3.12 (#1947) 2024-07-22 17:23:03 -07:00
mypy.ini
requirements.txt switch to simpler streaming download + multiwacz metadata improvements: (#1982) 2024-10-03 16:13:31 -07:00
test-requirements.txt Fix nightly tests: Add boto3 as test requirement (#2116) 2024-10-23 13:41:22 -07:00