tea/browsertrix
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
702c9ab3b7
browsertrix/backend/btrixcloud
History
Ilya Kreymer 702c9ab3b7
Better cacheing of presigned URLs + support for thumbnails (#2446) 
Overhauls URL presigning by:
- cache the presigned urls in a flat, separate mongodb collection which
has an expiring index
- update presigned urls if not found / expired automatically in index
- remove logic on storing presignedUrl in files
- support cacheing presigned URL for thumbnails.
- add endpoints to clear presigned urls for org or for all files in all
orgs (superadmin only)
- supersedes #2438, fix for #2437
- removes previous presignedUrl and expireAt data from crawls and QA
runs

---------

Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
2025-03-03 12:05:23 -08:00
..
migrations Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
operator Consolidate list page endpoints + better QA sorting + optimize pages fix (#2417) 2025-02-21 13:47:20 -08:00
__init__.py
auth.py Reformat with Black for 2025 ruleset (#2349) 2025-01-29 16:57:06 -05:00
background_jobs.py Rework crawl page migration + MongoDB Query Optimizations (#2412) 2025-02-20 15:26:11 -08:00
basecrawls.py Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
colls.py optimize public collection loading: (#2444) 2025-03-03 10:13:30 -08:00
crawlconfigs.py Add superadmin endpoint to readd scheduled workflow cronjobs (#2430) 2025-02-26 23:13:53 -08:00
crawlmanager.py Consolidate list page endpoints + better QA sorting + optimize pages fix (#2417) 2025-02-21 13:47:20 -08:00
crawls.py Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
db.py Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
emailsender.py Rework crawl page migration + MongoDB Query Optimizations (#2412) 2025-02-20 15:26:11 -08:00
invites.py Reformat with Black for 2025 ruleset (#2349) 2025-01-29 16:57:06 -05:00
k8sapi.py Reformat with Black for 2025 ruleset (#2349) 2025-01-29 16:57:06 -05:00
main_bg.py Rework crawl page migration + MongoDB Query Optimizations (#2412) 2025-02-20 15:26:11 -08:00
main_op.py Reformat with Black for 2025 ruleset (#2349) 2025-01-29 16:57:06 -05:00
main.py Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
models.py Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
ops.py Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
orgs.py Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
pages.py Fix replay.json 400 response for empty collection (#2445) 2025-03-03 09:38:19 -08:00
pagination.py
profiles.py Reformat with Black for 2025 ruleset (#2349) 2025-01-29 16:57:06 -05:00
storages.py Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00
subs.py Send subscription cancelation email (#2234) 2024-12-12 11:52:38 -08:00
uploads.py Rework crawl page migration + MongoDB Query Optimizations (#2412) 2025-02-20 15:26:11 -08:00
users.py Add superuser endpoint to get user emails with org info (#2211) 2024-12-09 16:38:01 -08:00
utils.py Reformat with Black for 2025 ruleset (#2349) 2025-01-29 16:57:06 -05:00
version.py version: bump to 1.14.2 2025-02-27 14:13:03 -08:00
webhooks.py Better cacheing of presigned URLs + support for thumbnails (#2446) 2025-03-03 12:05:23 -08:00