tea/browsertrix
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
0935d43a97
browsertrix/backend/btrixcloud
History
Ilya Kreymer 0935d43a97
exclusion optimizations: dynamic exclusions (part of #1216): (#1268) 
- instead of restarting crawler when exclusion added/removed, add a
message to a redis list (per crawler instance)
- no longer filtering existing queue on backend, now handled via crawler (implemented in 0.12.0 via webrecorder/browsertrix-crawler#408)
- match response optimization: instead of returning first 1000 matches,
limits response to 500K and returns however many matches fit in that
response size (for optional pagination on frontend)
2023-11-06 09:36:25 -08:00
..
migrations Storage Refactor: Replication + Custom Storage Support (#1296) 2023-10-26 21:44:09 -07:00
templates Background Jobs Work (#1321) 2023-11-02 13:02:17 -07:00
__init__.py
auth.py Additional Type Hints / Type Fix Pass (#1320) 2023-10-30 12:59:24 -04:00
background_jobs.py Background Jobs Work (#1321) 2023-11-02 13:02:17 -07:00
basecrawls.py Background Jobs Work (#1321) 2023-11-02 13:02:17 -07:00
colls.py Additional Type Hints / Type Fix Pass (#1320) 2023-10-30 12:59:24 -04:00
crawlconfigs.py Additional Type Hints / Type Fix Pass (#1320) 2023-10-30 12:59:24 -04:00
crawlmanager.py Background Jobs Work (#1321) 2023-11-02 13:02:17 -07:00
crawls.py exclusion optimizations: dynamic exclusions (part of #1216): (#1268) 2023-11-06 09:36:25 -08:00
db.py Additional Type Hints / Type Fix Pass (#1320) 2023-10-30 12:59:24 -04:00
emailsender.py Additional Type Hints / Type Fix Pass (#1320) 2023-10-30 12:59:24 -04:00
invites.py Additional Type Hints / Type Fix Pass (#1320) 2023-10-30 12:59:24 -04:00
k8sapi.py Storage Refactor: Replication + Custom Storage Support (#1296) 2023-10-26 21:44:09 -07:00
main_op.py Background Jobs Work (#1321) 2023-11-02 13:02:17 -07:00
main.py Background Jobs Work (#1321) 2023-11-02 13:02:17 -07:00
models.py Background Jobs Work (#1321) 2023-11-02 13:02:17 -07:00
operator.py Background Jobs Work (#1321) 2023-11-02 13:02:17 -07:00
orgs.py Additional Type Hints / Type Fix Pass (#1320) 2023-10-30 12:59:24 -04:00
pagination.py
profiles.py Background Jobs Work (#1321) 2023-11-02 13:02:17 -07:00
storages.py Additional Type Hints / Type Fix Pass (#1320) 2023-10-30 12:59:24 -04:00
uploads.py Background Jobs Work (#1321) 2023-11-02 13:02:17 -07:00
users.py Additional Type Hints / Type Fix Pass (#1320) 2023-10-30 12:59:24 -04:00
utils.py Add slugs to org backend (#1250) 2023-10-10 18:30:09 -07:00
version.py version: bump to 1.8.0-beta.1 2023-10-27 14:35:24 -07:00
webhooks.py Additional Type Hints / Type Fix Pass (#1320) 2023-10-30 12:59:24 -04:00
zip.py Fix: Stream log downloading from WACZ (#1225) 2023-09-28 18:54:52 -07:00