browsertrix/backend
Ilya Kreymer 0935d43a97
exclusion optimizations: dynamic exclusions (part of #1216): (#1268)
- instead of restarting crawler when exclusion added/removed, add a
message to a redis list (per crawler instance)
- no longer filtering existing queue on backend, now handled via crawler (implemented in 0.12.0 via webrecorder/browsertrix-crawler#408)
- match response optimization: instead of returning first 1000 matches,
limits response to 500K and returns however many matches fit in that
response size (for optional pagination on frontend)
2023-11-06 09:36:25 -08:00
..
btrixcloud exclusion optimizations: dynamic exclusions (part of #1216): (#1268) 2023-11-06 09:36:25 -08:00
test users: add case-insensitive index to maintain backwards compatibility with fastapi-users (#1319) 2023-10-27 14:31:29 -07:00
test_nightly Background Jobs Work (#1321) 2023-11-02 13:02:17 -07:00
.pylintrc
Dockerfile
mypy.ini Improved type checking for backend with mypy (#1174) 2023-09-13 19:40:26 -07:00
requirements.txt Storage Refactor: Replication + Custom Storage Support (#1296) 2023-10-26 21:44:09 -07:00
test-requirements.txt Add slugs to org backend (#1250) 2023-10-10 18:30:09 -07:00