browsertrix/backend
Tessa Walsh fb80a04f18 Add crawl /log API endpoint
If a crawl is completed, the endpoint streams the logs from the log
files in all of the created WACZ files, sorted by timestamp.

The API endpoint supports filtering by log_level and context whether
the crawl is still running or not.

This is not yet proper streaming because the entire log file is read
into memory before being streamed to the client. We will want to
switch to proper streaming eventually, but are currently blocked by
an aiobotocore bug - see:

https://github.com/aio-libs/aiobotocore/issues/991?#issuecomment-1490737762
2023-04-11 11:51:17 -04:00
..
btrixcloud Add crawl /log API endpoint 2023-04-11 11:51:17 -04:00
test Configure crawler disk utilization threshold via helm chart (#748) 2023-04-05 21:51:53 -07:00
test_nightly Add crawl /log API endpoint 2023-04-11 11:51:17 -04:00
.pylintrc quickfix: pydantic / lint fix (#452) 2023-01-10 18:54:11 -08:00
Dockerfile
requirements.txt Filter and sort crawl and workflow list API endpoints in backend (#724) 2023-03-28 17:55:40 -04:00