browsertrix/backend/btrixcloud
Tessa Walsh fb80a04f18 Add crawl /log API endpoint
If a crawl is completed, the endpoint streams the logs from the log
files in all of the created WACZ files, sorted by timestamp.

The API endpoint supports filtering by log_level and context whether
the crawl is still running or not.

This is not yet proper streaming because the entire log file is read
into memory before being streamed to the client. We will want to
switch to proper streaming eventually, but are currently blocked by
an aiobotocore bug - see:

https://github.com/aio-libs/aiobotocore/issues/991?#issuecomment-1490737762
2023-04-11 11:51:17 -04:00
..
k8s Add crawl /log API endpoint 2023-04-11 11:51:17 -04:00
migrations Filter and sort crawl and workflow list API endpoints in backend (#724) 2023-03-28 17:55:40 -04:00
__init__.py
colls.py Add pageSize to pagination format (#736) 2023-04-03 15:57:47 -04:00
crawl_job.py backend: Fix for total crawl time limit. (#665) 2023-03-10 11:43:16 -08:00
crawlconfigs.py config: add 'pageLoadTimeout' and 'pageExtraDelay' options to backend… (#742) 2023-04-04 19:52:23 -07:00
crawlmanager.py backend: Fix for total crawl time limit. (#665) 2023-03-10 11:43:16 -08:00
crawls.py Add crawl /log API endpoint 2023-04-11 11:51:17 -04:00
db.py Make pending invites expire via TTL index (#568) 2023-02-14 16:07:14 -05:00
emailsender.py Reformat backend for black 23.1.0 (#548) 2023-02-01 20:01:09 -05:00
invites.py Filter and sort crawl and workflow list API endpoints in backend (#724) 2023-03-28 17:55:40 -04:00
main.py config: add 'pageLoadTimeout' and 'pageExtraDelay' options to backend… (#742) 2023-04-04 19:52:23 -07:00
orgs.py Add pageSize to pagination format (#736) 2023-04-03 15:57:47 -04:00
pagination.py Add pageSize to pagination format (#736) 2023-04-03 15:57:47 -04:00
profile_job.py
profiles.py Add pageSize to pagination format (#736) 2023-04-03 15:57:47 -04:00
storages.py Add crawl /log API endpoint 2023-04-11 11:51:17 -04:00
users.py Add pageSize to pagination format (#736) 2023-04-03 15:57:47 -04:00
version.py version: bump to 1.4.0! 2023-04-06 10:12:43 -07:00
worker.py Fix logic for creating pidfile parent dir (#512) 2023-01-23 17:02:25 -08:00
zip.py Add crawl /log API endpoint 2023-04-11 11:51:17 -04:00