tea/browsertrix
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6b19f72a89
browsertrix/backend/btrixcloud
History
Tessa Walsh 6b19f72a89
Add crawl errors endpoint (#757) 
* Add crawl errors endpoint

If this endpoint is called while the crawl is running, errors are
pulled directly from redis.

If this endpoint is called when the crawl is finished, errors are
pulled from mongodb, where they're written when crawls complete.

* Add nightly backend test for errors endpoint

* Add errors for failed and cancelled crawls to mongo

Co-authored-by: Ilya Kreymer <ikreymer@users.noreply.github.com>
2023-04-17 12:59:25 -04:00
..
k8s Add crawl /log API endpoint 2023-04-11 11:51:17 -04:00
migrations Filter and sort crawl and workflow list API endpoints in backend (#724) 2023-03-28 17:55:40 -04:00
__init__.py refactoring to use statefulsets + job (#245) 2022-06-05 10:37:17 -07:00
colls.py Update collections backend API (#759) 2023-04-14 12:17:18 -04:00
crawl_job.py Add crawl errors endpoint (#757) 2023-04-17 12:59:25 -04:00
crawlconfigs.py backend: add 'lastCrawlStartTime' and 'lastStartedByName' fields to crawlconfigs apis (#753) 2023-04-17 08:34:29 -07:00
crawlmanager.py backend: Fix for total crawl time limit. (#665) 2023-03-10 11:43:16 -08:00
crawls.py Add crawl errors endpoint (#757) 2023-04-17 12:59:25 -04:00
db.py Update collections backend API (#759) 2023-04-14 12:17:18 -04:00
emailsender.py Reformat backend for black 23.1.0 (#548) 2023-02-01 20:01:09 -05:00
invites.py Filter and sort crawl and workflow list API endpoints in backend (#724) 2023-03-28 17:55:40 -04:00
main.py Update collections backend API (#759) 2023-04-14 12:17:18 -04:00
orgs.py Add pageSize to pagination format (#736) 2023-04-03 15:57:47 -04:00
pagination.py Add pageSize to pagination format (#736) 2023-04-03 15:57:47 -04:00
profile_job.py VNC-Based Profile Browser (#433) 2023-01-10 14:42:42 -08:00
profiles.py Add pageSize to pagination format (#736) 2023-04-03 15:57:47 -04:00
storages.py Add crawl /log API endpoint 2023-04-11 11:51:17 -04:00
users.py Add pageSize to pagination format (#736) 2023-04-03 15:57:47 -04:00
version.py Upgrade to mongo 6 and use sortArray for workflow crawls (#764) (#765) 2023-04-11 18:22:07 -07:00
worker.py Fix logic for creating pidfile parent dir (#512) 2023-01-23 17:02:25 -08:00
zip.py Add crawl /log API endpoint 2023-04-11 11:51:17 -04:00