browsertrix/backend/btrixcloud
Ilya Kreymer 12f7db3ae2
tests: fixes for crawl cancel + crawl stopped (#864)
* tests:
- fix cancel crawl test by ensuring state is not running or waiting
- fix stop crawl test by ensuring stop is only initiated after at least one page has been crawled,
otherwise result may be failed, as no crawl data has been crawled yet (separate fix in crawler to avoid loop if stopped
before any data written webrecorder/browsertrix-crawler#314)
- bump page limit to 4 for tests to ensure crawl is partially complete, not fully complete when stopping
- allow canceled or partial_complete due to race condition

* chart: bump frontend limits in default, not just for tests (addresses #780)

* crawl stop before starting:
- if crawl stopped before it started, mark as canceled
- add test for stopping immediately, which should result in 'canceled' crawl
- attempt to increase resync interval for immediate failure
- nightly tests: increase page limit to test timeout

* backend:
- detect stopped-before-start crawl as 'failed' instead of 'done'
- stats: return stats counters as int instead of string
2023-05-22 20:17:29 -07:00
..
migrations backend: fixes to 0005 migration: (#843) 2023-05-10 12:00:41 +02:00
templates crawlconfig: fix default filename template, make configurable (#835) 2023-05-08 14:03:27 -07:00
__init__.py
colls.py Add collection sorting and filtering (#863) 2023-05-22 16:53:49 -04:00
crawlconfigs.py Add crawlSuccessfulCount to workflows (#871) 2023-05-22 19:06:37 -04:00
crawlmanager.py crawlconfig: fix default filename template, make configurable (#835) 2023-05-08 14:03:27 -07:00
crawls.py tests: fixes for crawl cancel + crawl stopped (#864) 2023-05-22 20:17:29 -07:00
db.py Pre-compute workflow last crawl information (#812) 2023-05-05 15:12:52 -07:00
emailsender.py Reformat backend for black 23.1.0 (#548) 2023-02-01 20:01:09 -05:00
invites.py Filter and sort crawl and workflow list API endpoints in backend (#724) 2023-03-28 17:55:40 -04:00
k8sapi.py Refactor to use new operator on backend (#789) 2023-04-24 18:30:52 -07:00
main_op.py startup fixes: (#793) 2023-04-24 18:32:52 -07:00
main_scheduled_job.py Improve sorting workflows by lastUpdated (#826) 2023-05-22 18:42:30 -04:00
main.py Use collection uuid as id (instead of name) (#855) 2023-05-19 09:03:48 -04:00
operator.py tests: fixes for crawl cancel + crawl stopped (#864) 2023-05-22 20:17:29 -07:00
orgs.py Refactor to use new operator on backend (#789) 2023-04-24 18:30:52 -07:00
pagination.py Add pageSize to pagination format (#736) 2023-04-03 15:57:47 -04:00
profiles.py Add pageSize to pagination format (#736) 2023-04-03 15:57:47 -04:00
storages.py Add crawl /log API endpoint 2023-04-11 11:51:17 -04:00
users.py startup fixes: (#793) 2023-04-24 18:32:52 -07:00
utils.py stopping fix: backend fixes for #836 + prep for additional status fields (#837) 2023-05-08 14:02:20 -07:00
version.py version: bump to 1.6.0-beta.0 2023-05-19 11:29:31 -07:00
zip.py Add crawl /log API endpoint 2023-04-11 11:51:17 -04:00