Commit Graph

4 Commits

Author SHA1 Message Date
Ilya Kreymer
20b19f932f make crawlTimeout a per-crawconfig property
allow crawl complete/partial complete to update existing crawl state, eg. timeout
enable handling backofflimitexceeded / deadlineexceeded failure, with possible success able to override the failure state
filter out only active jobs in running crawls listing
2021-08-24 11:29:15 -07:00
Ilya Kreymer
66c4e618eb crawls work (#1), support for:
- canceling a crawl (via sigterm)
- stopping a crawl gracefully (via custom exec sigint)
2021-08-23 12:25:04 -07:00
Ilya Kreymer
a8255a76b2 crawljob:
- support run once on existing crawl job
- support updating/patching existing crawl job with new crawl config, new schedule and run once
2021-08-21 22:10:31 -07:00
Ilya Kreymer
170958be37 rename crawls -> crawlconfigs.py
add crawls for crawl api management
2021-08-20 15:15:51 -07:00