8 Commits

Author	SHA1	Message	Date
Ilya Kreymer	60b48ee8a6	dockermanager + scheduler: ... - run as child process using aioprocessing - cleanup: support cleanup of orphaned containers - timeout: support crawlTimeout via check in cleanup loop - support crawl listing + crawl stopping	2021-08-25 15:28:57 -07:00
Ilya Kreymer	b417d7c185	docker manager: support scheduling with apscheduler and separate 'scheduler' process	2021-08-25 12:21:03 -07:00
Ilya Kreymer	91e9fc8699	dockerman: initial pass ... - support for creating, deleting crawlconfigs, running crawls on-demand - config stored in volume - list to docker events and clean up containers when they exit	2021-08-24 22:49:06 -07:00
Ilya Kreymer	66c4e618eb	crawls work (#1), support for: ... - canceling a crawl (via sigterm) - stopping a crawl gracefully (via custom exec sigint)	2021-08-23 12:25:04 -07:00
Ilya Kreymer	4b08163ead	support usage counters per archive, per user -- handle crawl completion	2021-08-20 23:05:42 -07:00
Ilya Kreymer	627e9a6f14	cleanup crawl config, add separate 'runNow' field ... crawler: add cpu/memory limits minio: auto-create bucket for local minio	2021-08-19 14:15:21 -07:00
Ilya Kreymer	eaa87c8b43	support for user roles (owner, crawler, viewer), owner users can issue invites to other existing users by email to join existing archives	2021-08-18 20:35:51 -07:00
Ilya Kreymer	61a608bfbe	update models: ... - replace storages with archives, which have a single storage (for now) - crawls associated with archives - users below to archive, with one admin user (if archive created by default) - update crawlconfig for latest browsertrix-crawler (0.4.4) - k8s: fix permissions for crawler role - k8s: fix minio service (now requiring two ports)	2021-08-18 16:53:49 -07:00