Ilya Kreymer b6d1e492d7 add redis for storing crawl state data! ... - supported in both docker and k8s - additional pods with same job id automatically use same crawl state in redis - support dynamic scaling (#2) via /scale endpoint - k8s job parallelism adjusted dynamically for running job (only supported in k8s so far)		2021-09-17 15:02:11 -07:00
..
ingress.yaml	add k8s support	2021-06-30 21:48:44 -07:00
main.yaml	add redis for storing crawl state data!	2021-09-17 15:02:11 -07:00
minio.yaml	crawls work (#1):	2021-08-23 18:01:29 -07:00
mongo.yaml	crawls work (#1):	2021-08-23 18:01:29 -07:00
namespaces.yaml	add redis for storing crawl state data!	2021-09-17 15:02:11 -07:00
role.yaml	update models:	2021-08-18 16:53:49 -07:00