b6d1e492d7
- supported in both docker and k8s - additional pods with same job id automatically use same crawl state in redis - support dynamic scaling (#2) via /scale endpoint - k8s job parallelism adjusted dynamically for running job (only supported in k8s so far)
9 lines
175 B
YAML
9 lines
175 B
YAML
apiVersion: v1
|
|
kind: Namespace
|
|
metadata:
|
|
name: {{ .Values.crawler_namespace }}
|
|
labels:
|
|
release: {{ .Release.Name }}
|
|
annotations:
|
|
"helm.sh/resource-policy": keep
|