5b6aa3bc95
* k8s: add tolerations for 'nodeType=crawling:NoSchedule' to allow scheduling crawling on designated nodes for crawler and profiles jobs and statefulsets * add affinity for 'nodeType=crawling' on crawling and profile browser statefulsets * refactor crawljob: combine crawl_updater logic into base crawl_job * increment new 'crawlAttemptCount' counter crawlconfig when crawl is started, not necessarily finished, to avoid deleting configs that had attempted but not finished crawls. * better external mongodb support: use MONGO_DB_URL to set custom url directly, otherwise build from username, password and mongo host |
||
|---|---|---|
| .. | ||
| backend.yaml | ||
| configmap.yaml | ||
| frontend.yaml | ||
| ingress.yaml | ||
| minio.yaml | ||
| mongo.yaml | ||
| namespaces.yaml | ||
| role.yaml | ||
| secrets.yaml | ||
| signer.yaml | ||