Commit Graph

152 Commits

Author SHA1 Message Date
Ilya Kreymer
f77eaccf41 support committing to s3 storage
move mongo into separate optional deployment along with minio
support for configuring storages
support for deleting crawls, associated config and secrets
2021-07-02 15:56:24 -07:00
Ilya Kreymer
a111bacfb5 add k8s support
- working apis for adding crawls, removing crawls in mongo, mapped to k8s cronjobs
- more complete crawl spec
- option to start on-demand job from cronjobs
- optional minio in separate deployment/service
2021-06-30 21:48:44 -07:00