tea/browsertrix
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

README + docker-restart.sh add

Browse Source
This commit is contained in:
Ilya Kreymer 2021-08-25 16:27:22 -07:00
parent 9a3356ad0d
commit f1a816be48
2 changed files with 36 additions and 2 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

37
README.md
View File

@ -4,6 +4,7 @@ Browsertrix Cloud is a cloud-native crawling system, which supports a multi-user
The system currently includes support for the following:
- Fully API-driven, with OpenAPI specification for all APIs.
- Multiple users, registered via email and/or invited to join Archives.
- Crawling centered around Archives which are associated with an S3-compatible storage bucket.
- Users may be part of multiple archives and have different roles in different archives
@ -14,9 +15,41 @@ The system currently includes support for the following:
- Crawl status is tracked in the DB (possible crawl states include: Completed, Partially-Complete (due to timeout or cancelation), Cancelation, Failure)
When deployed in Kubernetes, failed crawls are automatically retried. Scheduling is handled via Kubernetes Cronjobs.
## Deploying to Docker
To deploy via local Docker instance, copy the `config.sample.env` to `config.env`.
Docker Compose is required.
Then, run `docker-compose build; docker-compose up -d` to launch.
To update/relaunch, use `./docker-restart.sh`.
The API should be available at: `http://localhost:8000/docs`
Note: When deployed in local Docker, failed crawls are not retried currently. Scheduling is handled by a subprocess, which stores active schedule in the DB.
## Deploying to Kubernetes
To deploy to K8s, `helm` is required. Browsertrix Cloud comes with a helm chart, which can be installed as follows:
`helm install -f ./chart/values.yaml btrix`
This will create a `browsertrix-cloud` service in the default namespace.
For a quick update, the following is recommended:
`helm upgrade -f ./chart/values.yaml btrix ./chart/ --recreate-pods
Note: When deployed in Kubernetes, failed crawls are automatically retried. Scheduling is handled via Kubernetes Cronjobs, and crawl jobs are run in the `crawlers` namespace.
when deployed in local Docker, failed crawls are not retried currently. Scheduling is handled by a subprocess, which stores active schedule in the DB.
Browsertrix Cloud is currently in pre-alpha stages and not ready for production.

1
docker-restart.sh Executable file
View File

@ -0,0 +1 @@
docker-compose build; docker-compose kill; docker-compose rm -f; docker-compose up -d