82ffc0dfbc
* k8s local deployment work: - make it easier to deploy w/o ingress by setting 'local_service_port' (suggested port 30870) - if using local minio, ensure file endpoints set to /data/ and /data/ proxies correctly to local bucket - if not using minio, ensure file endpoints point to correct access / endpoint url. - setup should work with docker desktop, minikube, microk8s and k3s! - nginx chart: bump nginx memory limit to 20Mi - nginx image: 00-default-override-resolver-config -> 00-browsertrix-nginx-init for clarity - nginx image: use default nginx.conf, pin to nginx 1.23.2 - mongo: readd readiness probe, bump connect wait timeout (needed for ci) - config: set superadmin username to 'admin' - config schema: set 'name' as required - add sample chart values overrides: - chart values: local-config.yaml for running locally with 'local_service_port' - chart values: add microk8s-hosted.yaml for configuring a hosted microk8s setup - chart values: add microk8s-ci.yaml for ci tests - ci: remove docker swarm tests - ci: add microk8s integration tests: launching cluster, logging in, running a crawl of example.com, downloading/checking WACZ - bump to 1.1.0-beta.2 |
||
|---|---|---|
| .github/workflows | ||
| ansible | ||
| backend | ||
| chart | ||
| configs | ||
| frontend | ||
| scripts | ||
| test | ||
| .gitignore | ||
| Deployment.md | ||
| docker-compose.yml | ||
| LICENSE | ||
| NOTICE | ||
| pylintrc | ||
| README.md | ||
| update-version.sh | ||
| version.txt | ||
Browsertrix Cloud
Browsertrix Cloud is an open-source cloud-native high-fidelity browser-based crawling service designed to make web archiving easier and more accessible for everyone.
The service provides an API and UI for scheduling crawls and viewing results, and managing all aspects of crawling process. This system provides the orchestration and management around crawling, while the actual crawling is performed using Browsertrix Crawler containers, which are launched for each crawl.
The system is designed to run in both Kubernetes and Docker Swarm, as well as locally under Podman.
See Features for a high-level list of planned features.
Deployment
See the Deployment page for information on how to deploy Browsertrix Cloud.
Development Status
Browsertrix Cloud is currently in an alpha stage and not ready for production. This is an ambitious project and there's a lot to be done!
If you would like to help in a particular way, please open an issue or reach out to us in other ways.
License
Browsertrix Cloud is made available under the AGPLv3 License.
If you would like to use it under a different license or have a question, please reach out as that may be a possibility.