Go to file
Tessa Walsh be41c48c27
Add extra and gifted execution minutes (#1361)
Fixes #1358 

- Adds `extraExecMinutes` and `giftedExecMinutes` org quotas, which are
not reset monthly but are updateable amounts that carry across months
- Adds `quotaUpdate` field to `Organization` to track when quotas were
updated with timestamp
- Adds `extraExecMinutesAvailable` and `giftedExecMinutesAvailable`
fields to `Organization` to help with tracking available time left
(includes tested migration to initialize these to 0)
- Modifies org backend to track time across multiple categories, using
monthlyExecSeconds, then giftedExecSeconds, then extraExecSeconds.
All time is also written into crawlExecSeconds, which is now the monthly
total and also contains any overage time above the quotas
- Updates Dashboard crawling meter to include all types of execution
time if `extraExecMinutes` and/or `giftedExecMinutes` are set above 0
- Updates Dashboard Usage History table to include all types of
execution time (only displaying columns that have data)
- Adds backend nightly test to check handling of quotas and execution
time
- Includes migration to add new fields and copy crawlExecSeconds to
monthlyExecSeconds for previous months

Co-authored-by: emma <hi@emma.cafe>
2023-12-07 14:34:37 -05:00
.github Adds a link to the new ReplayWebpage replay issue form in GitHub (#1404) 2023-12-05 14:28:50 -08:00
.vscode Organize components into folders by function (#1411) 2023-11-29 14:12:29 -05:00
ansible
backend Add extra and gifted execution minutes (#1361) 2023-12-07 14:34:37 -05:00
chart
configs
docs
frontend Add extra and gifted execution minutes (#1361) 2023-12-07 14:34:37 -05:00
scripts
test
.gitignore
.pre-commit-config.yaml
btrix
CHANGES.md
LICENSE
mkdocs.yml
NOTICE
pylintrc
README.md
update-version.sh
version.txt
yarn.lock

Browsertrix Cloud

Browsertrix Cloud is an open-source cloud-native high-fidelity browser-based crawling service designed to make web archiving easier and more accessible for everyone.

The service provides an API and UI for scheduling crawls and viewing results, and managing all aspects of crawling process. This system provides the orchestration and management around crawling, while the actual crawling is performed using Browsertrix Crawler containers, which are launched for each crawl.

See Browsertrix Cloud for a feature overview and information about Browsertrix Cloud Hosting.

Documentation

The full docs for using, deploying and developing Browsertrix Cloud are available at: https://docs.browsertrix.cloud

Deployment

The latest deployment documentation is available at: https://docs.browsertrix.cloud/deploy

The docs cover deploying Browsertrix Cloud in different environments using Kubernetes, from a single-node setup to scalable clusters in the cloud.

Previously, Browsertrix Cloud also supported Docker Compose and podman-based deployment. This is now deprecated due to the complexity of maintaining feature parity across different setups, and with various Kubernetes deployment options being available and easy to deploy, even on a single machine.

Making deployment of Browsertrix Cloud as easy as possible remains a key goal, and we welcome suggestions for how we can further improve our Kubernetes deployment options.

If you are looking to just try running a single crawl, you may want to try Browsertrix Crawler first to test out the crawling capabilities.

Development Status

Browsertrix Cloud is currently in a beta, though the system and backend API is fairly stable, we are working on many additional features.

Additional developer documentation is available at https://docs.browsertrix.cloud/develop

Please see the GitHub issues and this GitHub Project for our current project plan and tasks.

License

Browsertrix Cloud is made available under the AGPLv3 License.

Documentation is made available under the Creative Commons Attribution 4.0 International License