Go to file
Tessa Walsh 0c8c397fca
Add option to fail crawl if not logged in (#2754)
This PR adds a new checkbox to both page and seed crawl workflow types,
which will fail the crawl if behaviors detect the browser is not logged
in for supported sites.

Changes include:

- Backend support for the new crawler flag
- A new `failed_not_logged_in` crawl state
- Checkbox workflow editor and config details in the frontend (currently
in the Scope section - I think it makes sense to have this option up
front, but worth considering)
- User Guide documentation of new option
- A new nightly test for the new workflow option and
`failed_not_logged_in` state


---------
Co-authored-by: Ilya Kreymer <ikreymer@gmail.com>
Co-authored-by: sua yoo <sua@webrecorder.org>
2025-07-28 22:58:43 -07:00
.github docs: Update UI localization docs + fix broken links (#2756) 2025-07-23 15:56:00 -07:00
.vscode devex: Add Storybook for component development (#2556) 2025-04-21 13:06:31 -07:00
ansible Update ansible pipfile (#2088) 2024-09-20 11:41:21 -07:00
assets refactor: Implement brand colors (#2141) 2024-11-12 08:54:11 -08:00
backend Add option to fail crawl if not logged in (#2754) 2025-07-28 22:58:43 -07:00
chart feat: allow specifying custom annotations on ingress (#2725) 2025-07-25 16:06:57 -07:00
configs Remove Code and Configs for Swarm/podman support (#407) 2022-12-08 18:19:58 -08:00
frontend Add option to fail crawl if not logged in (#2754) 2025-07-28 22:58:43 -07:00
scripts Configure browsertrix proxies (#1847) 2024-10-02 18:35:45 -07:00
test fix(build): use /usr/bin/env bash instead of /bin/bash (#1020) 2023-07-28 21:50:04 -07:00
.gitattributes Add linguist-generated attribute to generated files (#2221) 2024-12-07 01:27:50 -05:00
.gitignore Make btrix helper work with microk8s (#768) 2023-04-18 08:50:46 -04:00
.pre-commit-config.yaml Add Repository Index + Chart Rename + Docs Rename (#1708) 2024-04-21 09:42:25 -07:00
btrix Add seed file support to Browsertrix backend (#2710) 2025-07-22 19:11:02 -07:00
LICENSE Add License, Logo and README updates for release (#157) 2022-02-23 12:10:46 -08:00
NOTICE Add Repository Index + Chart Rename + Docs Rename (#1708) 2024-04-21 09:42:25 -07:00
pylintrc misc tweaks: 2021-08-25 18:34:49 -07:00
README.md [Docs:] Add docs for quality assurance (#2769) 2025-07-24 13:10:49 -04:00
update-version.sh style change: remove spaces from python version docstring 2025-02-17 16:52:49 -08:00
version.txt version: bump to 1.18.0 2025-07-22 21:21:24 -07:00
yarn.lock docs: add public collections gallery howto (#2462) 2025-03-08 15:57:13 -08:00

Browsertrix

 

Browsertrix is a cloud-native, high-fidelity, browser-based crawling service designed to make web archiving easier and more accessible for everyone.

The service provides an API and UI for starting, scheduling, sharing, and managing crawls. This repo includes the orchestration and management tools around crawling, while the actual crawling is performed using browsertrix-crawler containers.

Install Browsertrix to self host, or access hosted Browsertrix from app.browsertrix.com.

See webrecorder.net/browsertrix for a full overview of features.

Documentation

Documentation for using, deploying, and developing Browsertrix is available at docs.browsertrix.com.

See instructions in frontend/docs to run the docs locally.

Installation

Instructions for setting up Browsertrix is available at docs.browsertrix.com/deploy.

The docs cover deploying Browsertrix in different environments using Kubernetes, from a single-node setup to scalable clusters in the cloud.

Support

Ask us your questions about Browsertrix and web archiving in the community help forum.

Dedicated professional support is available with a custom subscription. For details, see hosted Browsertrix plans.

Bugs

For bug reports or feature requests, please open a GitHub issue.

Changelog

See release notes.

Contributing

Developing

Guides for getting started with local development are available at docs.browsertrix.com/develop.

See Code of Conduct.

Translating

Translations are managed through Weblate, a web-based and open source translation tool. View translations in progress and register to contribute at our Weblate project.

Translation status

License

Browsertrix is made available under the AGPLv3 License.

Documentation is made available under the Creative Commons Attribution 4.0 International License.