bb6e703f6a
Resolves #1354 Supports crawling through pre-configured proxy servers, allowing users to select which proxy servers to use (requires browsertrix crawler 1.3+) Config: - proxies defined in btrix-proxies subchart - can be configured via btrix-proxies key or separate proxies.yaml file via separate subchart - proxies list refreshed automatically if crawler_proxies.json changes if subchart is deployed - support for ssh and socks5 proxies - proxy keys added to secrets in subchart - support for default proxy to be always used if no other proxy configured, prevent starting cluster if default proxy not available - prevent starting manual crawl if previously configured proxy is no longer available, return error - force 'btrix' username and group name on browsertrix-crawler non-root user to support ssh Operator: - support crawling through proxies, pass proxyId in CrawlJob - support running profile browsers which designated proxy, pass proxyId to ProfileJob - prevent starting scheduled crawl if previously configured proxy is no longer available API / Access: - /api/orgs/all/crawlconfigs/crawler-proxies - get all proxies (superadmin only) - /api/orgs/{oid}/crawlconfigs/crawler-proxies - get proxies available to particular org - /api/orgs/{oid}/proxies - update allowed proxies for particular org (superadmin only) - superadmin can configure which orgs can use which proxies, stored on the org - superadmin can also allow an org to access all 'shared' proxies, to avoid having to allow a shared proxy on each org. UI: - Superadmin has 'Edit Proxies' dialog to configure for each org if it has: dedicated proxies, has access to shared proxies. - User can select a proxy in Crawl Workflow browser settings - Users can choose to launch a browser profile with a particular proxy - Display which proxy is used to create profile in profile selector - Users can choose with default proxy to use for new workflows in Crawling Defaults --------- Co-authored-by: Ilya Kreymer <ikreymer@gmail.com> Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
128 lines
3.4 KiB
YAML
128 lines
3.4 KiB
YAML
site_name: Browsertrix Docs
|
|
repo_url: https://github.com/webrecorder/browsertrix-cloud/
|
|
repo_name: Browsertrix
|
|
edit_uri: edit/main/docs/
|
|
extra_css:
|
|
- stylesheets/extra.css
|
|
extra_javascript:
|
|
- js/insertversion.js
|
|
theme:
|
|
name: material
|
|
custom_dir: docs/overrides
|
|
features:
|
|
- navigation.tabs
|
|
- navigation.tabs.sticky
|
|
- navigation.instant
|
|
- navigation.tracking
|
|
- navigation.footer
|
|
- content.code.copy
|
|
- content.action.edit
|
|
- content.tooltips
|
|
- search.suggest
|
|
palette:
|
|
scheme: webrecorder
|
|
logo: assets/brand/browsertrix-icon-white.svg
|
|
favicon: assets/brand/favicon.svg
|
|
|
|
icon:
|
|
admonition:
|
|
note: bootstrap/pencil-fill
|
|
abstract: bootstrap/file-earmark-text-fill
|
|
info: bootstrap/info-circle-fill
|
|
tip: bootstrap/exclamation-circle-fill
|
|
success: bootstrap/check-circle-fill
|
|
question: bootstrap/question-circle-fill
|
|
warning: bootstrap/exclamation-triangle-fill
|
|
failure: bootstrap/x-octagon-fill
|
|
danger: bootstrap/exclamation-diamond-fill
|
|
bug: bootstrap/bug-fill
|
|
example: bootstrap/mortarboard-fill
|
|
quote: bootstrap/quote
|
|
|
|
repo: bootstrap/github
|
|
edit: bootstrap/pencil
|
|
view: bootstrap/eye
|
|
|
|
nav:
|
|
- Overview: index.md
|
|
- User Guide:
|
|
- Getting Started:
|
|
- user-guide/index.md
|
|
- user-guide/signup.md
|
|
- user-guide/getting-started.md
|
|
- Orgs:
|
|
- user-guide/org.md
|
|
- user-guide/join.md
|
|
- user-guide/overview.md
|
|
- Crawling:
|
|
- user-guide/crawl-workflows.md
|
|
- user-guide/workflow-setup.md
|
|
- user-guide/running-crawl.md
|
|
- Archived Items:
|
|
- user-guide/archived-items.md
|
|
- user-guide/review.md
|
|
- user-guide/collections.md
|
|
- Collections:
|
|
- user-guide/collection.md
|
|
- Browser Profiles:
|
|
- user-guide/browser-profiles.md
|
|
- Org Settings:
|
|
- user-guide/org-settings.md
|
|
- user-guide/org-members.md
|
|
- Account Settings:
|
|
- user-guide/user-settings.md
|
|
- user-guide/contribute.md
|
|
- Self-Hosting:
|
|
- Overview: deploy/index.md
|
|
- deploy/local.md
|
|
- deploy/remote.md
|
|
- deploy/customization.md
|
|
- deploy/proxies.md
|
|
- Ansible:
|
|
- deploy/ansible/digitalocean.md
|
|
- deploy/ansible/microk8s.md
|
|
- deploy/ansible/k3s.md
|
|
- Administration:
|
|
- deploy/admin/org-import-export.md
|
|
- Development:
|
|
- develop/index.md
|
|
- develop/local-dev-setup.md
|
|
- develop/frontend-dev.md
|
|
- develop/docs.md
|
|
|
|
markdown_extensions:
|
|
- toc:
|
|
toc_depth: 3
|
|
permalink: true
|
|
- pymdownx.highlight:
|
|
anchor_linenums: true
|
|
- pymdownx.emoji:
|
|
emoji_index: !!python/name:material.extensions.emoji.twemoji
|
|
emoji_generator: !!python/name:material.extensions.emoji.to_svg
|
|
options:
|
|
custom_icons:
|
|
- docs/overrides/.icons
|
|
- admonition
|
|
- pymdownx.inlinehilite
|
|
- pymdownx.details
|
|
- pymdownx.superfences
|
|
- pymdownx.keys
|
|
- def_list
|
|
- attr_list
|
|
|
|
extra:
|
|
generator: false
|
|
social:
|
|
- icon: bootstrap/globe
|
|
link: https://webrecorder.net
|
|
- icon: bootstrap/chat-left-text-fill
|
|
link: https://forum.webrecorder.net/
|
|
- icon: bootstrap/mastodon
|
|
link: https://digipres.club/@webrecorder
|
|
- icon: bootstrap/youtube
|
|
link: https://www.youtube.com/@webrecorder
|
|
copyright: "Creative Commons Attribution 4.0 International (CC BY 4.0)"
|
|
|
|
plugins:
|
|
- search
|