bb6e703f6a
Resolves #1354 Supports crawling through pre-configured proxy servers, allowing users to select which proxy servers to use (requires browsertrix crawler 1.3+) Config: - proxies defined in btrix-proxies subchart - can be configured via btrix-proxies key or separate proxies.yaml file via separate subchart - proxies list refreshed automatically if crawler_proxies.json changes if subchart is deployed - support for ssh and socks5 proxies - proxy keys added to secrets in subchart - support for default proxy to be always used if no other proxy configured, prevent starting cluster if default proxy not available - prevent starting manual crawl if previously configured proxy is no longer available, return error - force 'btrix' username and group name on browsertrix-crawler non-root user to support ssh Operator: - support crawling through proxies, pass proxyId in CrawlJob - support running profile browsers which designated proxy, pass proxyId to ProfileJob - prevent starting scheduled crawl if previously configured proxy is no longer available API / Access: - /api/orgs/all/crawlconfigs/crawler-proxies - get all proxies (superadmin only) - /api/orgs/{oid}/crawlconfigs/crawler-proxies - get proxies available to particular org - /api/orgs/{oid}/proxies - update allowed proxies for particular org (superadmin only) - superadmin can configure which orgs can use which proxies, stored on the org - superadmin can also allow an org to access all 'shared' proxies, to avoid having to allow a shared proxy on each org. UI: - Superadmin has 'Edit Proxies' dialog to configure for each org if it has: dedicated proxies, has access to shared proxies. - User can select a proxy in Crawl Workflow browser settings - Users can choose to launch a browser profile with a particular proxy - Display which proxy is used to create profile in profile selector - Users can choose with default proxy to use for new workflows in Crawling Defaults --------- Co-authored-by: Ilya Kreymer <ikreymer@gmail.com> Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
59 lines
1.6 KiB
YAML
59 lines
1.6 KiB
YAML
name: Publish Helm Chart
|
|
|
|
on:
|
|
push:
|
|
branches:
|
|
- main
|
|
- "*-release"
|
|
|
|
jobs:
|
|
package_chart:
|
|
runs-on: ubuntu-latest
|
|
|
|
steps:
|
|
- name: Check out Git repository
|
|
uses: actions/checkout@v2
|
|
|
|
- name: Install Helm
|
|
uses: azure/setup-helm@v3
|
|
with:
|
|
version: 3.10.2
|
|
|
|
- name: Package Chart
|
|
run: |
|
|
mkdir .chart-out
|
|
helm package chart/ --destination .chart-out
|
|
helm package chart/proxies/ --destination .chart-out
|
|
|
|
- name: Get Version
|
|
run: |
|
|
echo "version=$(cat version.txt)" >> "$GITHUB_ENV"
|
|
|
|
- name: Upload Chart To GitHub
|
|
uses: softprops/action-gh-release@v1
|
|
with:
|
|
body: |
|
|
## Deployment
|
|
|
|
This release can be deployed with Helm directly by running:
|
|
|
|
```
|
|
helm upgrade --install btrix https://github.com/webrecorder/browsertrix/releases/download/v${{ env.version }}/browsertrix-v${{ env.version }}.tgz
|
|
```
|
|
|
|
Alternatively, you can add the Browsertrix Helm Chart Repo with:
|
|
|
|
```
|
|
helm repo add browsertrix https://docs.browsertrix.com/helm-repo/
|
|
helm upgrade --install btrix browsertrix/browsertrix --version ${{ env.version }}
|
|
```
|
|
|
|
See [the development guide](https://docs.browsertrix.com/deploy/) for more info how to deploy Browsertrix.
|
|
|
|
files: |
|
|
.chart-out/browsertrix-v${{ env.version }}.tgz
|
|
.chart-out/btrix-proxies-0.1.0.tgz
|
|
tag_name: v${{ env.version }}
|
|
draft: true
|
|
fail_on_unmatched_files: true
|