Update backend local development docs (#884)
* docs refactor: - add local deployment guide local-dev-setup.md - deploy/local.md focuses only on deployment with latest release, links to local-dev-setup.md for local image deployment - add nav to mkdocs.yml to ensure correct order of pages - update microk8s specific info - update minikube specific info --------- Co-authored-by: Ilya Kreymer <ikreymer@gmail.com>
This commit is contained in:
sua yoo
GitHub
parent
4852532866
commit
965aa7ff90
@ -1,6 +1,6 @@
|
||||
# Local Deployment
|
||||
|
||||
To just test out Browsertrix Cloud on your local machine, you'll first need to have a working Kubernetes cluster.
|
||||
To try out the latest release of Browsertrix Cloud on your local machine, you'll first need to have a working Kubernetes cluster.
|
||||
|
||||
## Installing Kubernetes
|
||||
|
||||
@ -24,7 +24,6 @@ Here are some environment specific instructions for setting up a local cluster f
|
||||
|
||||
4. Install [Helm](https://helm.sh/), which can be installed with `brew install helm` (Mac) or `choco install kubernetes-helm` (Windows) or following some of the [other install options](https://helm.sh/docs/intro/install/)
|
||||
|
||||
|
||||
??? info "MicroK8S (recommended for Ubuntu)"
|
||||
|
||||
For Ubuntu and other linux distros, we recommend using MicroK8S for both local deployment and production.
|
||||
@ -39,13 +38,11 @@ Here are some environment specific instructions for setting up a local cluster f
|
||||
|
||||
??? info "Minikube (Windows, Mac or Linux)"
|
||||
|
||||
1. Install Minikube [following installation instructions](https://minikube.sigs.k8s.io/docs/start/), eg. `brew install minikube`
|
||||
1. Install Minikube [following installation instructions](https://minikube.sigs.k8s.io/docs/start/), eg. `brew install minikube`.
|
||||
Note that Minikube also requires Docker or another container management system to be installed as well.
|
||||
|
||||
2. Install [Helm](https://helm.sh/), which can be installed with `brew install helm` (Mac) or `choco install kubernetes-helm` (Windows) or following some of the [other install options](https://helm.sh/docs/intro/install/)
|
||||
|
||||
3. Run the Helm command as described above.
|
||||
|
||||
4. Mac Only: To access Browsertrix Cloud running in minikube on a mac, run `minikube service browsertrix-cloud-frontend --url` and then access Browsertrix Cloud via the provided URL. This is needed as Browsertrix Cloud is running in a VM.
|
||||
|
||||
??? info "K3S (recommended for non-Ubuntu Linux)"
|
||||
|
||||
@ -55,15 +52,27 @@ Here are some environment specific instructions for setting up a local cluster f
|
||||
|
||||
3. Set `KUBECONFIG` to point to the config for K3S: `export KUBECONFIG=/etc/rancher/k3s/k3s.yaml` to ensure Helm will use the correct version.
|
||||
|
||||
|
||||
## Launching Browsertrix Cloud with Helm
|
||||
|
||||
Once you have a running Kubernetes cluster with one of the options above, and Helm 3 installed, you can then run from the Browsertrix Cloud repo directory:
|
||||
|
||||
```shell
|
||||
helm upgrade --install -f ./chart/values.yaml -f ./chart/examples/local-config.yaml btrix ./chart/
|
||||
helm upgrade --install -f ./chart/values.yaml \
|
||||
-f ./chart/examples/local-config.yaml btrix ./chart/
|
||||
```
|
||||
|
||||
??? info "MicroK8S"
|
||||
|
||||
If using microk8s, the command will be:
|
||||
|
||||
```sh
|
||||
microk8s helm3 upgrade --install -f ./chart/values.yaml \
|
||||
-f ./chart/examples/local-config.yaml btrix ./chart/
|
||||
```
|
||||
|
||||
Subsequent commands will also use `microk8s helm3` instead of `helm`.
|
||||
|
||||
|
||||
The local setup includes the full Browsertrix Cloud system, with frontend, backend api, db (via MongoDB) and storage (via Minio)
|
||||
|
||||
An admin user with name `admin@example.com` and password `PASSW0RD!` will be automatically created.
|
||||
@ -72,8 +81,7 @@ This config uses the standard config (`./chart/values.yaml`) with a couple addit
|
||||
|
||||
These settings can be changed in [charts/examples/local-config.yaml](https://github.com/webrecorder/browsertrix-cloud/blob/main/chart/examples/local-config.yaml).
|
||||
|
||||
Note that the admin user and password will not be reset after creation.
|
||||
|
||||
For example, to change the default superadmin, uncomment the `superadmin` block in `local-config.yaml`, and then change the username (`admin@example.com`) and password (`PASSW0RD!`) to different values. (The admin username and password will be updated with each deployment)
|
||||
|
||||
## Waiting for Cluster to Start
|
||||
|
||||
@ -97,6 +105,13 @@ The command will exit when all pods have been loaded, or if there is an error an
|
||||
|
||||
If the command succeeds, you should be able to access Browsertrix Cloud by loading: **[http://localhost:30870/](http://localhost:30870/)** in your browser.
|
||||
|
||||
??? info "Minikube (on Mac)"
|
||||
|
||||
When using Minikube on a Mac, the port will not be 30870. Instead, Minikube opens a tunnel to a random port,
|
||||
obtained by running `minikube service browsertrix-cloud-frontend --url` in a separate terminal.
|
||||
Use the provided URL (in the format `http://127.0.0.1:<TUNNEL_PORT>`) instead.
|
||||
|
||||
|
||||
### Debugging Pod Issues
|
||||
|
||||
If this command fails, you can also run `kubectl get pods` to see the status of each of the pods.
|
||||
@ -123,77 +138,17 @@ helm upgrade --install -f ./chart/values.yaml -f ./chart/examples/local-config.y
|
||||
|
||||
To uninstall, run `helm uninstall btrix`.
|
||||
|
||||
By default, the database + storage volumes are not automatically deleted, so you can run `helm upgrade...` again to restart the cluster in its current state.
|
||||
By default, the database + storage volumes are not automatically deleted, so you can run `helm upgrade ...` again to restart the cluster in its current state.
|
||||
|
||||
To fully delete all persistent data created in the cluster, also run `kubectl delete pvc --all` after uninstalling.
|
||||
If you are upgrading from a previous version, and run into issues with `helm upgrade ...`, we recommend
|
||||
uninstalling and then re-running upgrade.
|
||||
|
||||
## Deleting all Data
|
||||
|
||||
## Running With Local Images
|
||||
To fully delete all persistent data (db + archives) created in the cluster, also run `kubectl delete pvc --all` after uninstalling.
|
||||
|
||||
By default, this setup will pull the latest release of Browsertrix Cloud. However, if you are developing locally, you may want to use your local images instead.
|
||||
## Deploying for Local Development
|
||||
|
||||
First, open `./chart/examples/local-config.yaml` and add the following, which will ensure only local images are used:
|
||||
|
||||
```yaml
|
||||
backend_pull_policy: "Never"
|
||||
frontend_pull_policy: "Never"
|
||||
```
|
||||
|
||||
Now, rebuild either the backend and/or frontend images locally. The exact process depends on the Kubernetes deployment in use:
|
||||
|
||||
|
||||
??? info "Docker Desktop"
|
||||
|
||||
Rebuild the local images by running `./scripts/build-backend.sh` and/or `./scripts/build-frontend.sh` scripts to build the images in the local Docker.
|
||||
|
||||
|
||||
??? info "MicroK8S"
|
||||
|
||||
MicroK8s uses its own container registry, running on port 32000.
|
||||
|
||||
1. Set `export REGISTRY=localhost:32000/` and then run `./scripts/build-backend.sh` and/or `./scripts/build-frontend.sh` to rebuild the images into the MicroK8S registry.
|
||||
|
||||
2. In `./chart/examples/local-config.yaml`, uncomment out one or both of the following lines to use the local images:
|
||||
|
||||
```yaml
|
||||
backend_image: "localhost:32000/webrecorder/browsertrix-backend:latest"
|
||||
frontend_image: "localhost:32000/webrecorder/browsertrix-frontend:latest"
|
||||
```
|
||||
|
||||
??? info "Minikube"
|
||||
|
||||
Minikube comes with its own image builder to update the images used in Minikube.
|
||||
|
||||
To build the backend image, run:
|
||||
|
||||
```shell
|
||||
minikube image build -t webrecorder/browsertrix-backend:latest ./backend
|
||||
```
|
||||
|
||||
To build a local frontend image, run:
|
||||
|
||||
```shell
|
||||
minikube image build -t webrecorder/browsertrix-frontend:latest ./frontend
|
||||
```
|
||||
|
||||
??? info "K3S"
|
||||
|
||||
K3S uses `containerd` by default. To use local images, they need to be imported after rebuilding.
|
||||
|
||||
1. Rebuild the images with Docker by running by running `./scripts/build-backend.sh` and/or `./scripts/build-frontend.sh` scripts. (Requires Docker to be installed as well).
|
||||
|
||||
2. Serializer the images to .tar:
|
||||
|
||||
```shell
|
||||
docker save webrecorder/browsertrix-backend:latest > ./backend.tar
|
||||
docker save webrecorder/browsertrix-frontend:latest > ./frontend.tar
|
||||
```
|
||||
|
||||
3. Import images into k3s containerd:
|
||||
|
||||
```shell
|
||||
k3s ctr images import --base-name webrecorder/browsertrix-backend:latest ./backend.tar
|
||||
k3s ctr images import --base-name webrecorder/browsertrix-frontend:latest ./frontend.tar
|
||||
```
|
||||
|
||||
Once the images have been built and any other config changes made per the above instructions, simply run the `helm upgrade...` command again to restart with local images.
|
||||
These instructions are intended for deploying the cluster from the latest release.
|
||||
See [setting up cluster for local development](../develop/local-dev-setup.md) for additional customizations related to
|
||||
developing Browsertrix Cloud and deploying from local images.
|
||||
|
||||
@ -1,7 +1,15 @@
|
||||
# Running the Frontend
|
||||
# Running the Frontend for Development
|
||||
|
||||
This guide explains how to deploy an instance of the Browsertrix Cloud frontend for development.
|
||||
The frontend can connect to a Browsertrix Cloud API backend running locally or remotely.
|
||||
|
||||
This setup may be ideal for rapid development on the frontend, as it will allow running the frontend
|
||||
via `yarn` instead of the frontend image deployed in Kubernetes.
|
||||
|
||||
## Requirements
|
||||
|
||||
This setup does require a Browsertrix Cloud API backend already in a Kubernetes cluster.
|
||||
|
||||
Refer to [setup for local deployment](./local-dev-setup.md) for additional details on this setup.
|
||||
|
||||
## Quickstart
|
||||
|
||||
@ -31,6 +39,15 @@ If connecting to a local deployment cluster, set API_BASE_URL to:
|
||||
API_BASE_URL=http://localhost:30870/api
|
||||
```
|
||||
|
||||
??? info "Minikube (on Mac)"
|
||||
|
||||
When using Minikube on a Mac, the port will not be 30870. Instead, Minikube opens a tunnel to a random port,
|
||||
obtained by running `minikube service browsertrix-cloud-frontend --url` in a separate terminal.
|
||||
|
||||
Set API_BASE_URL to provided URL instead, eg. `API_BASE_URL=http://127.0.0.1:<TUNNEL_PORT>`
|
||||
|
||||
|
||||
|
||||
Start the dev server:
|
||||
|
||||
```sh
|
||||
|
||||
@ -1,13 +1,24 @@
|
||||
---
|
||||
hide:
|
||||
- toc
|
||||
---
|
||||
|
||||
# Developing Browsertrix Cloud
|
||||
|
||||
Browsertrix Cloud consists of a Python-based backend and TypeScript-based frontend.
|
||||
|
||||
To develop Browsertrix Cloud, the system must [first be deployed locally](../deploy/local.md) in a Kubernetes cluster.
|
||||
|
||||
The deployment can then be [further customized for local development](./local-dev-setup.md).
|
||||
|
||||
### Backend
|
||||
|
||||
The backend is an API-only system, using the FastAPI framework. The latest API reference is available
|
||||
under ./api of a running cluster.
|
||||
|
||||
*TODO Add additional info here*
|
||||
At this time, the backend must be deployed in the Kubernetes cluster.
|
||||
|
||||
<!-- *TODO Add additional info here* -->
|
||||
|
||||
### Frontend
|
||||
|
||||
@ -17,5 +28,4 @@ The static build of the frontend is bundled with nginx, but the frontend can be
|
||||
|
||||
See [Running Frontend](./frontend-dev) for more details.
|
||||
|
||||
*TODO Add additional info here*
|
||||
|
||||
<!-- *TODO Add additional info here* -->
|
||||
|
||||
136
docs/develop/local-dev-setup.md
Normal file
136
docs/develop/local-dev-setup.md
Normal file
@ -0,0 +1,136 @@
|
||||
# Setup for Local Development
|
||||
|
||||
## Installation
|
||||
|
||||
First, see our [Local Deployment guide](../deploy/local.md#installing-kubernetes) for instructions on how to install the latest release with [Kubernetes](https://kubernetes.io/) with [Helm 3](https://v3.helm.sh/).
|
||||
|
||||
## Local Dev Configuration
|
||||
|
||||
The local deployment guide explains how to deploy Browsertrix Cloud with latest published images.
|
||||
|
||||
However, if you are developing locally, you will need to use your local images instead.
|
||||
|
||||
We recommend the following setup:
|
||||
|
||||
1. Copy the provided `./chart/examples/local-config.yaml` Helm configuration file to a separate file `local.yaml`,
|
||||
so that local changes to it will not be accidentally committed to git. From the root directory:
|
||||
```sh
|
||||
cp ./chart/examples/local-config.yaml ./chart/local.yaml
|
||||
```
|
||||
|
||||
2. Uncomment pull policies in `./chart/local.yaml`, which will ensure the local images are used:
|
||||
```yaml
|
||||
backend_pull_policy: 'Never'
|
||||
frontend_pull_policy: 'Never'
|
||||
```
|
||||
|
||||
??? info "MicroK8S"
|
||||
|
||||
For microk8s, the pull policies actually need to be set to `IfNotPresent` instead of `Never`:
|
||||
|
||||
```yaml
|
||||
backend_pull_policy: 'IfNotPresent'
|
||||
frontend_pull_policy: 'IfNotPresent'
|
||||
```
|
||||
|
||||
This will ensure images are pulled from the MicroK8S registry (configured in next section).
|
||||
|
||||
|
||||
|
||||
3. Build the local backend and frontend images. The exact process depends on the Kubernetes environment you've selected in your initial deployment. Environment specific build instructions are as follows:
|
||||
|
||||
??? info "Docker Desktop"
|
||||
|
||||
Rebuild the local images by running `./scripts/build-backend.sh` and/or `./scripts/build-frontend.sh` scripts to build the images in the local Docker.
|
||||
|
||||
??? info "MicroK8S"
|
||||
|
||||
MicroK8s uses its own container registry, running on port 32000.
|
||||
|
||||
1. Ensure the registry add-on is enabled by running `microk8s enable registry`
|
||||
|
||||
2. Set `export REGISTRY=localhost:32000/` and then run `./scripts/build-backend.sh` and/or `./scripts/build-frontend.sh` to rebuild the images into the MicroK8S registry.
|
||||
|
||||
3. In `./chart/local.yaml`, also uncomment the following lines to use the local images:
|
||||
```yaml
|
||||
backend_image: "localhost:32000/webrecorder/browsertrix-backend:latest"
|
||||
frontend_image: "localhost:32000/webrecorder/browsertrix-frontend:latest"
|
||||
```
|
||||
|
||||
??? info "Minikube"
|
||||
|
||||
Minikube comes with its own image builder to update the images used in Minikube.
|
||||
|
||||
To build the backend image, run:
|
||||
|
||||
```shell
|
||||
minikube image build -t webrecorder/browsertrix-backend:latest ./backend
|
||||
```
|
||||
|
||||
To build a local frontend image, run:
|
||||
|
||||
```shell
|
||||
minikube image build -t webrecorder/browsertrix-frontend:latest ./frontend
|
||||
```
|
||||
|
||||
??? info "K3S"
|
||||
|
||||
K3S uses `containerd` by default. To use local images, they need to be imported after rebuilding.
|
||||
|
||||
1. Rebuild the images with Docker by running by running `./scripts/build-backend.sh` and/or `./scripts/build-frontend.sh` scripts. (Requires Docker to be installed as well).
|
||||
|
||||
2. Serializer the images to .tar:
|
||||
```shell
|
||||
docker save webrecorder/browsertrix-backend:latest > ./backend.tar
|
||||
docker save webrecorder/browsertrix-frontend:latest > ./frontend.tar
|
||||
```
|
||||
|
||||
3. Import images into k3s containerd:
|
||||
```shell
|
||||
k3s ctr images import --base-name webrecorder/browsertrix-backend:latest ./backend.tar
|
||||
k3s ctr images import --base-name webrecorder/browsertrix-frontend:latest ./frontend.tar
|
||||
```
|
||||
|
||||
4. If you'd like to change other options, you can uncomment them as needed in `./chart/local.yaml`
|
||||
or add additional overrides from `./chart/values.yaml`
|
||||
|
||||
For example, to set a superuser email to `my_super_user_email@example.com` and password to `MySecretPassword!`, uncomment that block and set:
|
||||
```yaml
|
||||
superuser:
|
||||
# set this to enable a superuser admin
|
||||
email: my_super_user_email@example.com
|
||||
|
||||
# optional: if not set, automatically generated
|
||||
# change or remove this
|
||||
password: MySecretPassword!
|
||||
```
|
||||
|
||||
5. Once the images have been built and config changes made in `./chart/local.yaml`,
|
||||
|
||||
the cluster can be re-deployed by running:
|
||||
```sh
|
||||
helm upgrade --install -f ./chart/values.yaml \
|
||||
-f ./chart/local.yaml btrix ./chart/
|
||||
```
|
||||
|
||||
??? info "MicroK8S"
|
||||
|
||||
If using microk8s, the commend will be:
|
||||
|
||||
```sh
|
||||
microk8s helm3 upgrade --install -f ./chart/values.yaml -f ./chart/local.yaml btrix ./chart/
|
||||
```
|
||||
|
||||
Refer back to the [Local Development guide](../deploy/local.md#waiting-for-cluster-to-start) for additional information on running and debugging your local cluster.
|
||||
|
||||
## Update the Images
|
||||
|
||||
After making any changes to backend code (in `./backend`) or frontend code (in `./frontend`),
|
||||
you'll need to rebuild the images as specified above, before running `helm upgrade ...` to re-deploy.
|
||||
|
||||
Changes to settings in `./chart/local.yaml` can be deployed with `helm upgrade ...` directly.
|
||||
|
||||
## Deploying Frontend Only
|
||||
|
||||
If you are just making changes to the frontend, you can also [deploy the frontend separately](frontend-dev.md)
|
||||
using a dev server for quicker iteration.
|
||||
17
mkdocs.yml
17
mkdocs.yml
@ -38,7 +38,22 @@ theme:
|
||||
edit: bootstrap/pencil
|
||||
view: bootstrap/eye
|
||||
|
||||
|
||||
nav:
|
||||
- index.md
|
||||
- Deploy:
|
||||
- deploy/index.md
|
||||
- deploy/local.md
|
||||
- deploy/production.md
|
||||
- Ansible:
|
||||
- deploy/ansible/digitalocean.md
|
||||
- deploy/ansible/microk8s.md
|
||||
- Develop:
|
||||
- develop/index.md
|
||||
- develop/local-dev-setup.md
|
||||
- develop/frontend-dev.md
|
||||
- develop/docs.md
|
||||
- User Guide:
|
||||
- user-guide/index.md
|
||||
|
||||
markdown_extensions:
|
||||
- toc:
|
||||
|
||||
Loading…
Reference in New Issue
Block a user