browsertrix/chart
Ilya Kreymer 90197b2a85
Backend mem usage fix - use fixed MOTOR_MAX_WORKERS + switch to gunicorn (#1468)
Refactors backend deployment to:
- Use MOTOR_MAX_WORKERS (defaulting to 1) to reduce threads used by
mongodb connections
- Also sets backend workers to 1 by default to reduce default memory
usage
- Switches to gunicorn with uvloop worker for production use instead of
uvicorn (as recommended by uvicorn)

Lower thread count should address memory leak/increased usage, which
resulted in 5x thread x cpus x workers, eg. potentially 20 or 40 threads
just for mongodb connections. Lower default number of workers should
make it easier to scale backend with HPA if additional capacity.

Fixes #1467
2024-01-16 15:32:42 -08:00
..
admin/logging Add lightweight logging mode (#668) 2023-03-10 14:34:37 -08:00
app-templates Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
btrix-crds Add max crawl size option to backend and frontend (#1045) 2023-08-26 22:00:37 -07:00
charts Operator refactor to control pods + pvcs directly instead of statefulsets (#1149) 2023-09-11 10:38:04 -07:00
email-templates Email Templates (#1375) 2023-11-15 15:22:12 -08:00
examples Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
templates Backend mem usage fix - use fixed MOTOR_MAX_WORKERS + switch to gunicorn (#1468) 2024-01-16 15:32:42 -08:00
test Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
.helmignore
Chart.lock Operator refactor to control pods + pvcs directly instead of statefulsets (#1149) 2023-09-11 10:38:04 -07:00
Chart.yaml version: bump to 1.9.0-beta.0 2023-12-20 00:08:16 -08:00
README.md Refactor to use new operator on backend (#789) 2023-04-24 18:30:52 -07:00
values.yaml Backend mem usage fix - use fixed MOTOR_MAX_WORKERS + switch to gunicorn (#1468) 2024-01-16 15:32:42 -08:00

Update Helm dependencies

  • It needs to update Helm charts after changing its dependencies (e.g. logging)
$ helm dependency update .

Update metacontroller

#!/bin/bash

# intall metacontroller
git clone --depth=1 https://github.com/metacontroller/metacontroller.git
cd metacontroller
helm package deploy/helm/metacontroller --destination deploy/helm
cd ..

# update dependency
helm dependency update
  • Bump up the metacontroller version in Chart.yaml