browsertrix/backend/requirements.txt
Ilya Kreymer 90197b2a85
Backend mem usage fix - use fixed MOTOR_MAX_WORKERS + switch to gunicorn (#1468)
Refactors backend deployment to:
- Use MOTOR_MAX_WORKERS (defaulting to 1) to reduce threads used by
mongodb connections
- Also sets backend workers to 1 by default to reduce default memory
usage
- Switches to gunicorn with uvloop worker for production use instead of
uvicorn (as recommended by uvicorn)

Lower thread count should address memory leak/increased usage, which
resulted in 5x thread x cpus x workers, eg. potentially 20 or 40 threads
just for mongodb connections. Lower default number of workers should
make it easier to scale backend with HPA if additional capacity.

Fixes #1467
2024-01-16 15:32:42 -08:00

31 lines
569 B
Plaintext

gunicorn
uvicorn[standard]
fastapi==0.103.2
motor==3.3.1
passlib
PyJWT==2.8.0
pydantic==1.10.13
email-validator
#fastapi-users[mongodb]==9.2.2
loguru
aiofiles
kubernetes-asyncio==25.11.0
kubernetes
aiobotocore
redis>=5.0.0
pyyaml
jinja2
humanize
python-multipart
pathvalidate
#https://github.com/ikreymer/stream-zip/archive/refs/heads/stream-uncompress.zip
https://github.com/ikreymer/stream-zip/archive/refs/heads/stream-ignore-local-crc32.zip
boto3
backoff>=2.2.1
python-slugify>=8.0.1
mypy_boto3_s3
types_aiobotocore_s3
types-redis
types-python-slugify
types-pyYAML