dfba4b3940
- Adds two new crawl finished state, stopped_by_user and stopped_quota_reached - Tracking other possible 'stop reasons' in operator, though not making them distinct states for now. - Updated frontend with 'Stopped by User' and 'Stopped: Time Quota Reached', shown with same icon as current partial_complete - Added migration of partial_complete to either stopped_by_user or complete (no historical quota data available) - Addresses edge case in scaling: if crawl never scaled (no redis entry, no pod), automatically scale down - Edge case in status: if crawl is somehow 'canceled' but not deleted, immediately delete crawl object and begin finalizing. --------- Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
30 lines
550 B
Plaintext
30 lines
550 B
Plaintext
uvicorn
|
|
fastapi==0.103.2
|
|
motor==3.3.1
|
|
passlib
|
|
PyJWT==2.8.0
|
|
pydantic==1.10.13
|
|
email-validator
|
|
#fastapi-users[mongodb]==9.2.2
|
|
loguru
|
|
aiofiles
|
|
kubernetes-asyncio==25.11.0
|
|
kubernetes
|
|
aiobotocore
|
|
redis>=5.0.0
|
|
pyyaml
|
|
jinja2
|
|
humanize
|
|
python-multipart
|
|
pathvalidate
|
|
#https://github.com/ikreymer/stream-zip/archive/refs/heads/stream-uncompress.zip
|
|
https://github.com/ikreymer/stream-zip/archive/refs/heads/stream-ignore-local-crc32.zip
|
|
boto3
|
|
backoff>=2.2.1
|
|
python-slugify>=8.0.1
|
|
mypy_boto3_s3
|
|
types_aiobotocore_s3
|
|
types-redis
|
|
types-python-slugify
|
|
types-pyYAML
|