tea/browsertrix
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3438133fcb
browsertrix/backend/btrixcloud
History
Ilya Kreymer 3438133fcb
Crawler pod memory padding + auto scaling (#1631) 
- set memory limit to 1.2x memory request to provide extra padding and
avoid OOM
- attempt to resize crawler pods by 1.2x when exceeding 90% of available
memory
- do a 'soft OOM' (send extra SIGTERM) to pod when reaching 100% of
requested memory, resulting in faster graceful restart, but avoiding a
system-instant OOM Kill
- Fixes #1632

---------
Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
2024-03-28 16:39:00 -07:00
..
migrations QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
operator Crawler pod memory padding + auto scaling (#1631) 2024-03-28 16:39:00 -07:00
__init__.py
auth.py Additional Type Hints / Type Fix Pass (#1320) 2023-10-30 12:59:24 -04:00
background_jobs.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
basecrawls.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
colls.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
crawlconfigs.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
crawlmanager.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
crawls.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
db.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
emailsender.py Email Templates (#1375) 2023-11-15 15:22:12 -08:00
invites.py Email Templates (#1375) 2023-11-15 15:22:12 -08:00
k8sapi.py Crawler pod memory padding + auto scaling (#1631) 2024-03-28 16:39:00 -07:00
main_op.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
main.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
models.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
orgs.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
pages.py Add single page QA GET endpoint (#1635) 2024-03-27 14:57:59 -07:00
pagination.py Format backend with Black 24 (#1507) 2024-02-07 11:35:34 -08:00
profiles.py Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
storages.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
uploads.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
users.py Add API endpoints for crawl statistics (#1461) 2024-01-10 13:30:47 -08:00
utils.py Add additional filters to page list endpoints (#1622) 2024-03-21 21:33:07 -07:00
version.py version: bump to 1.10.0-beta.0 2024-02-20 00:22:29 -08:00
webhooks.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00