browsertrix/backend/btrixcloud/operator
Ilya Kreymer 3438133fcb
Crawler pod memory padding + auto scaling (#1631)
- set memory limit to 1.2x memory request to provide extra padding and
avoid OOM
- attempt to resize crawler pods by 1.2x when exceeding 90% of available
memory
- do a 'soft OOM' (send extra SIGTERM) to pod when reaching 100% of
requested memory, resulting in faster graceful restart, but avoiding a
system-instant OOM Kill
- Fixes #1632

---------
Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
2024-03-28 16:39:00 -07:00
..
__init__.py Refactor operator class into module (#1564) 2024-02-29 14:40:12 -08:00
baseoperator.py Crawler pod memory padding + auto scaling (#1631) 2024-03-28 16:39:00 -07:00
bgjobs.py Refactor operator class into module (#1564) 2024-02-29 14:40:12 -08:00
crawls.py Crawler pod memory padding + auto scaling (#1631) 2024-03-28 16:39:00 -07:00
cronjobs.py QA Runs Initial Backend Implementation (#1586) 2024-03-20 22:42:16 -07:00
models.py Crawler pod memory padding + auto scaling (#1631) 2024-03-28 16:39:00 -07:00
profiles.py Refactor operator class into module (#1564) 2024-02-29 14:40:12 -08:00