browsertrix/backend/btrixcloud/operator
Ilya Kreymer 94e985ae13
optimize org quota lookups (#1973)
- instead of looking up storage and exec min quotas from oid, and
loading an org each time, load org once and then check quotas on the org
object - many times the org was already available, and was looked up
again
- storage and exec quota checks become sync
- rename can_run_crawl() to more generic can_write_data(), optionally
also checks exec minutes
- typing: get_org_by_id() always returns org, or throws, adjust methods
accordingly (don't check for none, catch exception)
- typing: fix typo in BaseOperator, catch type errors in operator
'org_ops'
- operator quota check: use up-to-date 'status.size' for current job,
ignore current job in all jobs list to avoid double-counting
- follow up to #1969
2024-07-25 14:00:16 -07:00
..
__init__.py
baseoperator.py optimize org quota lookups (#1973) 2024-07-25 14:00:16 -07:00
bgjobs.py
crawls.py optimize org quota lookups (#1973) 2024-07-25 14:00:16 -07:00
cronjobs.py optimize org quota lookups (#1973) 2024-07-25 14:00:16 -07:00
models.py Standardize handling of storage and execution time quotas (#1969) 2024-07-25 12:49:11 -07:00
profiles.py