tea/browsertrix
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
16e7a1d0a2
browsertrix/backend/btrixcloud
History
Ilya Kreymer 16e7a1d0a2
Storage Ops Refactor (#1257) 
* storage ops refactor:
- create StorageOps class similar to other ops classes
- init storages list in StorageOps, no longer require lookup up default storages via CrawlManager
- convert all storage functions to members, add storageops to operator
- remove unused params, ensure crawl exists for rollover restart
- add env var to determine if using local minio to use correct endpoint URL

* crawls /seeds endpoint: just return empty list if not a crawl (eg. upload)

* crawlmanager: remove unused code, rename check_storage -> has_storage
2023-10-10 15:04:23 -07:00
..
migrations optimization: convert all uses of 'async for' to use iterator directly (#1229) 2023-09-28 12:31:08 -07:00
templates Track pod resource usage, detect OOM crashes, handle auto-scaling (#1235) 2023-10-05 20:41:18 -07:00
__init__.py
basecrawls.py Storage Ops Refactor (#1257) 2023-10-10 15:04:23 -07:00
colls.py Storage Ops Refactor (#1257) 2023-10-10 15:04:23 -07:00
crawlconfigs.py Separate seeds into a new endpoints (#1217) 2023-10-02 10:56:12 -07:00
crawlmanager.py Storage Ops Refactor (#1257) 2023-10-10 15:04:23 -07:00
crawls.py Storage Ops Refactor (#1257) 2023-10-10 15:04:23 -07:00
db.py migration improvements: (#1228) 2023-09-28 12:04:19 -07:00
emailsender.py feat: use is_bool to check EMAIL_SMTP_USE_TLS (#1231) 2023-10-02 21:29:36 -07:00
invites.py feat: use is_bool to check EMAIL_SMTP_USE_TLS (#1231) 2023-10-02 21:29:36 -07:00
k8sapi.py Track pod resource usage, detect OOM crashes, handle auto-scaling (#1235) 2023-10-05 20:41:18 -07:00
main_op.py Storage Ops Refactor (#1257) 2023-10-10 15:04:23 -07:00
main.py Storage Ops Refactor (#1257) 2023-10-10 15:04:23 -07:00
models.py Compute crawl execution time in operator (#1256) 2023-10-09 17:45:00 -07:00
operator.py Compute crawl execution time in operator (#1256) 2023-10-09 17:45:00 -07:00
orgs.py Compute crawl execution time in operator (#1256) 2023-10-09 17:45:00 -07:00
pagination.py Move pydantic models to separate module + refactor crawl response endpoints to be consistent (#983) 2023-07-20 13:05:33 +02:00
profiles.py Storage Ops Refactor (#1257) 2023-10-10 15:04:23 -07:00
storages.py Storage Ops Refactor (#1257) 2023-10-10 15:04:23 -07:00
uploads.py Storage Ops Refactor (#1257) 2023-10-10 15:04:23 -07:00
users.py fix: lookup user object operator to pass to CrawlConfig.add_new_crawl (#1254) 2023-10-05 21:30:10 -07:00
utils.py Track pod resource usage, detect OOM crashes, handle auto-scaling (#1235) 2023-10-05 20:41:18 -07:00
version.py version: bump to 1.7.0-beta.2 2023-10-05 20:33:38 -07:00
webhooks.py Improved type checking for backend with mypy (#1174) 2023-09-13 19:40:26 -07:00
zip.py Fix: Stream log downloading from WACZ (#1225) 2023-09-28 18:54:52 -07:00