tea/browsertrix
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
6e9c14aea6
browsertrix/backend/btrixcloud
History
Ilya Kreymer e7af081af1
profile browser fixes: better resource usage + load retry (main) (#1604) 
- Backend: Use separate resource constraints for profiles: default
profile browser resources to either 'profile_browser_cpu' /
'profile_browser_memory' or single browser 'crawler_memory_base' /
'crawler_cpu_base', instead of scaled to the number of browser workers

- Frontend: check that profile html page is loading, keep retrying if
still getting nginx error instead of loading an iframe with the error.

Fixes #1598 (Copy of #1599 from 1.9.4)
2024-03-16 15:07:04 -07:00
..
migrations Temporarily remove pages migration (#1572) 2024-03-04 10:30:04 -08:00
operator profile browser fixes: better resource usage + load retry (main) (#1604) 2024-03-16 15:07:04 -07:00
__init__.py
auth.py Additional Type Hints / Type Fix Pass (#1320) 2023-10-30 12:59:24 -04:00
background_jobs.py Format backend with Black 24 (#1507) 2024-02-07 11:35:34 -08:00
basecrawls.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
colls.py Format backend with Black 24 (#1507) 2024-02-07 11:35:34 -08:00
crawlconfigs.py More friendly WARC prefix inside WACZ based on Org slug + Crawl Name / First Seed URL. (#1537) 2024-02-22 23:54:23 -08:00
crawlmanager.py More friendly WARC prefix inside WACZ based on Org slug + Crawl Name / First Seed URL. (#1537) 2024-02-22 23:54:23 -08:00
crawls.py Merge V1.9.3 changes into main (#1583) 2024-03-06 15:38:22 -08:00
db.py Temporarily remove pages migration (#1572) 2024-03-04 10:30:04 -08:00
emailsender.py Email Templates (#1375) 2023-11-15 15:22:12 -08:00
invites.py Email Templates (#1375) 2023-11-15 15:22:12 -08:00
k8sapi.py More friendly WARC prefix inside WACZ based on Org slug + Crawl Name / First Seed URL. (#1537) 2024-02-22 23:54:23 -08:00
main_op.py Refactor operator class into module (#1564) 2024-02-29 14:40:12 -08:00
main.py Configurable Max Scale for frontend (#1557) 2024-03-11 16:21:20 -07:00
models.py Add updatable QA reviewStatus field to crawls (#1575) 2024-03-05 16:49:23 -08:00
orgs.py Format backend with Black 24 (#1507) 2024-02-07 11:35:34 -08:00
pages.py pages: set page status to 200 if unset and loadState != 0 (#1563) 2024-02-29 15:15:17 -08:00
pagination.py Format backend with Black 24 (#1507) 2024-02-07 11:35:34 -08:00
profiles.py Support multiple crawler versions (#1420) 2024-01-16 15:32:12 -08:00
storages.py Make storage deletion work on more S3 providers, don't use access URL for deletion (#1600) 2024-03-16 04:17:23 -04:00
uploads.py Background Jobs Work (#1321) 2023-11-02 13:02:17 -07:00
users.py Add API endpoints for crawl statistics (#1461) 2024-01-10 13:30:47 -08:00
utils.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00
version.py version: bump to 1.10.0-beta.0 2024-02-20 00:22:29 -08:00
webhooks.py Add crawl, upload, and collection delete webhook event notifications (#1363) 2023-11-09 18:19:08 -08:00
zip.py Add crawl pages and related API endpoints (#1516) 2024-02-28 12:11:35 -05:00