browsertrix/backend/btrixcloud/migrations
Tessa Walsh 094f27bcff
Track bytes stored per file type and include in org metrics (#1207)
* Add bytes stored per type to org and metrics

The org now tracks bytesStored by type of crawl, uploads, and browser profiles
in addition to the total, and returns these values in the org metrics endpoint.

A migration is added to precompute these values in existing deployments.

In addition, all /metrics storage values are now returned solely as bytes, as
the GB form wasn't being used in the frontend and is unnecessary.

* Improve deletion of multiple archived item types via `/all-crawls` delete endpoint

- Update `/all-crawls` delete test to check that org and workflow size values
are correct following deletion.
- Fix bug where it was always assumed only one crawl was deleted per cid
and size was not tracked per cid
- Add type check within delete_crawls
2023-09-22 12:55:21 -04:00
..
__init__.py
migration_0001_archives_to_orgs.py
migration_0002_crawlconfig_crawlstats.py
migration_0003_mutable_crawl_configs.py
migration_0004_config_seeds.py
migration_0005_operator_scheduled_jobs.py
migration_0006_precompute_crawl_stats.py
migration_0007_colls_and_config_update.py
migration_0008_precompute_crawl_file_stats.py
migration_0009_crawl_types.py
migration_0010_collection_total_size.py Refactor / Cleanup: move ops functions back into classes (#1171) 2023-09-13 11:56:09 -07:00
migration_0011_crawl_timeout_configmap.py Add migration to replace None with 0 for configmap CRAWL_TIMEOUT (#1008) 2023-07-24 15:49:26 -04:00
migration_0012_notes_to_description.py Rename notes to description in frontend and backend (#1011) 2023-07-26 13:00:04 -07:00
migration_0013_crawl_name.py Backend: standardize search values, filters, and sorting for archived items (#1039) 2023-08-04 09:56:52 -07:00
migration_0014_to_collection_ids.py feat: implement 'collections' array with {name, id} for archived item details (#1098) 2023-08-25 00:26:46 -07:00
migration_0015_org_storage_usage.py Track bytes stored per file type and include in org metrics (#1207) 2023-09-22 12:55:21 -04:00
migration_0016_operator_scheduled_jobs_v2.py Scheduled Crawl Refactor: Handle via Operator + Add Skipped Crawls on Quota Reached (#1162) 2023-09-12 13:05:43 -07:00
migration_0017_storage_by_type.py Track bytes stored per file type and include in org metrics (#1207) 2023-09-22 12:55:21 -04:00