6797b41de0
Fixes #2257 This is a follow-up to the public collections work, which adds pages to the database for uploads. All crawls and uploads now have a `pageCount` field which is populated when the item is successfully added. A new migration is also added to populate the field for existing archived items that don't have it set yet. OrgMetrics have also been modified to include `crawlPageCount` and `uploadPageCount`, and to include the total of both in `pageCount`, and all three included in the frontend org dashboard. The frontend has been updated to use `pageCount` rather than `stats.done` wherever appropriate, meaning that in archived item lists and details we now have a consistent page count for both crawls and uploads. ### New functionality - Deploy this branch - Create new crawls and uploads and verify that page count appears correctly throughout the frontend for all new crawls and uploads ### Migration - Deploy from latest main - Create some crawls and uploads - Change to this branch and re-deploy - Verify migration ran without errors in backend logs - Verify that page count has been populated successfully by checking archived items lists, crawl and upload detail pages, and dashboard to ensure there are no longer any missing page counts. --------- Co-authored-by: emma <hi@emma.cafe> |
||
|---|---|---|
| .. | ||
| data | ||
| __init__.py | ||
| conftest.py | ||
| echo_server.py | ||
| test_api.py | ||
| test_collections.py | ||
| test_crawl_config_search_values.py | ||
| test_crawl_config_tags.py | ||
| test_crawlconfigs.py | ||
| test_filter_sort_results.py | ||
| test_login.py | ||
| test_org_subs.py | ||
| test_org.py | ||
| test_permissions.py | ||
| test_profiles.py | ||
| test_qa.py | ||
| test_run_crawl.py | ||
| test_stop_cancel_crawl.py | ||
| test_uploads.py | ||
| test_users.py | ||
| test_utils.py | ||
| test_webhooks.py | ||
| test_workflow_auto_add_to_collection.py | ||
| test_y_org_import_export.py | ||
| test_z_delete_org.py | ||
| utils.py | ||