4f676e4e82
Supports running QA Runs via the QA API! Builds on top of the `issue-1498-crawl-qa-backend-support` branch, fixes #1498 Also requires the latest Browsertrix Crawler 1.1.0+ (from webrecorder/browsertrix-crawler#469 branch) Notable changes: - QARun objects contain info about QA runs, which are crawls performed on data loaded from existing crawls. - Various crawl db operations can be performed on either the crawl or `qa.` object, and core crawl fields have been moved to CoreCrawlable. - While running,`QARun` data stored in a single `qa` object, while finished qa runs are added to `qaFinished` dictionary on the Crawl. The QA list API returns data from the finished list, sorted by most recent first. - Includes additional type fixes / type safety, especially around BaseCrawl / Crawl / UploadedCrawl functionality, also creating specific get_upload(), get_basecrawl(), get_crawl() getters for internal use and get_crawl_out() for API - Support filtering and sorting pages via `qaFilterBy` (screenshotMatch, textMatch) along with `gt`, `lt`, `gte`, `lte` params to return pages based on QA results. --------- Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
37 lines
728 B
YAML
37 lines
728 B
YAML
|
|
{{- range untilStep 0 (int .Values.max_crawl_scale) 1 }}
|
|
---
|
|
apiVersion: scheduling.k8s.io/v1
|
|
kind: PriorityClass
|
|
metadata:
|
|
name: crawl-instance-{{ . }}
|
|
value: -{{ . }}
|
|
globalDefault: false
|
|
description: "Priority for crawl instance #{{ . }}"
|
|
|
|
{{- end }}
|
|
|
|
{{- range untilStep 0 (int .Values.max_crawl_scale) 1 }}
|
|
---
|
|
apiVersion: scheduling.k8s.io/v1
|
|
kind: PriorityClass
|
|
metadata:
|
|
name: qa-crawl-instance-{{ . }}
|
|
value: -{{ add 100 . }}
|
|
globalDefault: false
|
|
description: "Priority for QA crawl instance #{{ . }}"
|
|
|
|
{{- end }}
|
|
|
|
# Lower Priority for Background Jobs
|
|
---
|
|
apiVersion: scheduling.k8s.io/v1
|
|
kind: PriorityClass
|
|
metadata:
|
|
name: bg-job
|
|
value: -1000
|
|
globalDefault: false
|
|
description: "Priority for background jobs"
|
|
|
|
|