Initial implementation of #1892
- Modifies the backend to return `duplicate_org_name` or
`duplicate_org_slug` as appropriate on a pymongo `DuplicateKeyError`
- Updates frontend to handle `duplicate_org_name`, `duplicate_org_slug`,
and `invalid_slug` error details
- Update errors to be more consistent, also return `duplicate_org_subscription.subId` for duplicate subscription instead of the more generic `already_exists`
---------
Co-authored-by: Ilya Kreymer <ikreymer@gmail.com>
No issue created, but noticed issue here
ed0d489cda
### Changes
- Removes unused node module mocks and use commonjs plugin to import
modules in tests
- Fixes org form test after removing temporary stub
The default org will always be sorted first, regardless of sort options.
Orgs after the first will be sorted by name ascending by default.
Sorting currently supported on name, slug, and readOnly.
Fixes#890
This PR introduces new streaming superuser-only API endpoints to export
and import database information for an organization. New Adminstrator
deployment documentation on how to manage the process and copy files
between S3 buckets as needed is also included.
---------
Co-authored-by: Henry Wilkinson <henry@wilkinson.graphics>
Co-authored-by: Ilya Kreymer <ikreymer@gmail.com>
Resolves https://github.com/webrecorder/browsertrix/issues/1874
Support for new two-part sign up flow if first admin user is added to org
- If new user, user registers first, then is able to change the org name / slug on following screen
- If existing user, user accepts invite, then is able to change the org name / slug on following screen
- After confirming org slug name, user is taken to dashboard, or error is shown if org name or slug already taken.
- If org name == org id, org name and slug is automatically set to `{Your Name}'s Archive` when first user is registered / accepts invite
- Email templates updated to better reflect new / existing users and not show org name if it is 'unset' (org name == org id internally)
- tests: frontend unit testing for accept + invite screens.
---------
Co-authored-by: Ilya Kreymer <ikreymer@gmail.com>
Co-authored-by: sua yoo <sua@suayoo.com>
Co-authored-by: sua yoo <sua@webrecorder.org>
Co-authored-by: Henry Wilkinson <henry@wilkinson.graphics>
Co-authored-by: Ilya Kreymer <ikreymer@users.noreply.github.com>
Co-authored-by: Emma Segal-Grossman <hi@emma.cafe>
Closes#1851
This changes how we use the `<btrix-meter>` component: previously, we
didn't set a `max` value on it, so it was drawing its internal (rounded)
bar at 100% of the meter width, and then drawing the
`<btrix-meter-bar>`s inside that 100% width bar. This changes that, and
instead it now does the following:
- Draws the internal rounded bar to the width of the proportion of their
counts to the total page count;
- Adjusts the `<btrix-meter-bar>`s to be proportional to the completed
page count rather than the total page count
- Uses the `"available"` slot in `<btrix-meter>` to draw the remaining
"No data" bar & tooltip in the remaining space outside of the internal
rounded bar
This also includes a couple unrelated but adjacent changes:
- Changes the page count text to say "Analysis starting" when the
analysis run is starting, because there's no meaningful page count at
that point.
- Uses the pending state when the total page count is falsy (`undefined`
or `0`), rather than just `undefined`
- Switches from spans with classes to `ins` and `del` tags for text
comparison
- Adds `aria-label` attributes for screen readers. Text comparison is
accessible now! 🎉
- I kept the strikethrough text on the `del` tag but can remove? Only
visible on hover.
- Removes autofocus on the delete analysis run cancel button
- Adds `aria-label` attribute for the heuristic nav bar to describe the
context of the content users will be seeing on each page.
Fixes#1846
- Ensure meter auto-updates as new stats are ready
- Switch meter to new QA run when new analysis run is started
- Remove Files from QA meter (files and errors will be reported separately)
Co-authored-by: emma <hi@emma.cafe>
Co-authored-by: sua yoo <sua@webrecorder.org>
Switches `sl-progress-ring` and `sl-icon` to inherit font size from
table cell for consistent sizing as base size, while also fixing
`var(--font-size-base)` not being available when progress ring size is
first calculated.
Fixes#1835
Fixes#1833
- Add firstSeed and seedCount to workflow information in profile detail
API endpoint (tests updated accordingly), update name of model used for
limited workflow information to be more accurate
- Fix name display in Crawl Workflows list at bottom of Profile detail
page to be consistent with rest of application
---------
Co-authored-by: Emma Segal-Grossman <hi@emma.cafe>
This PR adds Identical Files to the QA Page Match Analysis meter bars.
To do this, the backend calculates the number of non-HTML pages once and
includes it under the key `Files` in each of the `screenshotMatch` and
`textMatch` QA stats return arrays.
The backend additionally removes the file count from "No Data" to
prevent these from being counted twice.
---------
Co-authored-by: emma <hi@emma.cafe>
Resolves https://github.com/webrecorder/browsertrix/issues/1409
### Changes
- Enables clicking on Browser Profiles column header to sort the table, including by starting URL
- More consistent column widths throughout app
---------
Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
Co-authored-by: emma <hi@emma.cafe>
Co-authored-by: Henry Wilkinson <henry@wilkinson.graphics>
Follow up to #1815
- Improved detection / responsiveness to unavailable browser
- Disable Save buttons when browser disconnected / unavailable
- Only add unload handler when browser is available
- Consistency between new and edit views: new profile view also has
cancel button with confirmation
- Don't show cancel confirmation if browser is already unavailable.
Various text tweaks for consistency / accuracy:
- Save New Profile... to indicate new profile will be saved after dialog
- Load new browser to indicate new browser will be created
- Mention connection to browser is lost in disconnected message.
Also: migrated to LiteElement -> TailwindElement for browser-profiles-new
-------
Co-authored-by: emma <hi@emma.cafe>
- Shows browser profile last modified or created by name, if available
- Moves backed-up status to browser profile subsection header
- Moves "Last Updated" column to last and displays user name on hover,
to match archived items list view
- Updates browser profile docs
- Paginates browser profile list view
- Moves back-up status icon to list view
- Display last updated date if available (fallback to created date)
- Adds max length validation to description
- Better handle description and crawl workflow list
Closes#1818
- Refactors image renderers, and changes the condition for which the
loading indicator is displayed from "`<data>.blobUrl` exists and has a
truthey value" to "`<data>` is not nullish"
- Adds an additional class config to the shared QA loading indicator,
and uses it in the image slider view to add a background and aspect
ratio to the loading indicator's container so that it takes up the space
it should and has an opaque background
| Before | After |
| --- | --- |
| <img width="813" alt="Screenshot 2024-05-22 at 5 17 14 PM"
src="https://github.com/webrecorder/browsertrix/assets/5727389/fb8a3a83-a0c8-46c1-b59b-a59adff2a175">
| <img width="813" alt="Screenshot 2024-05-22 at 5 17 31 PM"
src="https://github.com/webrecorder/browsertrix/assets/5727389/6fe5d04a-ffa9-4fc3-bc3b-bdb9b7b30948">
|
Resolves https://github.com/webrecorder/browsertrix/issues/438
<!-- Fixes #issue_number -->
### Changes
- Layout and design pass on profile detail view to better match other
detail views
- Ensures profile save buttons are always visible when creating or
editing a profile
- Handle temporarily disconnected browser pings
- Enable recreating browser after it expires
- Show confirmation dialog when attempting to navigate away from active
browser
- Tweak copy referencing profiles
- Refactors `profile-browser` into `TailwindComponent`
### Follow-ups
UX improvements to the browser profile detail layout and list view will
be handled in https://github.com/webrecorder/browsertrix/issues/1816
---------
Co-authored-by: Ilya Kreymer <ikreymer@users.noreply.github.com>
Co-authored-by: Emma Segal-Grossman <hi@emma.cafe>
Co-authored-by: Henry Wilkinson <henry@wilkinson.graphics>
- Fixes browser profile metadata form submission
- Fixes rendering of browser profile description in list view
- Changes label to "metadata" for consistency with other features
Resolves https://github.com/webrecorder/browsertrix/issues/1398
### Changes
- Updates org list in user info state after an org is created
- Minor refactor to add `btrix` prefix to custom `update-user-info`
event and replace unnecessary `defaultOrg` property
- Adds user role select to superadmin dashboard
- Requires org selection (unless there's only one org) to prevent
accidental crawler invites to default testing org
- Shows link to org members after invite, retaining form for quick
re-invite
- Refactor `invite-form` into `TailwindComponent`
Closes#1789
### Changes
- Converts stylesheet in js to css now that it doesn't use any runtime
values
- Removes fixed height on QA container, and instead sets a min height
- Applies `contain: size` to page list & tab content so they don't cause
the QA container to stretch
- Also updates instances of `h-{n} w-{n}` to `size-{n}`
- Removes leftover outlines on text and resource comparison areas from
early QA prototyping
Closes#1787
- Crawls with only one seed will load viewing that seed URL
- URL list crawls with >1 URLs will continue to load the pages list as no
ReplayWeb.page enbed URL has been defined
Co-authored-by: sua yoo <sua@webrecorder.org>
Fixes#1782
- Dash icons are now used to convey status exclusively
- Slash icons are now used to convey no data states
- Updates status icons to filled in the docs (also required for QA
docs!)
Resolves https://github.com/webrecorder/browsertrix/issues/1761
<!-- Fixes #issue_number -->
### Changes
- Switches QA review page chip tooltip colors to dark text on light
background
- Shows newest comment when hovering over comment cell in QA tab page
list to match tooltip in QA review page list
- Validates textarea max entry for description and comments
- Prevents misleading "no text" message shown when switching tabs
- Reset data and show loading indicator when switching pages
- Fix replay tab height and visually differentiates empty replay window
from empty tab
- Refactors console.logs to debugs
Fixes https://github.com/webrecorder/browsertrix/issues/1638
### Changes
- Fixes collection selection during upload and in workflow settings
- Refactors `btrix-collections-add` to `TailwindComponent` with lit
`Task` usage
### Manual testing
1. Log in as crawler
2. Go to "Archived Items"
3. Click "Upload WACZ"
4. Select wacz file to upload
5. Search for collection in "Add to Collection". Verify you're able to
select a search result
6. Save. Verify collection saves as expected.
7. Go to "Crawling"
8. Create a new workflow
9. Go to "Metadata"
10. Repeat 5-6.
Disable the 'Review Crawl' button only when there are no finished QA
Runs:
- track last finished qa run 'lastFinishedQARunId'
- track if qa run is running via 'isQAActive'
- only add tooltip / disabled state when there are no finished qa runs
Also, only add href when not disabled, to avoid clickable link with
disabled button
---------
Co-authored-by: emma <hi@emma.cafe>
Fixes https://github.com/webrecorder/browsertrix/issues/1743
On the backend, support adding user to new org and improved error
messaging:
- if user exists is not part of the org to be added to, add user to the
registration org, return 201
- if user is already part of the org, return 400 with
'user_already_is_org_member' error
- if user is not being added to a new org but already exists, return
'user_already_exists'
frontend:
- if user user same password, they will just be logged in and added to
registration org (same as before)
- if user uses wrong password, show alert message
- handle both user_already_is_org_member and user_already_registered
with alert message and link to log-in page.
note: this means existing user is added to the registration org even if
they provide wrong password, but they won't be able
to login until they use correct password/reset/etc...
Now that we're using a bunch of tooltips, I found when there are very
few items listed the tooltips sometimes get squished in weird ways —
this fixes that.
Related to https://github.com/webrecorder/browsertrix/issues/1477, minor
UI tweaks as fast follow:
- Makes archived item status icons all the same size
- Increases hover hit target of archived item cells
- Reduces top padding of QA review page
- Adds "Back" button to QA review page to match all other pages
- Updates percentage formatting of tab labels to match pages
- Refreshes archived items page every 5 seconds to get in progress QA
runs.
- Refactors archived items to use `TailwindComponent` and lit tasks
- Adds back all columns to upload list to prevent layout shift (see
follow-ups)
- Fixes list not updating after search value is selected.
- Resolves https://github.com/webrecorder/browsertrix/issues/1705
- Resolves https://github.com/webrecorder/browsertrix/issues/1477
Changes:
- Takes user back to archived item QA tab after submitting review.
- Prevents clicking on page in QA tab when there's no analysis runs.
- Disables analysis-related sort options in QA tab when there's no
analysis runs.
- Handles pages without a title in QA tab.
- Adds initial loading state to QA review page list.
Sort of relevant to #1696
- Improves a number of layout elements at smaller viewport sizes
(specifically, letting button groups wrap onto next lines & ensure
titles can't shrink to 0 width)
- Adds "No page title" to places where there'd normally be a page title
but isn't
- Applies grid styling to page list area to fix overflow issues
- When selecting or loading with a page selected that's farther down on
the page list, the top of the page header could be scrolled away from,
and there'd be no way to scroll back because the area had `overflow:
hidden` applied
- Adds default width and height to image comparer element, so that it
displays correctly when images haven't loaded and doesn't change layout
when images load
---------
Co-authored-by: Henry Wilkinson <henry@wilkinson.graphics>
- Shows QA review status, latest QA state, and QA run count in archived
item list. "Created by" column replaced for space and moved to "Date
Created" tooltip
- Enables sort archived items by QA columns
- Adds tooltip to all columns except name
- Hides non-applicable columns in when archived items are filtered to
uploads only
- Fixes issue where tooltips weren't activated on hover when using
`btrix-table`
- Minor refactor to reuse status labels/colors, renames page review ->
page approval to clarify
Repository Index: Generate an index.yaml in ./docx/helm-repo/index.yaml
to allow for browsertrix to be a helm repository.
docs: rename docs.browsertrix.cloud -> docs.browsertrix.com
docs: update deployment doc to mention helm repo as preferred way to
install
docs build action: generate repository index in GH action
publish action: update auto-generated message to mention installing from
the repo.
---------
Co-authored-by: Tessa Walsh <tessa@bitarchivist.net>
Follow-up to #1689
Instead of just using resource types, using a combination of mime type, resource type and extension analysis.
Current logic is:
- if resourceType is script, stylesheet, image, font, use that definitely
- if resourceType is fetch/xhr, also check for extension, specifically for images
- categorize favicon.ico separately as 'favicon'
- also check for extension for json
- also check for pdf by type + extension
- categorize 'ping' as 'other'
- by default, use the first part of mime as the type
### Motivation
While using the browser default locale is often good enough, we probably
want more full control over locales, especially when we allow users to
choose their own locale — it's a poor experience to not have dates and
numbers formatted consistently with your chosen locale.
### Changes
Ensures (almost) all instances of `<sl-format-date>` as well as `Intl.*`
have the correct locale passed into them from our Lit localization
system.
The one exception here is in `frontend/src/utils/number.ts` where
ordinal suffixes aren't localized, so the locale is hardcoded to `en` —
I'll revisit this in the future.
