sua yoo
7a476e31ed
hotfix: localize msg string throwing error
...
Created https://github.com/webrecorder/browsertrix-cloud/issues/495 to catch issues before merge
2023-01-17 16:54:45 -08:00
sua yoo
b0745a63f7
Improve "Show Only Mine" switch visibility ( #494 )
2023-01-17 16:28:06 -08:00
sua yoo
ebb634a20d
Crawl config form panel UX enhancement & fix ( #489 )
2023-01-17 16:20:35 -08:00
sua yoo
c67af660ab
Fix localization build ( #488 )
...
* fix localization issues
* don't track translation files
2023-01-17 17:51:03 -05:00
sua yoo
d32428134a
Crawl config frontend fixes ( #482 )
...
* fix current time limit (#480 )
* fix browser profile name not shown (#474 )
* fix finish setup button label (#473 )
* fix paginated exclusion, border and page size (#475 )
2023-01-16 19:19:42 -08:00
sua yoo
6a30660728
Apply suggestions from code review
2023-01-16 10:08:45 -08:00
Henry Wilkinson
7e93ec57e0
Updates name (again) to actually reflect what this is
...
Because they _aren't_ additional seed URLs!
2023-01-13 17:13:42 -05:00
Henry Wilkinson
77a46aa2d9
Allowed URL Prefixes → Extra Start URLs
...
Changes name, hopefully people have an easier time understanding this now!
Will close #476
2023-01-13 16:50:00 -05:00
sua yoo
e3d34ff08f
Crawl config tag editor UI ( #422 )
...
Allow users to set tags on a crawl config. Resolves #362
2023-01-12 13:59:42 -08:00
sua yoo
52d9ae9661
fix skipping to confirm settings when duplicating config ( #454 )
2023-01-12 13:57:59 -08:00
sua yoo
a7457ca334
Filter crawls, configs, browser profiles by user ID ( #463 )
...
* filter crawls by user id
* filter crawl configs by user id
* text change: Filter My Crawls -> Show Only Mine
Co-authored-by: Ilya Kreymer <ikreymer@users.noreply.github.com>
2023-01-11 20:32:14 -08:00
Ilya Kreymer
30bda8c75d
VNC-Based Profile Browser ( #433 )
...
* profile browser vnc support + fixes:
- switch profile browser rendering to use VNC
- frontend: add @novnc/novnc as dependency, create separate bundle novnc.js to load into vnc browser (to avoid loading from each container)
- frontend: update proxy paths to proxy websocket, index page to crawler
- frontend: allow browser profiles in all browsers, remove browser compatibility check
- frontend: update webpack dev config, apply prettier
- frontend: node version fix
- backend: get vncpassword, build new URL for proxying to crawler iframe
- backend: fix profile / crawl job pull policy from 'Always' -> 'Never', should use existing image for job
- backend: fix kill signal to use bash -c to work with latest backend image
- backend/chart: add 'profile_browser_timeout_seconds' to chart values to control how long profile browser to remain when idle (default to 60)
- backend: remove utils.py, now using secret.token_hex() for random suffix
Co-authored-by: sua yoo <sua@suayoo.com>
2023-01-10 14:42:42 -08:00
sua yoo
303df2869c
Sticky the crawl config progress indicator position ( #445 )
...
* fix tabs on scroll
* adjust for smaller creen size
* scroll to top of section on change
* only scroll if needed
2023-01-09 19:58:33 -08:00
sua yoo
2dcf5cb36b
Always sub-navigation bar for selected team ( #444 )
...
* move nav
* highlight correct tab
2023-01-09 19:58:19 -08:00
sua yoo
c4fc3146b7
Fix app not loading on older Safari <15.4 ( #436 )
2023-01-04 15:09:57 -08:00
sua yoo
adc7ea6fe0
Persist currently selected team/archive ( #441 )
2023-01-04 15:09:23 -08:00
sua yoo
5daf550cb8
Frontend archives -> teams migration ( #429 )
2023-01-03 15:37:32 -08:00
sua yoo
d33d5f7700
Compute crawl config name from seed URLs ( #435 )
2023-01-03 15:07:21 -08:00
sua yoo
4a23dd12cb
Crawl config detail view & edit workflow UI updates ( #415 )
2022-12-22 09:37:43 -08:00
sua yoo
3b1db019c4
disable cache on replay web page ( #419 )
2022-12-21 10:48:37 -08:00
Ilya Kreymer
dfca09fc9c
Add single crawl info api at /crawls/{crawl_id} ( #418 )
...
* backend: crawl info apis:
- add /crawls/{crawl_id} api endpoint which just lists the crawl info, without resolving the individual files
- move /crawls/{crawl_id}.json -> /crawls/{crawl_id}/replay.json for clarity that it's used for replay
* frontend: update api for new replay.json endpoint
2022-12-19 14:54:48 -08:00
sua yoo
28346e0a54
New create crawl config user workflow ( #391 )
2022-12-12 13:50:33 -08:00
sua yoo
5d18ffa938
Fix language configuration UI ( #388 )
2022-12-01 10:02:13 -08:00
sua yoo
e7f1a00411
Fix authentication getting out of sync between tabs ( #380 )
...
Fixes regression to #361 found after increasing the token timeout by preventing app load until the authentication service is initialized (and finishing check if another tab is logged in.)
2022-11-23 23:36:36 -08:00
sua yoo
da8260a028
Make exclusion table cells editable ( #379 )
2022-11-23 09:43:52 -08:00
sua yoo
003b3c7a78
Allow users to set crawl config language ( #377 )
2022-11-22 15:15:35 -08:00
sua yoo
52e6b6199a
fix: prevent login event on storage event
2022-11-21 15:51:47 -08:00
sua yoo
321f78b861
Upgrade Shoelace 2.0.0-beta.61 -> 2.0.0-beta.83 ( #358 )
2022-11-21 08:16:51 -08:00
sua yoo
13d5ab7d2b
Fix exclusion dropdown & removal in crawl config edit ( #372 )
...
* fix dialog closing on regex
* fix unable to delete regex with same text
2022-11-18 14:47:40 -08:00
sua yoo
4d4ce40443
Refactor & sync user session across tab/windows ( #370 )
2022-11-15 19:49:18 -08:00
sua yoo
40054d1501
Update global footer ( #354 )
2022-11-15 14:15:04 -08:00
sua yoo
1ef9f7df6d
Fix auth not persisting on reload ( #360 )
2022-11-15 13:17:29 -08:00
sua yoo
8c4348b9f8
Show exclusion editor when creating & editing crawl templates ( #353 )
2022-11-14 19:34:15 -08:00
sua yoo
d41b582ef6
Remove exclusion from running crawl ( #352 )
2022-11-14 10:58:33 -08:00
sua yoo
95ec1599ef
Add exclusion to running crawl ( #347 )
2022-11-08 18:09:11 -06:00
sua yoo
baacbbdc52
Highlight regular expression syntax in Exclusions Table ( #341 )
2022-11-01 15:31:01 -07:00
sua yoo
97eb17784d
Display exclusions & list of URLs in crawl queue ( #337 )
...
- including pagination of queue results (30 results per page currently)
- show numbering on paginated results
- allow user navigation to each result page
2022-10-12 20:19:13 -07:00
sua yoo
8708c24a74
Improve crawl elapsed time UX ( #323 )
...
Smoother elapsed crawl timer:
- Crawls list: show seconds increment up to 2 minutes, then show minutes only
- Crawls detail: show seconds increment up to one day
2022-10-05 21:12:31 -04:00
sua yoo
0bbb7905bd
Add crawl queue editor UI components ( #331 )
...
WIP #304
2022-10-04 13:13:40 -07:00
sua yoo
e696104ffa
Update crawl template copy ( #325 )
2022-09-27 19:49:24 -07:00
sua yoo
709936dfa7
hotfix: decrease size of running crawl action button
2022-09-27 19:09:49 -07:00
sua yoo
63ada3e5b3
Update base fonts and text sizes ( #327 )
2022-09-27 14:32:57 -07:00
sua yoo
94e3dff27f
update sentry CDN script
2022-09-27 12:29:02 -07:00
sua yoo
20bd8ceecb
Fix browser profile table alignment ( #322 )
2022-09-26 17:14:08 -07:00
sua yoo
2ebd1eb2f6
Continue to watch crawl while stopping ( #316 )
...
* show when running
* redirect after done
* show banner that crawl is stopping
2022-09-21 12:39:00 -07:00
sua yoo
319a8a3c07
make clearer that profile selection is optional and that a default profile is used by default ( #290 )
...
- Rename 'Select Profile' -> 'Default Profile'
- Rename 'No Profiles' -> 'No Additional Profiles'
2022-08-10 15:54:39 -07:00
sua yoo
ee6161ad43
Frontend browser profile editor enhancements ( #288 )
...
- add button to duplicate profile from main view
- add save / cancel button when editing
- change location of 'full screen' button
2022-08-10 15:51:34 -07:00
sua yoo
9606d59c3d
Improve format of crawl template config error from server ( #281 )
...
* better display of api errors, such as fields missing or invalid urls, addresses #280
2022-06-29 17:57:03 -07:00
sua yoo
301b05ff4e
Refactor screencast websocket connection and retry ( #276 )
...
* replace ip with index and retry connection, fixes #252
2022-06-29 17:55:32 -07:00
Ilya Kreymer
2717a60763
improvements / bug fixes for stop/cancel handling: ( #279 )
...
- only send signal if stopping, no need for canceling as pods/containers will be removed
- refactor stop/cancel handling to be unified in manager, separate in job
- when stopping / graceful shutdown, return false if sending signal fails
- return success=true in json response if and only if stop/cancel actually succeeds, return 'error' message in error, should fix #270
- allow canceling after stopping / if stopping fails
- ensure finished time is set in case of cancelation before crawl starts, should fix #273
2022-06-29 17:47:25 -07:00
sua yoo
1c52902ea0
Update crawl scale label for UI consistency ( #275 )
...
closes #254
2022-06-29 16:14:03 -07:00
sua yoo
92292591ad
Re-run crawl from detail view + handle inactive crawl template ( #268 )
...
closes #253
2022-06-29 14:17:09 -07:00
sua yoo
d144591dbf
Display & edit crawl schedule in user local time ( #271 )
...
closes #255
2022-06-27 13:01:20 -07:00
sua yoo
c2aa4e6319
Fix AM/PM toggle ( #272 )
2022-06-23 16:35:47 -07:00
sua yoo
c2be1a27ce
Handle stopping state in UI ( #269 )
...
closes #262
2022-06-23 16:35:03 -07:00
sua yoo
b40765134c
Re-run crawl from crawls list view ( #264 )
...
* run crawl from crawls list, and show link to newly started crawl
* if crawl is already running, show link to previously running crawl
2022-06-15 18:54:57 -07:00
sua yoo
a8757e2e50
Screencast UX enhancements ( #251 )
...
* animate starting state
* consistent fixed-size slots for each browser (url + screencast)
* add tooltip for expected number of browsers (workers x scale)
2022-06-15 18:50:14 -07:00
Ilya Kreymer
9fce8cfc1d
frontend: fix missed renames
2022-06-11 12:37:24 -07:00
sua yoo
710639365b
adjust no files message ( #250 )
...
Change 'no files yet' -> 'no files to replay' when there are no files available for replay.
2022-06-07 22:59:34 -07:00
sua yoo
fa4b71288c
Fix watch crawl running state ( #249 )
2022-06-07 12:04:35 -07:00
sua yoo
502d687620
Enable duplicating and editing browser profile ( #237 )
...
* ensure editing other config options does not lose profile
* support adding/editing/removing profile of existing config
* when duplicating config, ensure profile setting is also copied in the duplicate
2022-06-04 08:26:19 -07:00
sua yoo
0c1dc2a1d1
Show crawl replay for running crawls ( #235 )
...
* show replay and watch at same time
* add separate section for watch
* only show replay if crawl has files, otherwise show 'no files' message
2022-06-04 08:19:09 -07:00
sua yoo
6a78bcd4aa
Delete browser profile ( #243 )
...
- delete browser profile, if not in use
- if in use, show error message, listing crawl configs that use the profile
- backend: fix check for confirming profile deletion
2022-06-01 19:18:41 -07:00
sua yoo
9cf1ed7d4d
copy yaml ( #239 )
2022-06-01 19:06:52 -07:00
sua yoo
2355de3067
docs: remove extra comment
2022-05-31 14:13:17 -07:00
sua yoo
6e19e854be
Fix "Run now" button ( #234 )
2022-05-30 16:15:10 -07:00
Ilya Kreymer
955197579e
frontend: support multi wacz replay using the crawl json as input
2022-05-20 09:11:23 -07:00
sua yoo
bda817dadd
View and edit browser profile ( #218 )
2022-04-23 20:12:16 -07:00
sua yoo
f157e2031f
Filter and sort crawl templates ( #217 )
2022-04-23 20:11:53 -07:00
sua yoo
cb80c6767e
hotfix: update profile ID in crawl template
2022-04-20 19:40:30 -07:00
Ilya Kreymer
38869cdd24
crawl templates: check that lastCrawlState is not null ( #220 )
2022-04-20 19:17:24 -07:00
sua yoo
db27b6aaaf
View and edit browser profile ( #214 )
2022-04-19 10:44:21 -07:00
sua yoo
71eec4d915
Create crawl template with browser profile ( #215 )
2022-04-18 10:36:28 -07:00
Ilya Kreymer
73b8c64ba4
frontend profile browser: cover devtools sidebar with profile sidebar, add try/catch for localStorage override
2022-04-13 21:41:51 -07:00
sua yoo
f5993e8ad8
Create browser profile UI ( #211 )
2022-04-13 21:11:13 -07:00
sua yoo
d2653ae835
View browser profiles in UI ( #209 )
2022-04-13 21:10:22 -07:00
sua yoo
238ee8f7ee
delete unused component file
2022-04-11 13:18:23 -07:00
sua yoo
8828681e8e
hotfix: fix crawl sort control alignment
2022-04-11 13:13:53 -07:00
sua yoo
d4b3ae3795
delete unused component file
2022-04-11 13:10:23 -07:00
sua yoo
5307138202
enable opening crawl template in new tab
2022-04-11 13:03:19 -07:00
sua yoo
f90ef071de
enable opening crawl in new tab
2022-04-11 13:03:10 -07:00
sua yoo
29b586b03f
Edit crawl config as YAML ( #207 )
2022-04-06 17:40:25 -07:00
Ilya Kreymer
9a6483630e
Support for Admin interface for viewing web archives ( #198 )
...
* backend api
- superadmin has admin access to all archives
- new superadmin endpoints: /archives/all/crawls and /archives/all/crawls/<crawl_id>.json for list all running crawls
and loading crawl data by id
- frontend superadmin view (fixes #201 )
* show all archives on superadmin home page
* show jump to crawl for super admin (#200 )
* navbar links for: all archives, all running crawls and jump to crawl
Co-authored-by: sua yoo <sua@suayoo.com>
2022-04-06 12:42:04 -07:00
sua yoo
ec3a77b71e
Mobile layout fixes ( #206 )
...
closes #202
2022-03-30 15:54:25 -07:00
sua yoo
9e2274f612
remove temp file
2022-03-30 13:51:02 -07:00
Ilya Kreymer
9e45dc35d2
minor frontend-tweaks: ( #196 )
...
* frontend-tweaks:
- treat 'starting' state same as 'running'
- default to no schedule instead of weekly for default
- add 'Domain' scopeType
* backend: also allow 'domain' as a scopeType
2022-03-15 21:19:23 -07:00
sua yoo
8863776c54
Define websocket host in common webpack config ( #195 )
...
* move websocket host var to common config, better fix for #193
2022-03-15 18:34:49 -07:00
Ilya Kreymer
912004751d
quickfix: partial mitigation for #193 , use current host for websock address
2022-03-14 15:29:35 -07:00
sua yoo
6fabea3e7a
Frontend build fixes ( #191 )
...
* copy specific files
* replace api host env var
* remove unused dotenv
* Update frontend/webpack.dev.js
Co-authored-by: Ilya Kreymer <ikreymer@users.noreply.github.com>
2022-03-10 23:26:21 -08:00
sua yoo
4190e40964
Show last crawl state in UI ( #192 )
...
* update crawl list status
* show on detail page
2022-03-10 23:25:42 -08:00
sua yoo
edf6b9ded7
Update home page routing ( #186 )
...
closes #183
2022-03-04 16:18:41 -08:00
sua yoo
0fe54653be
Fix unable to save edits to simple view ( #185 )
2022-03-04 16:17:57 -08:00
sua yoo
f2f67c34af
Copy extra hops value when duplicating crawl config ( #184 )
...
closes #158
2022-03-04 16:17:37 -08:00
sua yoo
4383c5e8d8
Disable error tracking in prod ( #182 )
...
closes #161
2022-03-04 16:17:05 -08:00
sua yoo
c18418ff09
Show invite message to super admin & layout fixes ( #181 )
2022-03-02 18:09:26 -08:00
sua yoo
fe31f551b2
Add "crawler" role to members ( #174 )
...
closes #139
2022-03-02 18:09:10 -08:00
sua yoo
c888a45d97
Fix seed URLs reset on JSON view toggle ( #172 )
...
closes #160
2022-03-02 18:08:45 -08:00
sua yoo
373c489b00
Watch crawl from crawl detail page ( #156 )
...
closes #164
closes #134
Co-authored-by: Ilya Kreymer <ikreymer@users.noreply.github.com>
2022-03-02 18:08:08 -08:00
sua yoo
83ded98081
Set and update crawl scale ( #162 )
...
closes #143
2022-02-28 09:14:27 -08:00
sua yoo
3fe3691e74
Update crawl run duration at intervals ( #155 )
...
fixes #138
2022-02-23 16:14:01 -08:00
