15 Commits

Author SHA1 Message Date
55e1ec7dad Rename desktop e2e scripts and tags for consistency (#7240)
* Rename desktop e2e scripts and tags for consistency

* Show local command in main test step

* Restore 'e2e' prefix to clarify GitHub UI

* Add web script to contributor guide
2025-05-29 09:29:03 -04:00
1dafbf105e Identify distinct test suites (#7109) 2025-05-20 12:56:55 -04:00
17b1120a27 Include metadata environment variables with results (#6838)
* Include metadata environment variables with results

* Share updated report after uploading each shard

* Hide raw curl progress
2025-05-11 21:40:11 -04:00
92f7a62af2 Create JUnit reports from Rust tests (#6743) 2025-05-11 00:49:55 -04:00
2978b34b7b Skip TAB-disabled tests locally by default (#6601) 2025-04-30 15:32:37 -04:00
f00ea4cf5e Use TAB API to determine when merges are blocked (#6405)
* Use TAB API to determine when merges are blocked

* Enable a test that is known to fail
2025-04-22 09:02:02 -04:00
167445ef8a WIP: migrate from yarn v1 to npm (#6274)
* WIP: migrate from yarn v1 to npm

* Add tsc mapping script

* More fixes

* Add playwright as script

* Fix more

* more and more people just want more

* Merge branch 'main' into pierremtb/adhoc/npm

* use workspaces and fix circular deps

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* fmt

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* fix

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* New machine-api types

* better lock file

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* ignore typos in machine-api generated files

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* updates

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* install from root

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* fix tsc

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* New machine-api types

* bettter install

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* bettter install

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* bettter install

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* bettter install

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* bettter install

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* add comment

Signed-off-by: Jess Frazelle <github@jessfraz.com>

* add comment

Signed-off-by: Jess Frazelle <github@jessfraz.com>

---------

Signed-off-by: Jess Frazelle <github@jessfraz.com>
Co-authored-by: Jess Frazelle <jessfraz@users.noreply.github.com>
Co-authored-by: Jess Frazelle <github@jessfraz.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2025-04-12 02:49:49 +00:00
962eb0e376 Install and start Vector on macOS CI runners (#6147)
* Install vector on macOS

* Include platform information
2025-04-07 20:22:27 +00:00
2c6404f671 Continue running broken tests for Axiom metrics (#5883)
* Install Vector on Ubuntu to log failed test to Axiom

* Allow flaky tests to run on main for Axiom metrics

* Enable problematic tests on a dedicated branch
2025-03-20 16:28:08 -04:00
388371b05d External retry of the playwright step instead of --last-failed (#5465)
* External retry

* Trigger CI

* More external attemps, less internal retries

* Trigger CI

* Trigger CI

* Clean up for PR

* Add --retries=2 to cargo test
2025-02-22 12:59:19 -05:00
96174f3cf6 Increase playwright retries to 5 (#5000) 2025-01-10 13:34:27 -05:00
02055a8b31 Move all tests over to electron (#4484)
* Move all tests over to electron

* Pass the correct param to playwright-electron.sh

* Add shebang to script and add macos-14-large as a target

* Get sketch-tests.spec.ts passing in electron

* Try out 4 workers

* Got testing-segment-overlays passing

* Pass testing-selections.spec.ts

* Go back to fix up sketch-tests test

* Pass various.spec.ts, by far the hardest one

* Pass can-sketch-on-all-planes... with ease

* Pass command bar tests

* fmt

* Completely fix code mirror text navigating for tests

* Pass debug pane tests

* Pass desktop export tests

* Pass editor tests

* Pass file tree tests

* Pass onboarding tests

* Corrected a fixme in file-tree.spec!

* Painfully fix hardcoded coordinates in point-click.spec

* Pass machine.spec tests

* Pass projects, fought hard with filechooser

* Pass regresion-tests.spec tests

* Pass network and connection tests

* Pass camera-movement.spec tests

* Extreme time eaten by gizmo test fixes. All passing now.

* Merge main (tests changed x_x) and pass all constraints.spec tests (pain)

* Pass another painful spec suite: testing-settings

* Pass perspective-toggle, interesting note

* Pass samples loading tests

* Pass app header tests

* Pass text-to-cad tests

* Pass segment-overlays (minor ache) and ability to switch to web if needed :)

* Fix a ton of syntax changes and deflake 2 more tests (pain)

* Correct all tsc errors

* Remove to-electron script

* Add an f-ton of shit because playwright doesnt want S P R E A D

* Try CI again

* Stop snapshots of exports (already test in e2e)

* Fix flake in double click editor

* Hopefully help CI flake

* Fixmes, fixmes everywhere

* One more fixme to settings

* Skip another code pane flake

* Port jess's projects.spec tests

* fixup

* Reuse electron window; difficult task

* Rebased and refixed

* Remove duplicate cases

* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest-8-cores)

* Reduce the workers to something CI can handle

* Lower it further, we need to think about the others

* Update package.json

Co-authored-by: Pierre Jacquier <pierrejacquier39@gmail.com>

* Update package.json

Co-authored-by: Pierre Jacquier <pierrejacquier39@gmail.com>

* Fix the last tests and tsc errors

* Timeout to 120 and windows-2022-16core

* Fix windows runner detection, enable concurrency temporarily

* Hopefully this time fix windows runner detection

* Comment out Vector, add back removed camera test code

* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest-8-cores)

* A snapshot a day keeps the bugs away! 📷🐛 (OS: macos-14-large)

* A snapshot a day keeps the bugs away! 📷🐛 (OS: macos-14-large)

* Fix camera tests again

* Massively deflake a whole class of tests

* A snapshot a day keeps the bugs away! 📷🐛 (OS: macos-14-large)

* A snapshot a day keeps the bugs away! 📷🐛 (OS: macos-14-large)

* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest-8-cores)

* Try new CI and fix small onboarding test

* Derp

* No github tuning

* Try mac

* Add back all the OS

* Lord, hallow be thy name

* A snapshot a day keeps the bugs away! 📷🐛 (OS: namespace-profile-ubuntu-8-cores)

* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest-8-cores)

* One last try with window-16-cores

* Trigger CI

* Try AWS Windows runner

* Passing on windows locally with a few skips

* Skip more win tests, add back all three oses

* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-16-cores)

* Add two more fixmes

* 2 more fixmes

* skip segment overlays on win32

* Another fixme

* Trigger CI

* Trigger CI

* Quick clean up

* Move all tests over to electron

* Pass the correct param to playwright-electron.sh

* Add shebang to script and add macos-14-large as a target

* Get sketch-tests.spec.ts passing in electron

* Try out 4 workers

* Got testing-segment-overlays passing

* Pass testing-selections.spec.ts

* Go back to fix up sketch-tests test

* Pass various.spec.ts, by far the hardest one

* Pass can-sketch-on-all-planes... with ease

* Pass command bar tests

* fmt

* Completely fix code mirror text navigating for tests

* Pass debug pane tests

* Pass desktop export tests

* Pass editor tests

* Pass file tree tests

* Pass onboarding tests

* Corrected a fixme in file-tree.spec!

* Painfully fix hardcoded coordinates in point-click.spec

* Pass machine.spec tests

* Pass projects, fought hard with filechooser

* Pass regresion-tests.spec tests

* Pass network and connection tests

* Pass camera-movement.spec tests

* Extreme time eaten by gizmo test fixes. All passing now.

* Merge main (tests changed x_x) and pass all constraints.spec tests (pain)

* Pass another painful spec suite: testing-settings

* Pass perspective-toggle, interesting note

* Pass samples loading tests

* Pass app header tests

* Pass text-to-cad tests

* Pass segment-overlays (minor ache) and ability to switch to web if needed :)

* Fix a ton of syntax changes and deflake 2 more tests (pain)

* Correct all tsc errors

* Remove to-electron script

* Add an f-ton of shit because playwright doesnt want S P R E A D

* Try CI again

* Stop snapshots of exports (already test in e2e)

* Fix flake in double click editor

* Hopefully help CI flake

* Fixmes, fixmes everywhere

* One more fixme to settings

* Skip another code pane flake

* Port jess's projects.spec tests

* fixup

* Reuse electron window; difficult task

* Rebased and refixed

* Remove duplicate cases

* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest-8-cores)

* Reduce the workers to something CI can handle

* Lower it further, we need to think about the others

* Update package.json

Co-authored-by: Pierre Jacquier <pierrejacquier39@gmail.com>

* Update package.json

Co-authored-by: Pierre Jacquier <pierrejacquier39@gmail.com>

* Fix the last tests and tsc errors

* Timeout to 120 and windows-2022-16core

* Fix windows runner detection, enable concurrency temporarily

* Hopefully this time fix windows runner detection

* Comment out Vector, add back removed camera test code

* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest-8-cores)

* A snapshot a day keeps the bugs away! 📷🐛 (OS: macos-14-large)

* A snapshot a day keeps the bugs away! 📷🐛 (OS: macos-14-large)

* Fix camera tests again

* Massively deflake a whole class of tests

* A snapshot a day keeps the bugs away! 📷🐛 (OS: macos-14-large)

* A snapshot a day keeps the bugs away! 📷🐛 (OS: macos-14-large)

* Try new CI and fix small onboarding test

* Derp

* No github tuning

* Try mac

* Add back all the OS

* Lord, hallow be thy name

* A snapshot a day keeps the bugs away! 📷🐛 (OS: namespace-profile-ubuntu-8-cores)

* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest-8-cores)

* Try AWS Windows runner

* Passing on windows locally with a few skips

* Trigger CI

* A snapshot a day keeps the bugs away! 📷🐛 (OS: namespace-profile-ubuntu-8-cores)

* fmt, tsc, lint

* Enable two fixmes again

* Fix lint, codespell, fmt

* Fix lint

* Don't run e2e on draft, add back concurrency, clean up

* One last windows skip

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Pierre Jacquier <pierrejacquier39@gmail.com>
Co-authored-by: Pierre Jacquier <pierre@zoo.dev>
2024-12-18 17:58:03 -05:00
2abfbb9788 Move tests from no_visuals into simulation tests (#4367)
Now you can properly inspect program memory for the no_visuals tests, instead of relying on a lot of KCL asserts.
2024-11-05 02:34:22 +00:00
11dfd87240 adhoc/Bug fixing the last-failed retries across jobs (#4086)
* chore: adding Kurt's fix for electron in the new CI CD files. Adding a forced failure to test

* chore: increasing max retry for electron to match the playwright browser retry count

* fix: debugging ci cd for playwright last report

* fix: changing the output dir for snapshot to a custom one to not overwrite the previous job runs failure

* fix: found out hidden files are excluded automatically, was a breaking change :(

* fix: output typo

* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)

* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)

* Delete test-results-snapshots/.last-run.json

* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)

* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)

* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)

* fix: cleanup

* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)

* Delete test-results-snapshots/.last-run.json

* fix: removing this folder, should have been git ignored

* fix: do not need these anymore since the hidden files is fixed

* fix: removed hard coded failure for debugging

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-03 20:09:30 -05:00
e1406012b4 Nadro/4012/ci cd update (#4062)
* chore: saving off package.json progress unit tests fail in main

* fix: implementing a one liner for unit tests

* fix: renaming test:unit:local

* chore: adding playwright tests

* fix: making package.json not destructive to keep same pipeline commands for now

* fix: reordering

* fix: added tags for OS tests, moved kill-port to dev depen

* fix: OS skipping at tag level

* fix: lint, fmt, tsc, etc...

* Look at this (photo)Graph *in the voice of Nickelback*

* fix: new formatting

* fix: removing the ci copy, do not like it

* Look at this (photo)Graph *in the voice of Nickelback*

* chore: updating readme with explanation on the commands for CI CD simulation locally

* fix: package.json command for unit test, removing cached breaking cache in unit tests

* fix: fixing copy and typos in README.md for CI CD section

* fix: adding a duplicate command for a better name. CI CD will use it in a future PR

* chore: trying to clean up the copy and commands for CI CD tests

* chore: porting the bash code in the YAML to a bash script then using matrix permutations to control the runtime

* fix: typos

* fix: another typo, missed these went porting to the bash script logic

* fix: I think I need the checkout action since it has the repo code?

* fix: wrote absolute path not the relative hidden path, ope

* fix: does this cache give me the yarn install of playwright?

* fix: yarn cannot find the binary, use the yarn command

* fix: remove all uses...?

* chore: adding bash script for electron runtimes

* fix: copy cleanup

* fix: typo when copy and pasting the exclude logic, ope

* fix: this is wrong

* fix: build:wasm is a requirement for yarn tsc

* fix: reorder?

* fix: renaming integrations to e2e

* fix: windows is complaining about a pipe issue in the bash script?

* fix: escaping double quotes in windows?

* chore: consolidating commands into 1 file and easier YAML configuation for electron

* chore: mapped multiple OS playwright browser into a single bash script

* fix: removing old bash scripts, renaming matrix jobs

* fix: missed deleting this when I added the if statements.

* chore: removing unused job, xstate typegen more more since v5

* fix: trying to get these two tests to pass on first try

* fix: auto fixes

* fix: removing old unit test command

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
2024-10-03 09:40:12 -05:00