The idea behind this is to test all the various stages of executing KCL
separately, i.e.
- Start with a program
- Tokenize it
- Parse those tokens into an AST
- Recast the AST
- Execute the AST, outputting
- a PNG of the rendered model
- serialized program memory
Each of these steps reads some input and writes some output to disk.
The output of one step becomes the input to the next step. These
intermediate artifacts are also snapshotted (like expectorate or 2020)
to ensure we're aware of any changes to how KCL works. A change could
be a bug, or it could be harmless, or deliberate, but keeping it checked
into the repo means we can easily track changes.
Note: UUIDs sent back by the engine are currently nondeterministic, so
they would break all the snapshot tests. So, the snapshots use a regex
filter and replace anything that looks like a uuid with [uuid] when
writing program memory to a snapshot. In the future I hope our UUIDs will
be seedable and easy to make deterministic. At that point, we can stop
filtering the UUIDs.
We run this pipeline on many different KCL programs. Each keeps its
inputs (KCL programs), outputs (PNG, program memory snapshot) and
intermediate artifacts (AST, token lists, etc) in that directory.
I also added a new `just` command to easily generate these tests.
You can run `just new-sim-test gear $(cat gear.kcl)` to set up a new
gear test directory and generate all the intermediate artifacts for the
first time. This doesn't need any macros, it just appends some new lines
of normal Rust source code to `tests.rs`, so it's easy to see exactly
what the code is doing.
This uses `cargo insta` for convenient snapshot testing of artifacts
as JSON, and `twenty-twenty` for snapshotting PNGs.
This was heavily inspired by Predrag Gruevski's talk at EuroRust 2024
about deterministic simulation testing, and how it can both reduce bugs
and also reduce testing/CI time. Very grateful to him for chatting with
me about this over the last couple of weeks.
* fix auth test in engine
Signed-off-by: Jess Frazelle <github@jessfraz.com>
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* emoty
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
---------
Signed-off-by: Jess Frazelle <github@jessfraz.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Add lsystem.kcl to tests
* Reduce iterations
* Fix the user settings flake shit (NOTE TO ALL FUTURE PEOPLE MODELING-APP DOES NOT WAIT FOR I/O IN SOME CASES BEFORE ROUTER NAVIGATION)
* Fix just lint to check all targets
* Fix yarn test:rust to lint all targets
* Remove redundant options
* Change cargo --all to --workspace
* Update readme to use just command
Without this, you can get a funny error message, such as:
assert failed because 42 != 42: number is not 42
I'm assuming an epsilon of zero is never useful because we're dealing
with floating point.
* accessing toast error correctly
* wrapping try-catch around fs.stat on cli arg
* implemented array push
* changing arg execution order for sketch arc
* addressing sketchFromKclValue error for Sketches in Uservals
* addressing 'update to He inside a test not wrapped in act(...' error
* yarn fmt fix
* implemented polygon stdlib function
* changing polygon inscribed arg description in docs
* addressing cargo clippy warning
* Add tangential arc unavailable reason tooltip
* fixing tsc errors
* preventing hidden dirs from showing up as projects and prohibits renaming projects as hidden
* adding unit test for desktop listProjects
* showing no completions when last typed word is a number
* fmt
* Make clippy happy
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* yarn tsc fix: added missing toast import in Home.tsx
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* regenerating markdown docs for incoming merge from main
---------
Co-authored-by: arnav <arnav@agupta.org>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Rename `homeMachine` and accessories to `projectsMachine`
* Separate out `/home` route from `projectsMachine`
* Add logic to navigate out from deleted or renamed project
* Show a warning in the command palette for deleting a project
* Make it navigate when you create a project
* Update "New project" button to use command bar flow
Closes#2585
* More explicit warning message text
* Make projects watching code not run in web
* Tests first version: nested loops
* Tests second version: flattened
* Remove console logs
* Fix tsc
* @jtran feedback, use the type guard util
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* Fix tests that relied on one-click, no-navigation project creation
* Revert "A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)"
This reverts commit 7545b61b49.
* Revert "A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)"
This reverts commit 3d2e48732c.
* Add a mask to the state indicator to client-side scale test
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* Fix lint
* Fix tsc
* Fix a couple stray tests that still relied on the old way of creating projects
* De-flake another text that could be thrown off by toast-based selectors
* FMT
* Dumb test error because I was rushing
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* Ahhh more flaky toasts, they're everywhere!
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* Re-run CI
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* Re-run CI
* Fix one test added since this PR was made
* Fix a few tests that failed due to changes since PR was made
* Prevent double selector issue in Ubuntu test
* Prevent *a different* double selector issue
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Tags can refer to either a surface, or a path. Tags track what they're
tagging with two fields: `Option<Path>` and `Option<Surface>`. This means
as more things can be tagged, we'll have a bunch of options and only one
can be Some at a given time. This should be an enum instead, so the
compiler can enforce that only one thing is being tagged.
Also I want to allow tags to refer to points, so I'd like to make this
enum before I add a new possible variant.
* Add a radius length indicator to the circle sketch tool
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* Revert "A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)"
This reverts commit 15b078f641.
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: 49fl <ircsurfer33@gmail.com>
See "Problem 2" in https://github.com/KittyCAD/modeling-app/issues/4297
This is a pure refactor, it should not change any behaviour at all.
It adds more information into the tag system, but nothing reads that
extra information yet. It will be used to address problem 3 of the above
issue.
* Update CameraControls.ts
* fix static analyzer error
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* Revert "A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)"
This reverts commit 0b63016217.
* Don't perform sketch click operations if a camera movement interaction also matches
* Don't `stopPropogation`, make selection listener early return if `wasDragging`
+ consolidate `wasDragging` set statements, add comments
* Codespell
---------
Co-authored-by: 49fl <ircsurfer33@gmail.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Frank Noirot <frank@kittycad.io>
Co-authored-by: Frank Noirot <frank@zoo.dev>
* Make electron test setting overrides not entirely replace default settings
* Add failing test
* Fix test by checking for healthy engine connection before executing demo code
* Fix one electron test that assumed all settings got wiped if you override any.
* 🤷🏻♂️ an engine-side camera position in one of the E2E tests changed by 0.01 randomly
* Buffer file writes, because writing to file after every keypress does bad things
* fix: kevin -- added timeouts for the time being since the workflow for saving data to disk is now changed
---------
Co-authored-by: Kevin Nadro <kevin@zoo.dev>
* Fix test settings to actually get used
Co-authored-by: Frank Noirot <frank@zoo.dev>
* Export file size changed out from under us again, relax this test to just be above a reasonable size
* Missed on updated export expectation
* Wrong check, should just be greater than
* Fix E2E test, remove console log
* fmt
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* Sketchy rectangle commit fix
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* Re-run CI
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* Revert "A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)"
This reverts commit 2ace7a3b0e.
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* Bump timeouts for snapshots
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* Re-run CI
---------
Co-authored-by: Frank Noirot <frank@zoo.dev>
Co-authored-by: Frank Noirot <frank@kittycad.io>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
