The idea behind this is to test all the various stages of executing KCL
separately, i.e.
- Start with a program
- Tokenize it
- Parse those tokens into an AST
- Recast the AST
- Execute the AST, outputting
- a PNG of the rendered model
- serialized program memory
Each of these steps reads some input and writes some output to disk.
The output of one step becomes the input to the next step. These
intermediate artifacts are also snapshotted (like expectorate or 2020)
to ensure we're aware of any changes to how KCL works. A change could
be a bug, or it could be harmless, or deliberate, but keeping it checked
into the repo means we can easily track changes.
Note: UUIDs sent back by the engine are currently nondeterministic, so
they would break all the snapshot tests. So, the snapshots use a regex
filter and replace anything that looks like a uuid with [uuid] when
writing program memory to a snapshot. In the future I hope our UUIDs will
be seedable and easy to make deterministic. At that point, we can stop
filtering the UUIDs.
We run this pipeline on many different KCL programs. Each keeps its
inputs (KCL programs), outputs (PNG, program memory snapshot) and
intermediate artifacts (AST, token lists, etc) in that directory.
I also added a new `just` command to easily generate these tests.
You can run `just new-sim-test gear $(cat gear.kcl)` to set up a new
gear test directory and generate all the intermediate artifacts for the
first time. This doesn't need any macros, it just appends some new lines
of normal Rust source code to `tests.rs`, so it's easy to see exactly
what the code is doing.
This uses `cargo insta` for convenient snapshot testing of artifacts
as JSON, and `twenty-twenty` for snapshotting PNGs.
This was heavily inspired by Predrag Gruevski's talk at EuroRust 2024
about deterministic simulation testing, and how it can both reduce bugs
and also reduce testing/CI time. Very grateful to him for chatting with
me about this over the last couple of weeks.
* fix auth test in engine
Signed-off-by: Jess Frazelle <github@jessfraz.com>
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* emoty
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
---------
Signed-off-by: Jess Frazelle <github@jessfraz.com>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Add lsystem.kcl to tests
* Reduce iterations
* Fix the user settings flake shit (NOTE TO ALL FUTURE PEOPLE MODELING-APP DOES NOT WAIT FOR I/O IN SOME CASES BEFORE ROUTER NAVIGATION)
* Fix just lint to check all targets
* Fix yarn test:rust to lint all targets
* Remove redundant options
* Change cargo --all to --workspace
* Update readme to use just command
Without this, you can get a funny error message, such as:
assert failed because 42 != 42: number is not 42
I'm assuming an epsilon of zero is never useful because we're dealing
with floating point.
* accessing toast error correctly
* wrapping try-catch around fs.stat on cli arg
* implemented array push
* changing arg execution order for sketch arc
* addressing sketchFromKclValue error for Sketches in Uservals
* addressing 'update to He inside a test not wrapped in act(...' error
* yarn fmt fix
* implemented polygon stdlib function
* changing polygon inscribed arg description in docs
* addressing cargo clippy warning
* Add tangential arc unavailable reason tooltip
* fixing tsc errors
* preventing hidden dirs from showing up as projects and prohibits renaming projects as hidden
* adding unit test for desktop listProjects
* showing no completions when last typed word is a number
* fmt
* Make clippy happy
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* yarn tsc fix: added missing toast import in Home.tsx
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* regenerating markdown docs for incoming merge from main
---------
Co-authored-by: arnav <arnav@agupta.org>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* Rename `homeMachine` and accessories to `projectsMachine`
* Separate out `/home` route from `projectsMachine`
* Add logic to navigate out from deleted or renamed project
* Show a warning in the command palette for deleting a project
* Make it navigate when you create a project
* Update "New project" button to use command bar flow
Closes#2585
* More explicit warning message text
* Make projects watching code not run in web
* Tests first version: nested loops
* Tests second version: flattened
* Remove console logs
* Fix tsc
* @jtran feedback, use the type guard util
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* Fix tests that relied on one-click, no-navigation project creation
* Revert "A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)"
This reverts commit 7545b61b49.
* Revert "A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)"
This reverts commit 3d2e48732c.
* Add a mask to the state indicator to client-side scale test
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* Fix lint
* Fix tsc
* Fix a couple stray tests that still relied on the old way of creating projects
* De-flake another text that could be thrown off by toast-based selectors
* FMT
* Dumb test error because I was rushing
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* Ahhh more flaky toasts, they're everywhere!
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* Re-run CI
* A snapshot a day keeps the bugs away! 📷🐛 (OS: windows-latest)
* Re-run CI
* Fix one test added since this PR was made
* Fix a few tests that failed due to changes since PR was made
* Prevent double selector issue in Ubuntu test
* Prevent *a different* double selector issue
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Tags can refer to either a surface, or a path. Tags track what they're
tagging with two fields: `Option<Path>` and `Option<Surface>`. This means
as more things can be tagged, we'll have a bunch of options and only one
can be Some at a given time. This should be an enum instead, so the
compiler can enforce that only one thing is being tagged.
Also I want to allow tags to refer to points, so I'd like to make this
enum before I add a new possible variant.
