* WIP: No more empty outputs from modeling API
Part of https://github.com/KittyCAD/modeling-api/issues/518
* Remove unused import
* Keep Empty in the API
* Fix TS type error due to upgrade to ts_rs 10.0.0
* Fix warning about unused use
* Fix more type errors from ts_rs upgrade
* De-flake settings override desktop test
* Update export test file sizes
---------
Co-authored-by: Jonathan Tran <jonnytran@gmail.com>
Co-authored-by: Frank Noirot <frank@kittycad.io>
* Add the unexpected token to the error message
This is helpful during development where there's nothing to point at
the character index.
* Update more tests
Each AST node contains an Option<Digest> which can be optionally set by running compute_digest() over the AST. This will mutate the AST and set the Digest. Fundamentally, the digest is computed from the digest of AST children nodes (using compute_digest) or the explicit raw underlying value.
The underlying hash is changeable by modifying the macro and Digest type alias, and should enable us to determine when an AST -- or fragment of AST -- is the same.
This won't hash the start/end of the AST fragment, so it's, to some extent, position independent. It will hash newlines, comments, etc, which may change in the future depending on how we wind up using this.
* Make tag last optional param
* Update all test assertions with correct tag format
* Format ts
* Some progress on tests and code mods
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu)
* More sketch fixes
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu)
* Only 1 test left
* Clean up console.log
* A snapshot a day keeps the bugs away! 📷🐛 (OS: ubuntu)
* Fix last ts test
* Clean up fmt
* Fix clippy too
* Update docs and fix small oversight on angled lines
* Fix more rust tests
* Make typescript happy
* Fmt
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Jess Frazelle <jessfraz@users.noreply.github.com>
Part of https://github.com/KittyCAD/modeling-app/issues/1006#issuecomment-1816978586
This adds support for optional parameters to the AST. They are declared with a ? suffix, e.g. `(x, tag?) => {...}`.
This PR does not actually _use_ these optional parameters anywhere. In particular, it does not change the KCL stdlib or any existing function definitions. That will happen in a follow-up PR.
* AST function nodes no longer have stdlib function members
IMO the AST should not need an actual pointer to a stdlib function -- that is a completely separate concern from the AST.
Instead, the AST nodes can just store function names, and the executor will have a stdlib which it can look up those names in.
* Fix tests
* Update snapshot tests
* New parser built with Winnow
This new parser uses [winnow](docs.rs/winnow) to replace the handwritten recursive parser.
## Differences
I think the Winnow parser is more readable than handwritten one, due to reusing standard combinators. If you have a parsre like `p` or `q` you can combine them with standard functions like `repeat(0..4, p)`, `opt(p)`, `alt((p, q))` and `separated1(p, ", ")`. This IMO makes it more readable once you know what those standard functions do.
It's also more accurate now -- e.g. the parser no longer swallows whitespace between comments, or inserts it where there was none before. It no longer changes // comments to /* comments depending on the surrounding whitespace.
Primary form of testing was running the same KCL program through both the old and new parsers and asserting that both parsers produce the same AST. See the test `parser::parser_impl::tests::check_parsers_work_the_same`. But occasionally the new and old parsers disagree. This is either:
- Innocuous (e.g. disagreeing on whether a comment starts at the preceding whitespace or at the //)
- Helpful (e.g. new parser recognizes comments more accurately, preserving the difference between // and /* comments)
- Acceptably bad (e.g. new parser sometimes outputs worse error messages, TODO in #784)
so those KCL programs have their own unit tests in `parser_impl.rs` demonstrating the behaviour.
If you'd like to review this PR, it's arguably more important to review changes to the existing unit tests rather than the new parser itself. Because changes to the unit tests show where my parser changes behaviour -- usually for the better, occasionally for the worse (e.g. a worse error message than before). I think overall the improvements are worth it that I'd like to merge it without spending another week fixing it up -- we can fix the error messages in a follow-up PR.
## Performance
| Benchmark | Old parser (this branch) | New parser (this branch) | Speedup |
| ------------- | ------------- | ------------- | ------------- |
| Pipes on pipes | 922 ms | 42 ms | 21x |
| Kitt SVG | 148 ms | 7 ms | 21x |
There's definitely still room to improve performance:
- https://github.com/KittyCAD/modeling-app/issues/839
- https://github.com/KittyCAD/modeling-app/issues/840
## Winnow
Y'all know I love [Nom](docs.rs/nom) and I've blogged about it a lot. But I'm very happy using Winnow, a fork. It's got some really nice usability improvements. While writing this PR I found some bugs or unclear docs in Winnow:
- https://github.com/winnow-rs/winnow/issues/339
- https://github.com/winnow-rs/winnow/issues/341
- https://github.com/winnow-rs/winnow/issues/342
- https://github.com/winnow-rs/winnow/issues/344
The maintainer was quick to close them and release new versions within a few hours, so I feel very confident building the parser on this library. It's a clear improvement over Nom and it's used in toml-edit (and therefore within Cargo) and Gitoxide, so it's becoming a staple of the Rust ecosystem, which adds confidence.
Closes#716Closes#815Closes#599
* Basic CRA to Vite conversion
* Restore ESLint support
* Remove semicolons from vite config
* Add vite client types to tsconfig
* Migrate to Vitest for testing (not working on Mac)
* some test progress (#175)
* some test progress
* something maybe working
* remove local lib
* small clean up
* tweaks
* fix dependency
* clean up deps
* remove vitest import
* vitest config is needed even though we're not using vitest
* more tweaks to vite config
---------
Co-authored-by: Kurt Hutten <k.hutten@protonmail.ch>
* port recast to rust with massaged serialisation
* remove logs
* fix the last of white space test failures
* remove ts recastor
* clean up imports
* use serde serialise features
* unneeded async
* final clean up recast.ts
* more clean up tweaks
* improve Rust BinaryPart types
* Comments, fix warnings
* Run clippy --fix
* Remove unused variable
* serialise none_code_nodes manual to force strings to numbers
---------
Co-authored-by: Adam Chalmers <adam.s.chalmers@gmail.com>
* add segLen help to lang std
* adding helpers functions to sketchConstraints
* update tokeniser tests because they were annoying me not being 100%
* compare async lexer with sync lexer instead
* add helper functions
* remove unneeded nesting
* update add ast modifier function for angledLine
* initial equal ast modification
It adds a tag to the primary line, and converts any secondary lines to angledLine, but doesn't reference the taged/primary line yet
* Update fn call with refernce to previous line using segLen
* add test for giveSketchFnCallTag
* fix excutor bug, executing call expression in array expression
* fix small issue in executor
* add CallExpressions to BinaryExpressions
* add unary Expressions
* tweaks to unaryExpression logic
* add recasting for unaryExpressions and CallExpressions in BinaryExpressions
* ensure pipe substitution info is passed down to unary expressions and others
* allow binary expressions in function argumentns
* inital setup, new way of organising sketch fn transforms
Starting with equal length
* overhaul equalLength button
* add equal length support for angledLine
* line with one variable supports signed legLength
* fix indentation when recasting long arrayExpressions in a pipeExpression
* improve modifyAst consision
* further modify ast tidy
* equalLength transfroms far angledLineOfXLength
* add transforms for line-yRelative
* add equal constraint for angledLineOfYLength
* quick test fix
* add equal length constrain transforms for lineTo
* add equal length constraints for angledLineToX
* add equalLength constraints for angledLineToY
* test tidy
* setup new vertical-horizontal constraints
* Add equal Length constraints for vertical/horizontal lines
* migrate old tests, and refactor callback tag
* tweaks and refactor horzVert component
* fix leg len with small negative leg length
* initial tokeniser to wasm port
* fix tests
* add wasm to build script
* add rust tools to test action
* tweaks
* maybe tests will be happy
* tweak simple server
* trying to get tests to pass
* it pget vercel to build
* tidy up
* clean up rust files
* change lexer to use recursion so that it's consistent with the rest of the interpreter
* clean up nokeniser further
* rename variables
* readme typos
* run rust tests in CI
* follow clippy's advice
* more clippy clean up
* setup up proper serialzation to js-land
* tidy init promise in tests
* functional sketch working
With old sketch block still there
* get all version of lines working with add line and update line
* remove old ui state types
* some clean up
* rename some things
* add todo for multi cursor
* shorten useStore repitition
* small type improvement
* big overhaul to group sketch function and they ast modifying helpers together
* unneeded tweak
* ruthlessly rip out sketch logic
* clean up path keyword
* getting sketch on face working again with all the new sketch line types
* add a bunch of tests and re-arrage file structure
* Revert "wire up comments to ui (#11)"
This reverts commit bdf778530f.
* Revert "Add the ability to recast comments and some whitespace (#10)"
This reverts commit dd5022b38e.
* ast metadata
* clean up
* Add the ability to recast comments and some whitespace
Currently because whitespace or anything that's not needed for execution is not stored in the AST, it's hard to respect things like user formatting when recasting.
I think having a by-default-opinioned formatter is a good thing, but where this becomes problematic is when users wants to simply leave a blank space between some lines for a bit of breathing room, a code paragraph if you will, but maybe more importantly comments have not been implemented for the same reason, there wasn't a way with the current setup to insert them back in.
In some ways the most straightforward way to do this is to put whitespace and comments into the AST. Even though they are not crucial for execution, code-gen/recasting needs to be a first-class citizen in this lang so that's probably the long-term solution. However I'm trying to draw inspiration from other languages, and since it's not the norm to put comments et-al into the AST I haven't done so.
Because whitespace is tokenised already if not transformed into the AST, there is somewhat of a map of these things without going back to source code, so atm I'm experimenting with using this to insert extra linebreaks and comments back in between statements. I think this is a good compromise for the time being for what is a nice to have feature atm.
Because it's only going to respect non-code parts in between statements this will mean that you can't format objects or function params how you like (but I think this is good to have an opinioned fmt out of the box) and comments like myFunctionCall('a', /* inline comment */ b) will not work either.
* clean up
* Improved math expressions
Things are in a better state, + - / * work now for basic const var = 5 <operator> 1
Though the current method I'm using to make the ast isn't really going to work for dealing with precedence rules so some refactoring is needed going forward
* get complex math expressions working with precedence including parans
Node that identifiers are working, call expressions are not, that's a TODO
/ * % + - are working both other things like exponent and logical operators are also not working.
Recasting is the most important thing to implement next
* get recasting working for nested binary expressions
* clean up
