Files
modeling-app/rust/kcl-lib/benches/compiler_benchmark_criterion.rs

31 lines
1.2 KiB
Rust
Raw Normal View History

use std::hint::black_box;
use criterion::{Criterion, criterion_group, criterion_main};
New parser built in Winnow (#731) * New parser built with Winnow This new parser uses [winnow](docs.rs/winnow) to replace the handwritten recursive parser. ## Differences I think the Winnow parser is more readable than handwritten one, due to reusing standard combinators. If you have a parsre like `p` or `q` you can combine them with standard functions like `repeat(0..4, p)`, `opt(p)`, `alt((p, q))` and `separated1(p, ", ")`. This IMO makes it more readable once you know what those standard functions do. It's also more accurate now -- e.g. the parser no longer swallows whitespace between comments, or inserts it where there was none before. It no longer changes // comments to /* comments depending on the surrounding whitespace. Primary form of testing was running the same KCL program through both the old and new parsers and asserting that both parsers produce the same AST. See the test `parser::parser_impl::tests::check_parsers_work_the_same`. But occasionally the new and old parsers disagree. This is either: - Innocuous (e.g. disagreeing on whether a comment starts at the preceding whitespace or at the //) - Helpful (e.g. new parser recognizes comments more accurately, preserving the difference between // and /* comments) - Acceptably bad (e.g. new parser sometimes outputs worse error messages, TODO in #784) so those KCL programs have their own unit tests in `parser_impl.rs` demonstrating the behaviour. If you'd like to review this PR, it's arguably more important to review changes to the existing unit tests rather than the new parser itself. Because changes to the unit tests show where my parser changes behaviour -- usually for the better, occasionally for the worse (e.g. a worse error message than before). I think overall the improvements are worth it that I'd like to merge it without spending another week fixing it up -- we can fix the error messages in a follow-up PR. ## Performance | Benchmark | Old parser (this branch) | New parser (this branch) | Speedup | | ------------- | ------------- | ------------- | ------------- | | Pipes on pipes | 922 ms | 42 ms | 21x | | Kitt SVG | 148 ms | 7 ms | 21x | There's definitely still room to improve performance: - https://github.com/KittyCAD/modeling-app/issues/839 - https://github.com/KittyCAD/modeling-app/issues/840 ## Winnow Y'all know I love [Nom](docs.rs/nom) and I've blogged about it a lot. But I'm very happy using Winnow, a fork. It's got some really nice usability improvements. While writing this PR I found some bugs or unclear docs in Winnow: - https://github.com/winnow-rs/winnow/issues/339 - https://github.com/winnow-rs/winnow/issues/341 - https://github.com/winnow-rs/winnow/issues/342 - https://github.com/winnow-rs/winnow/issues/344 The maintainer was quick to close them and release new versions within a few hours, so I feel very confident building the parser on this library. It's a clear improvement over Nom and it's used in toml-edit (and therefore within Cargo) and Gitoxide, so it's becoming a staple of the Rust ecosystem, which adds confidence. Closes #716 Closes #815 Closes #599
2023-10-12 09:42:37 -05:00
pub fn bench_parse(c: &mut Criterion) {
for (name, file) in [
("pipes_on_pipes", PIPES_PROGRAM),
("big_kitt", KITT_PROGRAM),
("cube", CUBE_PROGRAM),
("math", MATH_PROGRAM),
("mike_stress_test", MIKE_STRESS_TEST_PROGRAM),
("koch snowflake", LSYSTEM_KOCH_SNOWFLAKE_PROGRAM),
New parser built in Winnow (#731) * New parser built with Winnow This new parser uses [winnow](docs.rs/winnow) to replace the handwritten recursive parser. ## Differences I think the Winnow parser is more readable than handwritten one, due to reusing standard combinators. If you have a parsre like `p` or `q` you can combine them with standard functions like `repeat(0..4, p)`, `opt(p)`, `alt((p, q))` and `separated1(p, ", ")`. This IMO makes it more readable once you know what those standard functions do. It's also more accurate now -- e.g. the parser no longer swallows whitespace between comments, or inserts it where there was none before. It no longer changes // comments to /* comments depending on the surrounding whitespace. Primary form of testing was running the same KCL program through both the old and new parsers and asserting that both parsers produce the same AST. See the test `parser::parser_impl::tests::check_parsers_work_the_same`. But occasionally the new and old parsers disagree. This is either: - Innocuous (e.g. disagreeing on whether a comment starts at the preceding whitespace or at the //) - Helpful (e.g. new parser recognizes comments more accurately, preserving the difference between // and /* comments) - Acceptably bad (e.g. new parser sometimes outputs worse error messages, TODO in #784) so those KCL programs have their own unit tests in `parser_impl.rs` demonstrating the behaviour. If you'd like to review this PR, it's arguably more important to review changes to the existing unit tests rather than the new parser itself. Because changes to the unit tests show where my parser changes behaviour -- usually for the better, occasionally for the worse (e.g. a worse error message than before). I think overall the improvements are worth it that I'd like to merge it without spending another week fixing it up -- we can fix the error messages in a follow-up PR. ## Performance | Benchmark | Old parser (this branch) | New parser (this branch) | Speedup | | ------------- | ------------- | ------------- | ------------- | | Pipes on pipes | 922 ms | 42 ms | 21x | | Kitt SVG | 148 ms | 7 ms | 21x | There's definitely still room to improve performance: - https://github.com/KittyCAD/modeling-app/issues/839 - https://github.com/KittyCAD/modeling-app/issues/840 ## Winnow Y'all know I love [Nom](docs.rs/nom) and I've blogged about it a lot. But I'm very happy using Winnow, a fork. It's got some really nice usability improvements. While writing this PR I found some bugs or unclear docs in Winnow: - https://github.com/winnow-rs/winnow/issues/339 - https://github.com/winnow-rs/winnow/issues/341 - https://github.com/winnow-rs/winnow/issues/342 - https://github.com/winnow-rs/winnow/issues/344 The maintainer was quick to close them and release new versions within a few hours, so I feel very confident building the parser on this library. It's a clear improvement over Nom and it's used in toml-edit (and therefore within Cargo) and Gitoxide, so it's becoming a staple of the Rust ecosystem, which adds confidence. Closes #716 Closes #815 Closes #599
2023-10-12 09:42:37 -05:00
] {
c.bench_function(&format!("parse_{name}"), move |b| {
b.iter(move || {
black_box(kcl_lib::Program::parse(file).unwrap());
New parser built in Winnow (#731) * New parser built with Winnow This new parser uses [winnow](docs.rs/winnow) to replace the handwritten recursive parser. ## Differences I think the Winnow parser is more readable than handwritten one, due to reusing standard combinators. If you have a parsre like `p` or `q` you can combine them with standard functions like `repeat(0..4, p)`, `opt(p)`, `alt((p, q))` and `separated1(p, ", ")`. This IMO makes it more readable once you know what those standard functions do. It's also more accurate now -- e.g. the parser no longer swallows whitespace between comments, or inserts it where there was none before. It no longer changes // comments to /* comments depending on the surrounding whitespace. Primary form of testing was running the same KCL program through both the old and new parsers and asserting that both parsers produce the same AST. See the test `parser::parser_impl::tests::check_parsers_work_the_same`. But occasionally the new and old parsers disagree. This is either: - Innocuous (e.g. disagreeing on whether a comment starts at the preceding whitespace or at the //) - Helpful (e.g. new parser recognizes comments more accurately, preserving the difference between // and /* comments) - Acceptably bad (e.g. new parser sometimes outputs worse error messages, TODO in #784) so those KCL programs have their own unit tests in `parser_impl.rs` demonstrating the behaviour. If you'd like to review this PR, it's arguably more important to review changes to the existing unit tests rather than the new parser itself. Because changes to the unit tests show where my parser changes behaviour -- usually for the better, occasionally for the worse (e.g. a worse error message than before). I think overall the improvements are worth it that I'd like to merge it without spending another week fixing it up -- we can fix the error messages in a follow-up PR. ## Performance | Benchmark | Old parser (this branch) | New parser (this branch) | Speedup | | ------------- | ------------- | ------------- | ------------- | | Pipes on pipes | 922 ms | 42 ms | 21x | | Kitt SVG | 148 ms | 7 ms | 21x | There's definitely still room to improve performance: - https://github.com/KittyCAD/modeling-app/issues/839 - https://github.com/KittyCAD/modeling-app/issues/840 ## Winnow Y'all know I love [Nom](docs.rs/nom) and I've blogged about it a lot. But I'm very happy using Winnow, a fork. It's got some really nice usability improvements. While writing this PR I found some bugs or unclear docs in Winnow: - https://github.com/winnow-rs/winnow/issues/339 - https://github.com/winnow-rs/winnow/issues/341 - https://github.com/winnow-rs/winnow/issues/342 - https://github.com/winnow-rs/winnow/issues/344 The maintainer was quick to close them and release new versions within a few hours, so I feel very confident building the parser on this library. It's a clear improvement over Nom and it's used in toml-edit (and therefore within Cargo) and Gitoxide, so it's becoming a staple of the Rust ecosystem, which adds confidence. Closes #716 Closes #815 Closes #599
2023-10-12 09:42:37 -05:00
})
});
}
}
criterion_group!(benches, bench_parse);
criterion_main!(benches);
Move the wasm lib, and cleanup rust directory and all references (#5585) * git mv src/wasm-lib rust Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv wasm-lib to workspace Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv kcl-lib Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv derive docs Signed-off-by: Jess Frazelle <github@jessfraz.com> * resolve file paths Signed-off-by: Jess Frazelle <github@jessfraz.com> * clippy Signed-off-by: Jess Frazelle <github@jessfraz.com> * move more shit Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix more paths Signed-off-by: Jess Frazelle <github@jessfraz.com> * make yarn build:wasm work Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix scripts Signed-off-by: Jess Frazelle <github@jessfraz.com> * fixups Signed-off-by: Jess Frazelle <github@jessfraz.com> * better references Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix cargo ci Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix reference Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix more ci Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix tests Signed-off-by: Jess Frazelle <github@jessfraz.com> * cargo sort Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix script Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix Signed-off-by: Jess Frazelle <github@jessfraz.com> * fmt Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix a dep Signed-off-by: Jess Frazelle <github@jessfraz.com> * sort Signed-off-by: Jess Frazelle <github@jessfraz.com> * remove unused deps Signed-off-by: Jess Frazelle <github@jessfraz.com> * Revert "remove unused deps" This reverts commit fbabdb062e275fd5cbc1476f8480a1afee15d972. * updates Signed-off-by: Jess Frazelle <github@jessfraz.com> * deps; Signed-off-by: Jess Frazelle <github@jessfraz.com> * fixes Signed-off-by: Jess Frazelle <github@jessfraz.com> * updates Signed-off-by: Jess Frazelle <github@jessfraz.com> --------- Signed-off-by: Jess Frazelle <github@jessfraz.com>
2025-03-01 13:59:01 -08:00
const KITT_PROGRAM: &str = include_str!("../e2e/executor/inputs/kittycad_svg.kcl");
const PIPES_PROGRAM: &str = include_str!("../e2e/executor/inputs/pipes_on_pipes.kcl");
const CUBE_PROGRAM: &str = include_str!("../e2e/executor/inputs/cube.kcl");
const MATH_PROGRAM: &str = include_str!("../e2e/executor/inputs/math.kcl");
const MIKE_STRESS_TEST_PROGRAM: &str = include_str!("../tests/mike_stress_test/input.kcl");
Move the wasm lib, and cleanup rust directory and all references (#5585) * git mv src/wasm-lib rust Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv wasm-lib to workspace Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv kcl-lib Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv derive docs Signed-off-by: Jess Frazelle <github@jessfraz.com> * resolve file paths Signed-off-by: Jess Frazelle <github@jessfraz.com> * clippy Signed-off-by: Jess Frazelle <github@jessfraz.com> * move more shit Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix more paths Signed-off-by: Jess Frazelle <github@jessfraz.com> * make yarn build:wasm work Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix scripts Signed-off-by: Jess Frazelle <github@jessfraz.com> * fixups Signed-off-by: Jess Frazelle <github@jessfraz.com> * better references Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix cargo ci Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix reference Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix more ci Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix tests Signed-off-by: Jess Frazelle <github@jessfraz.com> * cargo sort Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix script Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix Signed-off-by: Jess Frazelle <github@jessfraz.com> * fmt Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix a dep Signed-off-by: Jess Frazelle <github@jessfraz.com> * sort Signed-off-by: Jess Frazelle <github@jessfraz.com> * remove unused deps Signed-off-by: Jess Frazelle <github@jessfraz.com> * Revert "remove unused deps" This reverts commit fbabdb062e275fd5cbc1476f8480a1afee15d972. * updates Signed-off-by: Jess Frazelle <github@jessfraz.com> * deps; Signed-off-by: Jess Frazelle <github@jessfraz.com> * fixes Signed-off-by: Jess Frazelle <github@jessfraz.com> * updates Signed-off-by: Jess Frazelle <github@jessfraz.com> --------- Signed-off-by: Jess Frazelle <github@jessfraz.com>
2025-03-01 13:59:01 -08:00
const LSYSTEM_KOCH_SNOWFLAKE_PROGRAM: &str = include_str!("../e2e/executor/inputs/lsystem.kcl");