Files
modeling-app/rust/justfile

76 lines
4.0 KiB
Makefile
Raw Permalink Normal View History

KCL: New simulation test pipeline (#4351) The idea behind this is to test all the various stages of executing KCL separately, i.e. - Start with a program - Tokenize it - Parse those tokens into an AST - Recast the AST - Execute the AST, outputting - a PNG of the rendered model - serialized program memory Each of these steps reads some input and writes some output to disk. The output of one step becomes the input to the next step. These intermediate artifacts are also snapshotted (like expectorate or 2020) to ensure we're aware of any changes to how KCL works. A change could be a bug, or it could be harmless, or deliberate, but keeping it checked into the repo means we can easily track changes. Note: UUIDs sent back by the engine are currently nondeterministic, so they would break all the snapshot tests. So, the snapshots use a regex filter and replace anything that looks like a uuid with [uuid] when writing program memory to a snapshot. In the future I hope our UUIDs will be seedable and easy to make deterministic. At that point, we can stop filtering the UUIDs. We run this pipeline on many different KCL programs. Each keeps its inputs (KCL programs), outputs (PNG, program memory snapshot) and intermediate artifacts (AST, token lists, etc) in that directory. I also added a new `just` command to easily generate these tests. You can run `just new-sim-test gear $(cat gear.kcl)` to set up a new gear test directory and generate all the intermediate artifacts for the first time. This doesn't need any macros, it just appends some new lines of normal Rust source code to `tests.rs`, so it's easy to see exactly what the code is doing. This uses `cargo insta` for convenient snapshot testing of artifacts as JSON, and `twenty-twenty` for snapshotting PNGs. This was heavily inspired by Predrag Gruevski's talk at EuroRust 2024 about deterministic simulation testing, and how it can both reduce bugs and also reduce testing/CI time. Very grateful to him for chatting with me about this over the last couple of weeks.
2024-10-30 12:14:17 -05:00
cnr := "cargo nextest run"
cita := "cargo insta test --accept"
kcl_lib_flags := "-p kcl-lib --features artifact-graph"
KCL: New simulation test pipeline (#4351) The idea behind this is to test all the various stages of executing KCL separately, i.e. - Start with a program - Tokenize it - Parse those tokens into an AST - Recast the AST - Execute the AST, outputting - a PNG of the rendered model - serialized program memory Each of these steps reads some input and writes some output to disk. The output of one step becomes the input to the next step. These intermediate artifacts are also snapshotted (like expectorate or 2020) to ensure we're aware of any changes to how KCL works. A change could be a bug, or it could be harmless, or deliberate, but keeping it checked into the repo means we can easily track changes. Note: UUIDs sent back by the engine are currently nondeterministic, so they would break all the snapshot tests. So, the snapshots use a regex filter and replace anything that looks like a uuid with [uuid] when writing program memory to a snapshot. In the future I hope our UUIDs will be seedable and easy to make deterministic. At that point, we can stop filtering the UUIDs. We run this pipeline on many different KCL programs. Each keeps its inputs (KCL programs), outputs (PNG, program memory snapshot) and intermediate artifacts (AST, token lists, etc) in that directory. I also added a new `just` command to easily generate these tests. You can run `just new-sim-test gear $(cat gear.kcl)` to set up a new gear test directory and generate all the intermediate artifacts for the first time. This doesn't need any macros, it just appends some new lines of normal Rust source code to `tests.rs`, so it's easy to see exactly what the code is doing. This uses `cargo insta` for convenient snapshot testing of artifacts as JSON, and `twenty-twenty` for snapshotting PNGs. This was heavily inspired by Predrag Gruevski's talk at EuroRust 2024 about deterministic simulation testing, and how it can both reduce bugs and also reduce testing/CI time. Very grateful to him for chatting with me about this over the last couple of weeks.
2024-10-30 12:14:17 -05:00
# Run the same lint checks we run in CI.
2024-09-06 09:42:11 -05:00
lint:
cargo clippy --workspace --all-targets --all-features -- -D warnings
# Ensure we can build without extra feature flags.
cargo clippy -p kcl-lib --all-targets -- -D warnings
lint-fix:
cargo clippy --workspace --all-targets --all-features --fix
# Run the stdlib docs generation
redo-kcl-stdlib-docs-no-imgs:
EXPECTORATE=overwrite {{cnr}} {{kcl_lib_flags}} docs::gen_std_tests::test_generate_stdlib
# Generate the stdlib image artifacts
# Then run the stdlib docs generation
redo-kcl-stdlib-docs:
TWENTY_TWENTY=overwrite {{cnr}} {{kcl_lib_flags}} -- kcl_test_example
TWENTY_TWENTY=overwrite {{cnr}} {{kcl_lib_flags}} docs::kcl_doc::test::kcl_test_examples
EXPECTORATE=overwrite {{cnr}} {{kcl_lib_flags}} -- docs::gen_std_tests::test_generate_stdlib
EXPECTORATE=overwrite {{cnr}} {{kcl_lib_flags}} -- generate_settings_docs
KCL: New simulation test pipeline (#4351) The idea behind this is to test all the various stages of executing KCL separately, i.e. - Start with a program - Tokenize it - Parse those tokens into an AST - Recast the AST - Execute the AST, outputting - a PNG of the rendered model - serialized program memory Each of these steps reads some input and writes some output to disk. The output of one step becomes the input to the next step. These intermediate artifacts are also snapshotted (like expectorate or 2020) to ensure we're aware of any changes to how KCL works. A change could be a bug, or it could be harmless, or deliberate, but keeping it checked into the repo means we can easily track changes. Note: UUIDs sent back by the engine are currently nondeterministic, so they would break all the snapshot tests. So, the snapshots use a regex filter and replace anything that looks like a uuid with [uuid] when writing program memory to a snapshot. In the future I hope our UUIDs will be seedable and easy to make deterministic. At that point, we can stop filtering the UUIDs. We run this pipeline on many different KCL programs. Each keeps its inputs (KCL programs), outputs (PNG, program memory snapshot) and intermediate artifacts (AST, token lists, etc) in that directory. I also added a new `just` command to easily generate these tests. You can run `just new-sim-test gear $(cat gear.kcl)` to set up a new gear test directory and generate all the intermediate artifacts for the first time. This doesn't need any macros, it just appends some new lines of normal Rust source code to `tests.rs`, so it's easy to see exactly what the code is doing. This uses `cargo insta` for convenient snapshot testing of artifacts as JSON, and `twenty-twenty` for snapshotting PNGs. This was heavily inspired by Predrag Gruevski's talk at EuroRust 2024 about deterministic simulation testing, and how it can both reduce bugs and also reduce testing/CI time. Very grateful to him for chatting with me about this over the last couple of weeks.
2024-10-30 12:14:17 -05:00
# Copy a test KCL file from executor tests into a new simulation test.
copy-exec-test-into-sim-test test_name:
Move the wasm lib, and cleanup rust directory and all references (#5585) * git mv src/wasm-lib rust Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv wasm-lib to workspace Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv kcl-lib Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv derive docs Signed-off-by: Jess Frazelle <github@jessfraz.com> * resolve file paths Signed-off-by: Jess Frazelle <github@jessfraz.com> * clippy Signed-off-by: Jess Frazelle <github@jessfraz.com> * move more shit Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix more paths Signed-off-by: Jess Frazelle <github@jessfraz.com> * make yarn build:wasm work Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix scripts Signed-off-by: Jess Frazelle <github@jessfraz.com> * fixups Signed-off-by: Jess Frazelle <github@jessfraz.com> * better references Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix cargo ci Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix reference Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix more ci Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix tests Signed-off-by: Jess Frazelle <github@jessfraz.com> * cargo sort Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix script Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix Signed-off-by: Jess Frazelle <github@jessfraz.com> * fmt Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix a dep Signed-off-by: Jess Frazelle <github@jessfraz.com> * sort Signed-off-by: Jess Frazelle <github@jessfraz.com> * remove unused deps Signed-off-by: Jess Frazelle <github@jessfraz.com> * Revert "remove unused deps" This reverts commit fbabdb062e275fd5cbc1476f8480a1afee15d972. * updates Signed-off-by: Jess Frazelle <github@jessfraz.com> * deps; Signed-off-by: Jess Frazelle <github@jessfraz.com> * fixes Signed-off-by: Jess Frazelle <github@jessfraz.com> * updates Signed-off-by: Jess Frazelle <github@jessfraz.com> --------- Signed-off-by: Jess Frazelle <github@jessfraz.com>
2025-03-01 13:59:01 -08:00
mkdir -p kcl-lib/tests/{{test_name}}
cp kcl-lib/e2e/executor/inputs/{{test_name}}.kcl kcl-lib/tests/{{test_name}}/input.kcl
zoo kcl fmt -w kcl-lib/tests/{{test_name}}/input.kcl
just new-sim-test {{test_name}}
# Create a new, empty KCL deterministic simulation test case.
new-sim-test test_name render_to_png="true":
Move the wasm lib, and cleanup rust directory and all references (#5585) * git mv src/wasm-lib rust Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv wasm-lib to workspace Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv kcl-lib Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv derive docs Signed-off-by: Jess Frazelle <github@jessfraz.com> * resolve file paths Signed-off-by: Jess Frazelle <github@jessfraz.com> * clippy Signed-off-by: Jess Frazelle <github@jessfraz.com> * move more shit Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix more paths Signed-off-by: Jess Frazelle <github@jessfraz.com> * make yarn build:wasm work Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix scripts Signed-off-by: Jess Frazelle <github@jessfraz.com> * fixups Signed-off-by: Jess Frazelle <github@jessfraz.com> * better references Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix cargo ci Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix reference Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix more ci Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix tests Signed-off-by: Jess Frazelle <github@jessfraz.com> * cargo sort Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix script Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix Signed-off-by: Jess Frazelle <github@jessfraz.com> * fmt Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix a dep Signed-off-by: Jess Frazelle <github@jessfraz.com> * sort Signed-off-by: Jess Frazelle <github@jessfraz.com> * remove unused deps Signed-off-by: Jess Frazelle <github@jessfraz.com> * Revert "remove unused deps" This reverts commit fbabdb062e275fd5cbc1476f8480a1afee15d972. * updates Signed-off-by: Jess Frazelle <github@jessfraz.com> * deps; Signed-off-by: Jess Frazelle <github@jessfraz.com> * fixes Signed-off-by: Jess Frazelle <github@jessfraz.com> * updates Signed-off-by: Jess Frazelle <github@jessfraz.com> --------- Signed-off-by: Jess Frazelle <github@jessfraz.com>
2025-03-01 13:59:01 -08:00
mkdir kcl-lib/tests/{{test_name}}
touch kcl-lib/tests/{{test_name}}/input.kcl
# Add the various tests for this new test case.
Move the wasm lib, and cleanup rust directory and all references (#5585) * git mv src/wasm-lib rust Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv wasm-lib to workspace Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv kcl-lib Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv derive docs Signed-off-by: Jess Frazelle <github@jessfraz.com> * resolve file paths Signed-off-by: Jess Frazelle <github@jessfraz.com> * clippy Signed-off-by: Jess Frazelle <github@jessfraz.com> * move more shit Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix more paths Signed-off-by: Jess Frazelle <github@jessfraz.com> * make yarn build:wasm work Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix scripts Signed-off-by: Jess Frazelle <github@jessfraz.com> * fixups Signed-off-by: Jess Frazelle <github@jessfraz.com> * better references Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix cargo ci Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix reference Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix more ci Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix tests Signed-off-by: Jess Frazelle <github@jessfraz.com> * cargo sort Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix script Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix Signed-off-by: Jess Frazelle <github@jessfraz.com> * fmt Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix a dep Signed-off-by: Jess Frazelle <github@jessfraz.com> * sort Signed-off-by: Jess Frazelle <github@jessfraz.com> * remove unused deps Signed-off-by: Jess Frazelle <github@jessfraz.com> * Revert "remove unused deps" This reverts commit fbabdb062e275fd5cbc1476f8480a1afee15d972. * updates Signed-off-by: Jess Frazelle <github@jessfraz.com> * deps; Signed-off-by: Jess Frazelle <github@jessfraz.com> * fixes Signed-off-by: Jess Frazelle <github@jessfraz.com> * updates Signed-off-by: Jess Frazelle <github@jessfraz.com> --------- Signed-off-by: Jess Frazelle <github@jessfraz.com>
2025-03-01 13:59:01 -08:00
cat kcl-lib/tests/simtest.tmpl | sed "s/TEST_NAME_HERE/{{test_name}}/" | sed "s/RENDER_TO_PNG/{{render_to_png}}/" >> kcl-lib/src/simulation_tests.rs
# Run a KCL deterministic simulation test case and accept output.
overwrite-sim-test-sample test_name:
ZOO_SIM_UPDATE=always EXPECTORATE=overwrite TWENTY_TWENTY=overwrite {{cita}} {{kcl_lib_flags}} --no-quiet -- simulation_tests::kcl_samples::parse_{{test_name}}
ZOO_SIM_UPDATE=always EXPECTORATE=overwrite TWENTY_TWENTY=overwrite {{cita}} {{kcl_lib_flags}} --no-quiet -- simulation_tests::kcl_samples::unparse_{{test_name}}
ZOO_SIM_UPDATE=always EXPECTORATE=overwrite TWENTY_TWENTY=overwrite {{cita}} {{kcl_lib_flags}} --no-quiet -- simulation_tests::kcl_samples::kcl_test_execute_{{test_name}}
ZOO_SIM_UPDATE=always EXPECTORATE=overwrite TWENTY_TWENTY=overwrite {{cita}} {{kcl_lib_flags}} --no-quiet -- simulation_tests::kcl_samples::test_after_engine
overwrite-sim-test test_name:
ZOO_SIM_UPDATE=always EXPECTORATE=overwrite TWENTY_TWENTY=overwrite {{cita}} {{kcl_lib_flags}} --no-quiet -- simulation_tests::{{test_name}}::parse
ZOO_SIM_UPDATE=always EXPECTORATE=overwrite TWENTY_TWENTY=overwrite {{cita}} {{kcl_lib_flags}} --no-quiet -- simulation_tests::{{test_name}}::unparse
ZOO_SIM_UPDATE=always EXPECTORATE=overwrite TWENTY_TWENTY=overwrite {{cita}} {{kcl_lib_flags}} --no-quiet -- simulation_tests::{{test_name}}::kcl_test_execute
[ {{test_name}} != "kcl_samples" ] || ZOO_SIM_UPDATE=always EXPECTORATE=overwrite TWENTY_TWENTY=overwrite {{cita}} {{kcl_lib_flags}} --no-quiet -- simulation_tests::{{test_name}}::test_after_engine
# Regenerate all the simulation test output.
redo-sim-tests:
ZOO_SIM_UPDATE=always EXPECTORATE=overwrite TWENTY_TWENTY=overwrite {{cita}} {{kcl_lib_flags}} --no-quiet -- simulation_tests
test:
cargo install cargo-nextest
{{cnr}} --workspace --features=artifact-graph --no-fail-fast
2025-02-19 16:56:51 -06:00
bump-kcl-crate-versions bump='patch':
# First build the kcl-bumper tool.
cargo build -p kcl-bumper
./target/debug/kcl-bumper --bump {{bump}}
cargo check -p kcl-bumper # this way Cargo.lock gets updated
2025-02-19 16:56:51 -06:00
publish-kcl version:
git tag kcl-{{version}} -m "Release kcl-{{version}}"
Move the wasm lib, and cleanup rust directory and all references (#5585) * git mv src/wasm-lib rust Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv wasm-lib to workspace Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv kcl-lib Signed-off-by: Jess Frazelle <github@jessfraz.com> * mv derive docs Signed-off-by: Jess Frazelle <github@jessfraz.com> * resolve file paths Signed-off-by: Jess Frazelle <github@jessfraz.com> * clippy Signed-off-by: Jess Frazelle <github@jessfraz.com> * move more shit Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix more paths Signed-off-by: Jess Frazelle <github@jessfraz.com> * make yarn build:wasm work Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix scripts Signed-off-by: Jess Frazelle <github@jessfraz.com> * fixups Signed-off-by: Jess Frazelle <github@jessfraz.com> * better references Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix cargo ci Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix reference Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix more ci Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix tests Signed-off-by: Jess Frazelle <github@jessfraz.com> * cargo sort Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix script Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix Signed-off-by: Jess Frazelle <github@jessfraz.com> * fmt Signed-off-by: Jess Frazelle <github@jessfraz.com> * fix a dep Signed-off-by: Jess Frazelle <github@jessfraz.com> * sort Signed-off-by: Jess Frazelle <github@jessfraz.com> * remove unused deps Signed-off-by: Jess Frazelle <github@jessfraz.com> * Revert "remove unused deps" This reverts commit fbabdb062e275fd5cbc1476f8480a1afee15d972. * updates Signed-off-by: Jess Frazelle <github@jessfraz.com> * deps; Signed-off-by: Jess Frazelle <github@jessfraz.com> * fixes Signed-off-by: Jess Frazelle <github@jessfraz.com> * updates Signed-off-by: Jess Frazelle <github@jessfraz.com> --------- Signed-off-by: Jess Frazelle <github@jessfraz.com>
2025-03-01 13:59:01 -08:00
cargo publish -p kcl-derive-docs
cargo publish -p kcl-directory-test-macro
2025-02-19 16:56:51 -06:00
cargo publish -p kcl-lib
cargo publish -p kcl-test-server
# We push the tag at the end of publish since pushing the tag
# will trigger CI to release the kcl python bindings.
git push origin kcl-{{version}}