Retry when the server flakes on us

This isn't ideal, and I'd love to remove this eventually; but this is
helpful when we exhaust the Engine pool, or when running locally. This
will retry a few times, and only fail if there's a few repeated
failures.

Since the WebSocket fully handshakes, and then fails after handshake
(when api-deux realizes we're out of Engines, specifically), I can't
check the HTTP code directly where I wanted to.

This is kludgey for now, and it may mask flaky endpoints; but it may be
worth it in the short-term. I'd love to find a way for the modeling
endpoint to not upgrade via api-deux unless there's an engine for it
long-er term.

src/wasm-lib/tests/executor/main.rs

@@ -1,8 +1,22 @@
+use std::time::Duration;
+
 use anyhow::Result;
+use tokio::time::sleep;
+
+async fn execute_and_snapshot(code: &str, units: kittycad::types::UnitLength) -> Result<image::DynamicImage> {
+    for _ in 0..2 {
+        match execute_and_snapshot_once(code, units.clone()).await {
+            Ok(v) => return Ok(v),
+            _ => {}
+        }
+        sleep(Duration::from_secs(1)).await;
+    }
+    execute_and_snapshot_once(code, units).await
+}
 
 /// Executes a kcl program and takes a snapshot of the result.
 /// This returns the bytes of the snapshot.
-async fn execute_and_snapshot(code: &str, units: kittycad::types::UnitLength) -> Result<image::DynamicImage> {
+async fn execute_and_snapshot_once(code: &str, units: kittycad::types::UnitLength) -> Result<image::DynamicImage> {
     let user_agent = concat!(env!("CARGO_PKG_NAME"), ".rs/", env!("CARGO_PKG_VERSION"),);
     let http_client = reqwest::Client::builder()
         .user_agent(user_agent)