test: media tests flake under parallel runs — staging dir is shared across #[sqlx::test]s #170

New issue

Closed

opened 2026-08-10 01:33:01 +00:00 by rosa · 0 comments

rosa commented

2026-08-10 01:33:01 +00:00

Owner

Two media integration tests in crates/server/tests/app_test.rs fail intermittently when the suite runs with the default parallel test harness, but pass reliably single-threaded.

Affected tests

web_editor_uploads_then_removes_a_photo — after run_optimize_media, the page still renders "This photo is still being processed…" and the assertion assert_text_contains("u-photo") (app_test.rs:979) fails.
undecodable_upload_fails_the_media_and_deletes_the_staged_original — an undecodable image is a terminal failure, got Ok(()) (app_test.rs:1137).

Evidence

cargo test -p vernier-server (default parallelism): one or both fail, non-deterministically.
cargo test -p vernier-server --test app_test -- --test-threads=1: all 59 pass.

So it is a cross-test interference race, not a logic bug in either test.

Suspected root cause

staging_dir() (crates/infra/src/staging.rs:117) derives the media-staging directory as a sibling of the database file: <db-parent>/media-staging. The test harness builds staging from the test pool's filename:

let filename = pool.connect_options().get_filename().to_owned();
let staging = FsMediaStaging::new(staging_dir(&format!("sqlite://{}", filename.display())));

(crates/server/tests/common/mod.rs:110)

Under #[sqlx::test] the per-test SQLite databases share a parent directory, so every test resolves to the same media-staging folder. Staged originals are then keyed only by MediaId in one shared directory, and tests that scan or delete staged files step on each other. In particular reconciler_sweeps_an_orphaned_staged_original deletes staged originals it considers orphaned; running concurrently it can remove another test's staged original before that test's optimize job reads it — which matches the "still being processed" / "got Ok(())" symptoms (the optimize job finds no bytes to work from).

Suggested fix

Give each test its own staging directory instead of one derived from a shared temp parent — e.g. stage under a unique per-test subdirectory (keyed off the unique DB filename, a tempfile::TempDir, or the operator sequence already used in common/mod.rs), so no two tests share a media-staging folder. The production derivation in staging_dir() is correct and should stay as-is; only the test harness needs isolation.

Two media integration tests in `crates/server/tests/app_test.rs` fail intermittently when the suite runs with the default parallel test harness, but pass reliably single-threaded. ## Affected tests - `web_editor_uploads_then_removes_a_photo` — after `run_optimize_media`, the page still renders "This photo is still being processed…" and the assertion `assert_text_contains("u-photo")` (app_test.rs:979) fails. - `undecodable_upload_fails_the_media_and_deletes_the_staged_original` — `an undecodable image is a terminal failure, got Ok(())` (app_test.rs:1137). ## Evidence - `cargo test -p vernier-server` (default parallelism): one or both fail, non-deterministically. - `cargo test -p vernier-server --test app_test -- --test-threads=1`: all 59 pass. So it is a cross-test interference race, not a logic bug in either test. ## Suspected root cause `staging_dir()` (`crates/infra/src/staging.rs:117`) derives the media-staging directory as a **sibling of the database file**: `<db-parent>/media-staging`. The test harness builds staging from the test pool's filename: ``` let filename = pool.connect_options().get_filename().to_owned(); let staging = FsMediaStaging::new(staging_dir(&format!("sqlite://{}", filename.display()))); ``` (`crates/server/tests/common/mod.rs:110`) Under `#[sqlx::test]` the per-test SQLite databases share a parent directory, so every test resolves to the **same** `media-staging` folder. Staged originals are then keyed only by `MediaId` in one shared directory, and tests that scan or delete staged files step on each other. In particular `reconciler_sweeps_an_orphaned_staged_original` deletes staged originals it considers orphaned; running concurrently it can remove another test's staged original before that test's optimize job reads it — which matches the "still being processed" / "got Ok(())" symptoms (the optimize job finds no bytes to work from). ## Suggested fix Give each test its own staging directory instead of one derived from a shared temp parent — e.g. stage under a unique per-test subdirectory (keyed off the unique DB filename, a `tempfile::TempDir`, or the operator sequence already used in `common/mod.rs`), so no two tests share a `media-staging` folder. The production derivation in `staging_dir()` is correct and should stay as-is; only the test harness needs isolation.

rosa added the

labels

2026-08-10 01:33:23 +00:00

rosa referenced this issue from a commit

2026-08-10 01:41:28 +00:00

test: isolate media staging per test to fix parallel-run flake

rosa closed this issue

2026-08-10 01:41:28 +00:00