#Testing strategy

Testing is layered so fast feedback catches local regressions while the real Extension Host catches protocol, renderer, and lifecycle failures.

The cross-project priorities, baselines, and acceptance criteria are tracked in the Project Quality Improvement Roadmap. This file is the canonical policy for choosing test layers and defining stateful or live test behavior.

#Gates

The PR baseline is npm run verify:pr. Live proprietary databases are nightly or manual because they require credentials and controlled infrastructure. npm run test:coverage enforces the global floors from quality/quality-baseline.json; on a pull request, the CI unit job also runs the changed high-risk gate against the pull request base commit. Locally, the equivalent is npm run test:coverage:changed after fetching origin/master. The changed-code input is the repository-wide diff, and the gate merges LCOV from root Jest, ui-core, ui-react, Web, and migrated media. A changed executable file without an LCOV record fails; generated files, declarations, tests, and setup files are the only automatic exclusions. The high-risk changed-code minimum is 80% lines and 70% branches. A green root-only LCOV report is not evidence for package, app, or media paths.

test:sql-core also builds and loads the public package entrypoints through CommonJS and native ESM. This catches missing exports and duplicated Chevrotain token/parser identities that source-only Jest aliases would hide.

Shared-code migration slices additionally follow the desktop-first comparison and companion gates in Shared-code migration preparation. Capture old/new behavior on the same fixtures before replacing a facade; changing a baseline to accept a difference is not parity evidence.

The first SQL validation slice has an explicit parity gate. It compares the package-owned NetezzaSqlSemanticValidator with the legacy SqlValidator for validity, diagnostic order, code, message, severity, positions, offsets, suggested fixes and normalized scope. The corpus must include Netezza qualification forms, malformed SQL, procedures, typed/untyped metadata, macro masking with astral Unicode and SQL025/SQL026 through both desktop and LSP schema providers. Runtime code must not execute both implementations merely to obtain parity evidence; the dual run is test-only. After the boundary is switched, parser, linter, API and Extension Host authoring gates remain required, and non-Netezza dialects continue using their legacy path until their own dialect migration begins.

#Change risk and required layers

Every behavior change needs the lowest-cost direct test. Medium- and high-risk changes also need evidence at the nearest real runtime boundary.

A mocked happy path is not sufficient for high-risk behavior. The test set must cover identity, transitions, invalidation, recovery, disposal, and resource cleanup.

#Coverage and regression policy

Coverage is a risk indicator, not a substitute for behavioral assertions.

• The initial root floors were 71% statements, 58% branches, 76% functions, and 72% lines; they were raised to 72% statements, 59% branches, 77% functions, and 73% lines. These floors may only move upward.
• Changed high-risk code should reach at least 80% line and 70% branch coverage.
• Generated files must be excluded explicitly rather than lowering thresholds.
• Prioritize migration, activation, views, commands, editors, imports, and exports; their baseline is materially below parser and dialect coverage.
• Review uncovered conditions in touched code. Do not add trivial assertions only to improve a percentage.

Extended lint follows the same ratchet: the recorded baseline may not increase, and each cleanup phase lowers the allowed value. The current frozen baseline is 52 warnings (52 media, 0 packages, and 0 extensions); the retired Web Editor apps area no longer exists. The next cleanup target is zero warnings; the Phase 1B milestone of 100 warnings has been met.

#Stateful UI contract

Every stateful webview or React surface must document and test:

1. the stable identity of the state owner;
2. which fields are persisted and where;
3. transitions that preserve state;
4. events that invalidate or migrate state;
5. behavior after hide/reveal, reload/revival, and disposal;
6. loading, empty, error, cancellation, retry, and partial-success states;
7. cleanup of timers, listeners, workers, pending requests, and temporary data.

Test the pure state transition first, the bundled DOM/component behavior next, and a real browser or Extension Host boundary for high-risk lifecycle behavior. Use observable readiness or protocol acknowledgements instead of fixed sleeps.

#Result-panel scroll contract

Scroll behavior is a protocol contract, not an implementation detail. The deterministic Extension Host scenario must cover:

1. a wide, virtualized result with enough rows to leave the initial viewport;
2. a non-zero vertical scroll anchor and non-zero horizontal offset;
3. switching to Logs and back to the same result set;
4. switching to a different source and back;
5. hydrate/render settling and debounced scroll persistence;
6. an assertion on both axes, preferably the virtualizer anchor, with a pixel tolerance only as a fallback;
7. stable result identity distinct from the Logs identity.

The current fixture uses 12 deterministic rows crossed with itself (144 rows), drives scrollResult({ rowIndex: 75, scrollLeft: 320 }), and captures a sanitized viewport snapshot. Unit tests additionally verify stable-ID lookup, legacy timestamp fallback, and rejection of cached state belonging to another result identity. If the grid persistence algorithm changes, update both levels.

The complete matrix also includes sorting, filters, grouping, expansion, column order/width/visibility/pinning, formatting, and alternate/disk-backed views. Exercise Logs/result switching, multiple result sets, multiple sources, panel hide/reveal, webview revival, pin/close index shifts, refresh, new execution, active streaming, cancellation with partial data, disk-backed data, empty data, and initially zero-sized layouts. Prefer an exact virtualizer-anchor assertion; use pixel tolerance only when no stable anchor is available.

#Responsive collapsed result columns

The opt-in desktop layout is owned by the grid for one stable result identity. responsiveCollapseEnabled is persisted in that result's grid-state entry; row expansion is ephemeral and keyed by the memory row ID, or by disk query identity plus its absolute row index. A new disk filter/sort query invalidates expanded rows. The hidden-column set is derived from the wrapper width, current visible columns, saved widths, and pinned columns; it is never persisted. Grouping suspends the projection while retaining the preference, and returning to flat rows recalculates it. If a resize changes the visible-column projection, cell selection is cleared because its visible coordinates changed.

The virtualizer continues to count one item per source row. When a detail row is expanded, its measured height is combined with its source row's height. Resize observers and the window-resize fallback are enabled only while the option is on and are disconnected on toggle-off or grid disposal. Cover default-off, preference restore by exact result identity, ephemeral expansion after restore, memory and SQLite-backed rows, grouped suspension, wide/narrow resize, selection invalidation, vertical anchor and horizontal scroll preservation, zero-width initialization, and observer cleanup. The DOM test proves value rendering and state transitions; the Playwright test proves toolbar access and the real browser layout/virtualizer behavior.

#Execution and asynchronous ordering

Streaming and asynchronous UI tests must include more than normal ordering:

• missing, duplicate, delayed, and out-of-order messages or chunks;
• cancellation before start, during fetch, during render, and during finalize;
• retry/reconnect with retrying followed by exactly one terminal success or error status for the logical execution (never an error before retry);
• conservative replay safety: only one proven, call-free read-only statement may retry, never a write, executable macro, function/sequence expression, ambiguous/multi-statement payload, or a stream after its first delivered chunk;
• disposal or source switch while work is pending;
• late callbacks that must not update a new/disposed owner;
• row-limit, zero-row, partial-result, multi-statement, and multi-result cases;

The desktop Result Panel enforces this transport matrix with stable result-set identity, row offsets, monotonic chunk sequences, and authoritative hydrate recovery. Its scheduled Linux race gate runs the real Extension Host scenario 20 times and retains a per-iteration report plus aggregate summary.

• final assertions that no request, timer, listener, worker, command, or session remains active.

#Web, API, and accessibility

React behavior requires component-level jsdom tests in addition to reducer tests. Cover editor tabs/preferences, connection forms, schema navigation, result-grid state, dialogs, error/cancellation paths, and restoration after reload. Playwright covers the bundled application/webview and real keyboard behavior; Extension Host tests remain the authority for VS Code lifecycle and message integration.

High-traffic surfaces require automated accessibility checks and keyboard-only flows. Serious or critical accessibility violations fail the gate. Verify accessible names, focus entry/return, modal focus containment, Escape/Enter, grid selection/copy, high-contrast themes, zoom, and reduced-motion behavior.

#Live test hygiene

Live suites must be explicitly selected, fail when required environment variables are missing, create uniquely named fixtures, and drop them in finally. Unit Jest configurations install a network guard; live configurations replace it with their database setup. Tests must not call a real driver from a unit mock—inject a client/transport seam instead.

Conditional CI jobs may decide not to select a credentialed suite when secrets are unavailable. Once a user or workflow explicitly invokes that suite, missing configuration is an error rather than a passing skip. Local/container runtimes such as SQLite, DuckDB, Access, PostgreSQL, MySQL, MSSQL, and ClickHouse should provide deterministic PR coverage where practical; controlled credentialed systems provide scheduled contract coverage.

Artifacts are temporary and sanitized. Keep screenshots, traces, exports, and benchmark output outside source control; inspect screenshots before sharing.

#Flake, stress, and teardown policy

• Do not hide a deterministic failure with retries. Capture trace/artifacts and fix the readiness, identity, or cleanup condition.
• Track skipped tests and require an owner/reason for every non-configuration skip.
• The full Jest suite must exit naturally. A forced worker exit is a failure of the test infrastructure even when assertions pass. The unit network guard rejects sockets asynchronously so driver timeout cleanup can run; tests must still inject a client/transport seam instead of attempting a real connection.
• Run the Result Panel Extension Host scenario repeatedly on a schedule; 20 iterations is the standard race-stress value.
• Benchmark on stable runners using multiple samples. Investigate sustained three-run median regressions above 15% while preserving existing absolute budgets such as the parser construction limit.

#Evidence and maintenance

For a high-risk pull request, record:

• the changed contract or state matrix;
• commands and environments used;
• relevant coverage or benchmark result;
• cleanup and failure-path evidence;
• compatibility, migration, and documentation impact.

When a test becomes the regression contract for a production incident, add the scenario to the relevant runbook and the quality roadmap rather than relying on an issue or commit message as its only explanation.