User journey testing strategy

Purpose

This document defines how LMSTool describes, prioritizes, and maintains realistic user-journey and workflow-acceptance coverage. The user journey catalogue applies these rules to the current application and test suites.

The testing terminology glossary defines the eight core testing terms used across this documentation.

The model is evidence-based. A test counts only for the behaviour it actually asserts. A green test run, high line coverage, or a collection of component tests does not by itself prove a complete user journey.

Document responsibilities

Concern Authoritative document Explicitly not owned there
Core testing term definitions Testing terminology glossary Test-layer taxonomy, catalogue rules, current results, and fixture inventories
Test-layer taxonomy, catalogue rules, prioritization, evidence rules, maintenance This strategy Core term definitions, current run results, and fixture inventories
Behavioural journeys, matrices, evidence, gaps, golden backlog User journey catalogue Mutable pass, fail, skip, timing, or CI totals
Playwright execution profile, fixture/provider gaps, skip policy, CI gate Playwright E2E coverage General behavioural catalogue and live deployment acceptance
Public disposable demo lifecycle and owner-authorized release validation Demo environment acceptance testing Ordinary local E2E fixture status
Architecture ER and C4 diagram maintenance Diagram maintenance Journey and workflow diagrams

Each mutable concern has one source of truth. Other documents link to it instead of copying its status, commands, or backlog.

Test layers

Layer Responsibility Typical evidence
Unit One pure rule, parser, validator, policy, or transformation Fast isolated test
Integration/service Transactions, PostgreSQL/PostGIS invariants, workflow services, file processing Django TestCase or TransactionTestCase
API/view Request parsing, authentication, permissions, response semantics, persistence through server boundaries Django client or Playwright request context
Browser Real navigation, visible controls, forms, downloads, uploads, maps, reloads, and multi-page flows TypeScript or Python Playwright
Owner acceptance Production-like infrastructure, external integrations, performance, or destructive lifecycle validation Controlled runbook and recorded evidence
Exploratory Human investigation of unusual sequences, stale pages, retries, or unmodelled behaviour Session notes converted to regression evidence when defects are found
Stateful/model-based Generated action sequences checked against workflow invariants Future lower-layer tooling; not introduced by this feature
Production smoke Safe deployed checks for essential availability and configuration Deployment-specific evidence, never destructive workflow mutation

Classify evidence by the behaviour exercised, not its runner. A Playwright test that only calls JSON endpoints is API/view evidence. A Django StaticLiveServerTestCase that drives Chromium is browser evidence.

Personas

The catalogue uses concrete system identities:

  • ANONYMOUS
  • VIEWER
  • EDITOR
  • REVIEWER
  • APPROVER
  • SUPERUSER

VIEWER through APPROVER mean one exact DatasetRoleGrant unless a journey explicitly states that an actor has several grants. Higher dataset roles may satisfy hierarchical read access, but they do not inherit lower-stage mutation authority.

SUPERUSER means the Django system administrator. A superuser manages grants through administration but is deliberately excluded from dataset workflows. There is no separate dataset-administrator role.

Journey identifiers

Use:

UJ-<AREA>-<NNN>

Controlled area codes are:

Code Area
AUTH Authentication and navigation
ACCESS Dataset access and switching
PARCEL Parcel viewing and editing
DRAFT Ordinary parcel draft workflow
EXCH GeoPackage draft editing exchange
UPLOAD Bulk upload and draft creation
TOPO Split and merge topology workflow
OVERLAP Parcel overlap governance
SNAP Snapping and coordinate support
BASEMAP Basemap selection and inheritance
ROLE Exact-role and separation-of-duties boundaries

IDs are permanent and never reused. Do not encode priority, golden status, persona, test layer, or coverage disposition in an ID.

Each canonical definition uses:

### UJ-<AREA>-<NNN> — <Title>

Matrices and evidence mappings refer to that ID without creating another definition heading.

Coverage dispositions

Every journey has exactly one durable disposition:

Disposition Meaning
COVERED Reliable primary evidence proves the complete behaviour as documented
GAP The behaviour is supported but complete reliable evidence is absent
INTENTIONALLY_UNSUPPORTED An explicit product contract excludes the behaviour

A GAP has exactly one reason:

  • NO_TEST
  • PARTIAL_EVIDENCE
  • FIXTURE_BLOCKED
  • ENVIRONMENT_BLOCKED
  • UNRELIABLE_EVIDENCE
  • MANUAL_ONLY

PARTIAL_EVIDENCE is not a fourth disposition. Planned, skipped, stale, or unreliable tests cannot establish COVERED. Run-specific pass/fail/skip totals, retry counts, dates, and durations are operational results and do not appear in the catalogue; durable fixture-blocked status remains visible as a gap.

Evidence mappings

Tests and journeys have a many-to-many relationship. Each mapping records:

  • a stable locator and source link;
  • effective layer;
  • PRIMARY, SUPPORTING, or INVARIANT contribution;
  • assertions proved;
  • limitations;
  • non-standard execution profile, if applicable.

Stable human locators are:

apps/path/test_module.py::ClassName::test_method
tests/e2e/file.spec.ts::Describe title > exact test title

Line numbers are navigation aids, not identity.

Playwright's collector renders titles as:

[project] › file:line › describe › title

That output deliberately differs from the catalogue's compact :: and > notation. Resolve Playwright evidence manually by file and complete title. Never add a brittle scripted assertion expecting the catalogue locator to occur verbatim in npx playwright test --list.

Before a locator supports a COVERED disposition, confirm that the cited test exists and that its assertions prove the claimed behaviour. Several supporting or invariant tests do not automatically become primary end-to-end evidence.

Critical-browser evidence contract

Feature 030 keeps the fixed golden selection and gives each required subcase a stable scenario identity in tests/e2e/fixtures/scenarios.json. The validator in scripts/e2e_scenarios.py fixes required journey/subcase membership independently of editable registry content. Retained COMP-* scenarios are meaningful companions, not replacements for a missing primary case. Precision obligations have their own evidence IDs and do not create new golden journeys.

An authored test("CASE-ID", {tag: "@scenario:CASE-ID"}, ...) identifies a browser evidence candidate, not a passing result. Its compact locator is tests/e2e/file.spec.ts::CASE-ID. Record visible action, normal save/reload, independent persisted-state assertions, and unchanged-state controls separately. tests/e2e/fixtures/scenario.ts::observeScenario obtains fresh read-only observations; tests/e2e/fixtures/workflow.ts::expectApprovedChain and expectDeniedDraftUnchanged compare versions, attribution, events, and denial state. A sanitized denial event is permitted only where the product contract allows it; no successful or terminal event may be invented by a denied action.

Complete coverage requires every required subcase in its declared browser projects, not merely one tag for the journey. Collection proves membership only. A focused or single-project run is partial, and a retry pass does not erase its first failure or establish reliable full-profile acceptance. Required skips, expected failures, unexecuted cases, or missing projects cannot be compensated by a discovery minimum or a permissive historical skip budget. Mutable results and artifact paths remain in the operational coverage guide, not this strategy or the catalogue.

Browser correctness, rendered performance and real-container acceptance are separate gates. A passing behavioural journey does not waive a failed timing budget or prove container isolation. A baseline that also exceeds the budget documents an existing limitation; it is not a passing performance result or proof that every current timing difference is harmless. Record which assets and environment were actually held constant rather than calling a partial asset comparison a full-application baseline.

Promote a bounded journey to COVERED when its complete primary browser and invariant evidence meets this contract, while retaining separate performance or owner-acceptance blockers and explicitly approved deferrals. A correctness disposition is not publication approval. Feature 030 has a separate 2026-09-08 owner decision permitting publication with performance follow-up deferred and unaccepted; it does not change any measured failure into a passing result. Preserve failed attempts and their root-cause evidence alongside the clean run; do not construct a passing default outcome by combining partial reruns.

Real concurrency evidence must use separate committed database connections and interleaved transactions. The test-only apps/parcels/tests/concurrency_helpers.py::run_lock_contending_transactions verifies distinct PostgreSQL backends/transactions and simultaneous lock waiters. Ordinary approval, topology approval, and overlap-reservation races then independently assert the winning state and absence of partial writes. Sequential double submission, an injected IntegrityError, and a stale browser tab remain useful controls but are not substitutes for those races.

Correction follows supported product identity: REJECTED remains terminal; the editor creates a new correction draft on the same parcel/source chain. The test must distinguish the draft IDs and preserve the original rejection/audit, not invent a reopen transition. Likewise, Firefox page automation does not establish real extension installation/container/manual acceptance, and intercepted map resources do not prove a live external provider.

Required views

The catalogue maintains:

  1. A compact journey index.
  2. A many-to-many evidence registry and mapping.
  3. Role-action matrices.
  4. State-transition matrices for ordinary drafts and topology requests.
  5. A journey-layer matrix.
  6. Separate covered, gap, and intentionally-unsupported views.
  7. Detailed records for every golden journey and material gap.

Role matrices use ALLOW, DENY, NON_DISCLOSE, and N/A. Dataset columns assume only the named exact role. Multi-role and actor-separation cases are recorded separately.

Golden-journey prioritization

Rate each candidate HIGH, MEDIUM, or LOW across:

  • user impact;
  • data-integrity risk;
  • authorization risk;
  • regression likelihood.

Derive priority as follows:

  • P1 when data integrity or authorization is HIGH, or at least two dimensions are HIGH;
  • P2 when another single dimension is HIGH, or at least two dimensions are MEDIUM;
  • P3 otherwise.

Candidate selection occurs after eliminating duplicate golden implementations. A journey may remain P1 but be treated as a companion rather than a separate candidate only when a broader selected golden journey explicitly includes its assertions and links its ID. Intentionally unsupported journeys are not automation candidates. The initial backlog then selects all representative P1 candidates, followed by the highest-impact P2 candidates if capacity remains, with a maximum of 15. Golden status can change without changing the journey ID.

Every state-changing golden journey defines:

  • actors and exact-role assumptions;
  • isolated deterministic preconditions;
  • visible interaction sequence and outcome;
  • persisted changes and data that must remain unchanged;
  • audit/event evidence;
  • reload or independent re-query verification;
  • authorization and separation rules;
  • spatial/CRS boundary where applicable;
  • current evidence, limitation, and intended future layer.

Spatial and CRS evidence

A spatial journey identifies:

  1. Browser geometry or CRS-bearing file input.
  2. Declared projected source CRS and known linear units for imports.
  3. Server transformation boundary.
  4. Frozen parcel storage CRS.
  5. Authoritative PostGIS validation, measurement, and persistence.
  6. Source/storage CRS audit evidence.
  7. Expected storage-CRS geometry comparison.
  8. Rejection of missing, ambiguous, geographic/angular, or non-transformable CRS values.

GeoJSON and KML are export-only files. Their production file import is intentionally unsupported.

Mermaid and textual equivalents

Mermaid may clarify state, sequence, role, or cross-component relationships. Every diagram must be followed immediately by a complete table, ordered sequence, or paragraph beginning **Text equivalent:**. Text is authoritative.

The current MkDocs configuration has no Mermaid plugin or custom fence. The built documentation site therefore displays ```mermaid blocks as raw code, matching existing architecture documentation. This is an accepted outcome. Review diagrams manually in GitHub or another Mermaid-capable preview. Do not add a Mermaid dependency or plugin for this feature.

Maintenance workflow

Review this strategy and the catalogue whenever:

  • a workflow state, transition, role, or separation rule changes;
  • a user-visible path is added, removed, or materially changed;
  • a test is added, removed, renamed, skipped, or moved between profiles;
  • an E2E fixture gap is resolved or introduced;
  • a production incident exposes an unmodelled sequence;
  • a spatial format, CRS boundary, or authoritative invariant changes.

For each affected journey:

  1. Keep its ID.
  2. Update behaviour and matrices from application contracts.
  3. Re-resolve evidence locators.
  4. State precisely what each test proves and does not prove.
  5. Reassign disposition only when effective evidence changes.
  6. Recompute risk when impact or system boundaries change.
  7. Update the golden backlog without exceeding its bound.
  8. Leave run-specific results in CI or operational evidence.

Retired journeys keep their identifiers and link to the product decision that made them unsupported or obsolete.

Validation

Use the commands and manual checks in specs/021-user-journey-test-model/quickstart.md. At minimum:

  • build MkDocs strictly;
  • collect Playwright titles and reconcile them semantically;
  • resolve Python and Django locators;
  • verify unique canonical headings and all required area codes;
  • verify disposition and gap-reason rules;
  • reject volatile results in the new documents;
  • inspect Mermaid and its authoritative text;
  • run git diff --check.