AI coding agent acceptance testing should answer a harder question than whether a changed branch is green: can the suite show that a plausible but incorrect implementation fails? An agent can produce a compiling patch and passing local tests while still mishandling authorization, validation, state transitions, or client-server expectations.
The goal is not a separate testing discipline for AI-assisted code. It is to make the requested behavior executable at the boundaries where implementation can drift from intent. A useful suite gives the implementation a clear target and gives reviewers evidence that a passing result represents the intended behavior.
Start with a fail-to-pass definition of done
Write the acceptance checks before implementation. Ask: what must fail with the old behavior and pass only when the requested behavior exists? SWE-bench evaluates proposed patches against tests intended to distinguish unresolved issues from resolved ones. Its approach is a useful model for change-level acceptance checks: validate a behavioral delta rather than merely confirm that code runs. See the SWE-bench paper.
Not every ticket needs an entirely new suite, but every ticket needs an observable definition of success. “Add a discount field” is an implementation instruction. “An unauthorized caller cannot apply a discount, an eligible order receives it once, and checkout shows the recalculated total” is an acceptance definition.
Use four kinds of proof
For changes that cross Java/Spring and React, use the proofs relevant to the risk:
- Positive proof: the intended user or API workflow succeeds.
- Negative proof: invalid, unauthorized, duplicate, expired, or conflicting input is rejected correctly.
- Invariant proof: a rule remains true across inputs and state transitions.
- Boundary proof: a consumer and provider agree on requests, responses, and errors.
A patch is ready when it has the necessary proof, not simply when a happy-path test passes. This also sharpens review: the useful question is which proof covers the riskiest way the requirement could be misunderstood.
Translate requests into observable behavior
Before implementation, make a short acceptance map that is independent of classes, hooks, tables, and component names. Those implementation details may change. The user-visible and system-visible result should not.
| Requirement signal | Acceptance check | Mistake caught |
|---|---|---|
| “Only the owner can cancel” | The owner succeeds; another authenticated user receives the defined denial response; the record remains unchanged. | An authorization check is omitted or happens after mutation. |
| “Do not apply a code twice” | The first request changes the total once; a repeat preserves the total and returns the specified result. | A retry creates a second effect. |
| “Show a useful form error” | Invalid submission exposes an accessible error, retains safe input where appropriate, and does not show success. | The UI handles only HTTP success or clears state too early. |
| “Add a field to the API” | The client consumes the field and the server returns the required type and error shape consistently. | Client and server expectations diverge. |
Include preconditions, a stimulus, an observable result, and important non-results. Non-results can matter as much as success: no record changes, no success message appears, and no privilege is granted. They are useful when a requirement includes authorization, ordering, or state-transition constraints.
Test Spring at the HTTP boundary
For Spring APIs, acceptance tests should exercise the HTTP boundary where routing, deserialization, validation, security, and exception handling meet. Spring Boot documents mock web environments, MVC testing with MockMvc, and running-server test arrangements. See the Spring Boot testing reference.
Use a web-layer test when the risk involves request validation, authorization mapping, HTTP status selection, or JSON error shape. Use a running-server test selectively when the behavior depends on the deployed web environment. This keeps checks focused on the boundary that owns the behavior without making every test a full end-to-end test.
Make rejection behavior explicit
Generated implementations can handle the nominal case while leaving rejection behavior incomplete. Treat each important rejection as a contract. For a mutation endpoint, identify malformed, missing, forbidden, stale, duplicate, or incompatible requests. Assert the status, any documented machine-readable error identifier, and the absence of unintended state change.
An order-cancellation check should not stop at a pending order that can be cancelled. It can also cover a caller without access, an order that has already moved beyond the cancellable state, a repeated cancellation, and malformed input. Keep assertions on documented API behavior rather than private exception classes, repository calls, or log text.
Express business rules as invariants
An invariant describes what must remain true across a class of situations, not just one example. Examples include a total that does not become negative, an unavailable transition that does not mutate state, a caller that cannot observe another tenant’s resource, or an idempotency key that does not create a second effect.
Use representative examples for readability, then use parameterized or generated inputs where the domain has meaningful combinations. The purpose is to cover input dimensions an implementation may overlook, such as empty values, limits, repeated actions, ownership boundaries, partial data, and changing states.
Protect the React workflow, not internals
React acceptance checks should follow the observable user journey: enter data, submit, wait for the resulting state, and inspect what is exposed to assistive technology. Testing Library recommends tests that resemble how software is used and discourages unnecessary coupling to implementation details. See the Testing Library guiding principles.
That supports durable checks for roles, labels, validation messages, action availability, loading states, and visible success or failure outcomes. It avoids tying acceptance tests to hook state, CSS classes, or a specific component tree. Internal refactors can be valid; a user-facing failure state still needs to work.
Test the failure journey
For each optimistic success flow, add the relevant failed-request flow. Check that the user can understand the result and recover: an error is exposed accessibly, retry behavior works where it exists, safe form values remain when appropriate, and a rejected response does not produce a success state.
For stateful workflows, test transitions rather than isolated screens. A patch can display a new control but retain an old error after retry, fail to reset state after saving, or submit stale data after navigation. These are acceptance failures even when a snapshot or happy-path component test passes.
Where the input space is broad, property-based testing can supplement examples. fast-check supports property-based testing for JavaScript and TypeScript, enabling generated values and sequences to be checked against a stated rule. Keep generated tests domain-bounded and reproducible so failures remain actionable.
Use contracts to prevent client-server drift
When a React application consumes a Spring API, an HTTP integration check may not reveal that a response is technically valid but no longer meets client needs. A consumer-driven contract records a consumer expectation, while provider verification checks whether the provider can meet it. Pact documents this workflow in its official documentation.
Use contracts for boundaries with independent deployment, shared ownership, or a history of schema drift. Focus each contract on a meaningful interaction: request fields the UI sends, response fields it requires, error statuses it handles, and optional or null values that alter rendering. Narrow expectations are easier to understand than contracts that duplicate an entire API specification.
Spring web tests establish server behavior, React workflow tests establish user behavior, and consumer-provider contracts establish compatibility. They overlap slightly, but each answers a distinct question.
Prove the suite can detect a defect
A passing acceptance suite is useful only when it can fail for the right reason. Run a new acceptance check against baseline behavior and confirm the expected failure. Then apply the intended implementation and confirm it passes. For high-risk behavior, introduce a small temporary fault in a controlled local experiment, such as bypassing a validation condition, and verify that the relevant check fails. Remove the temporary fault before merging.
Mutation testing systematizes this question. PIT describes Java mutation testing that creates bytecode-level mutants and reports whether tests kill them. A surviving mutant can indicate that a test is missing or ineffective. See PIT’s basic concepts documentation. For JavaScript and TypeScript, Stryker Mutator documents mutation testing that changes program code and evaluates whether tests detect the changes.
Mutation results are an investigation tool, not necessarily a universal release gate. A mutant can survive because code is unreachable, the mutation is equivalent, or the behavior is not meaningful. Prioritize survivors in authorization, monetary calculations, data retention, workflow transitions, and public API mapping. The key question is whether a plausible implementation mistake could escape the acceptance proof.
Implementation checklist
- State the requested behavior as an observable outcome, including prohibited outcomes.
- Add or identify a check that fails on the baseline for the expected reason.
- Cover the normal workflow and the most consequential rejection path.
- Write an invariant for rules spanning inputs, retries, ownership, totals, or states.
- Assert Spring status and error behavior at the API boundary.
- Assert React behavior through accessible interactions and visible state changes.
- Add a consumer-provider contract where the client and API can drift independently.
- Use a deliberately failing control or targeted mutation exercise for high-risk behavior.
- Keep tests with the feature so later changes inherit the guardrail.
FAQ
Are unit tests enough for AI coding-agent acceptance testing?
Usually no. Unit tests are valuable for local logic, but acceptance checks should cover behavior users and dependent systems observe. For a Spring and React change, that can include API semantics, UI failure states, and the contract between them.
Should every generated change receive mutation testing?
No. Use it proportionately. A targeted mutation run is most useful when a change affects critical rules and ordinary tests might pass without exercising the decision that matters.
How many negative-path tests should a ticket have?
There is no fixed count. Cover rejection paths whose failure could cause incorrect access, incorrect money or data changes, broken recovery, or a misleading success state.
Can contract tests replace React workflow tests?
No. Contracts verify compatibility between a consumer and provider. They do not establish that a form communicates validation failure clearly, handles loading correctly, or lets a user complete the workflow.
Sources
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues?
- Spring Boot Reference Documentation: Testing Applications
- Pact Documentation
- PIT Mutation Testing: Basic Concepts
- Testing Library Guiding Principles
- fast-check Introduction
- Stryker Mutator: Mutation Testing
Editorial note: AI assisted with research and drafting. Sources were selected for verification.
Full-Stack Developer & Solutions Architect · Casablanca, Morocco
7+ years building Java/Spring Boot/Angular enterprise solutions. Former Senior Software Engineer at NTT Data and Satec. Authorized Google Workspace and Microsoft 365 Partner for Morocco.
