CI tests are standardized evidence
Local tests provide fast feedback, while CI repeats key assumptions in a clean standardized environment. A backend job may combine unit or service tests, integration or API checks, JAR build, and a health-endpoint smoke check.
Task names and test services depend on the project. The workflow should use the same basic commands as README and local development.
| Layer | Example signal |
|---|---|
| Unit or service | Local business logic and rules |
| Integration or API | Service, database, configuration, and contract interaction |
| Build | Compilation, dependencies, and packaging |
| Smoke | Start and health endpoint |
| Frontend | Install with lock file, tests, build, and short start |
How to read a red job
- Find the step that failed.
- Locate the first meaningful error message.
- Separate the primary error from cascading messages.
- Compare the failure with changed files and job configuration.
- Repeat only the related job after the smallest fix.
Failure classification
A failing test does not automatically prove that the product is broken. The behavior may have changed intentionally while the test remained old. Compare the assertion with the business rule and log evidence.
| Category | Signs | First direction |
|---|---|---|
| code | Exception, wrong response, or wrong result | Changed code and its contract |
| test | Outdated or incorrect expectation | Test alignment with the business rule |
| environment | Port, service, or environment is missing | Runtime and job configuration |
| dependency | npm or Gradle, lock file, or incompatible version | Dependencies and wrapper |
| flaky | Unstable result without a code change | Repeatability and run history |
| infrastructure or permissions | Runner, network, timeout, or access issue | CI state and permissions |
Claude as diagnostician, not automatic repairer
- Pass only the relevant log and changed-file list.
- Ask Claude to distinguish symptom, hypothesis, and confirmed fact.
- Check the hypothesis locally or with a narrow rerun.
- Make the smallest fix after confirmation.
- Do not accept an AI conclusion without primary evidence.
Input: failing job log and changed-file list
Return:
- failure category
- probable root cause
- evidence from the log
- smallest verification command
- next action
Do not edit files, disable tests, or hide the failure.Flaky tests, reruns, and evidence
One successful rerun does not prove that a problem disappeared; it only shows that the test can pass unstably. Record frequency, conditions, logs, and the deliberate decision to isolate or fix the cause.
Temporarily disabling a test makes a pipeline green while removing a regression signal. Classify the cause as code, test, environment, dependency, flaky, or infrastructure instead.
| Event | Evidence to record |
|---|---|
| Failed job | Step name, first error, command, and commit context |
| Hypothesis | Why it explains the symptom |
| Smallest fix | Changed files and non-goals |
| Repeat check | The same scenario and actual result |
| Flaky signal | Several runs, conditions, and follow-up decision |
Canonical lesson source · JavaRush
Open JavaRush material