Use a test workspace with real Jira/GitHub connections for this walkthrough. The earlier browser checks used fixtures; these steps will also verify live integration.
Load the updated application
From the repository root, if the stack is stopped:
./run.sh start b
If it is already running, finish or stop active agent runs before restarting the backend:
./run.sh restart backend
This loads the new backend and applies its database migration. Vite should reload frontend changes automatically.
Open your workspace URL—for the existing local Oreo workspace:
https://oreo.lvh.me:5174
Sign in as a company administrator.
Prepare a small, verifiable cohort
Choose two or three Jira tickets belonging to the same project/site:
| Ticket | Useful test data |
|---|---|
| Feature | An Ergatis task linked to the ticket, with its GitHub PR merged recently |
| Unfinished feature | An open or unmerged PR |
| Bug | A known bug that can be attributed to the merged feature PR |
Choose tickets with relevant events in the last 30 days. Sprint verification additionally needs a feature completed in a closed Scrum sprint during that period.
In Integrations, confirm that a usable workspace Jira service connection has its site selected. Confirm the project's GitHub repository connection also works.
Open the new screen
Navigate to Insights → Pilot KPIs and select Last 30 days.
Expected: Project and Pilot selectors, New pilot, Refresh sources, and Save report appear. With no pilot selected, the page shows guidance instead of invented metrics.
Create the pilot
Select the appropriate Project, then click New pilot.
Fill the form:
| Field | Select or enter |
|---|---|
| Pilot name | FE verification pilot |
| Jira service connection | Your selected test site |
| Jira ticket keys | Your two or three keys, separated by commas |
| Feature issue types | The actual feature types used by those tickets |
| Bug issue types | The actual Jira bug type |
| Done statuses | The statuses your team considers completed |
| Human effort source | Recorded here |
| Replay start | Leave empty for this first test |
Hold Ctrl or Command to select multiple types/statuses.
For sprint verification, click Load sprint boards, select a Scrum board, and select its Sprint field. Otherwise leave the board empty.
Click Create pilot.
Expected: The form closes, the new pilot becomes selected, and source refresh starts. Definitions are fixed after creation; changing the measurement policy requires another pilot.
Verify source collection
Watch the Source refresh status until the job finishes. It polls automatically while queued or running.
Expected:
If collection fails or is partial, open Sources for … and inspect the reason. An inaccessible ticket must remain incomplete rather than become zero.
Check cycle time against Jira and GitHub
Click Sources for … on the delivered feature.
Verify that the Jira link, GitHub PR links, and linked Ergatis tasks belong to the correct ticket.
Calculate:
Cycle hours = (latest required PR merge time − Jira creation time) / 1 hour
Compare that with the ticket's Cycle hours. The source panel also shows Created → PR and Review/delivery.
Expected: The unfinished ticket has no complete cycle time. For a ticket with multiple required PRs, delivery requires all of them to merge. A closed, unmerged PR does not count as delivered.
Only exclude a PR when it is genuinely abandoned or optional; save that change with Save confirmation.
Verify manual human effort
In the delivered feature's source panel, under Add evidence:
30FE verification: review effortClick Record evidence.
Then check Human effort records cover this ticket, including steering and review, and click Save confirmation.
Expected: For a ticket with only this effort record, Human hours becomes 0.5. Before completeness confirmation, it remains unknown.
Agent hours should reflect the linked task attempts. Hours per ticket combines human and agent hours for eligible delivered tickets with complete evidence.
Verify an evidence correction
Enter a Correction reason, then click Revoke incorrect entry beside the effort record.
Expected:
Add the correct record and confirm completeness again. Check that the recalculated value matches the corrected records.
Verify defect attribution
Open the bug ticket's source panel.
Select Defect introduced, choose the actual responsible merged PR, enter the discovery time and supporting note, then click Record evidence.
Review defect attribution for every cohort ticket, checking the defect-review confirmation and clicking Save confirmation.
Expected: With fresh sources and complete review coverage, the introduced count reflects distinct attributed bugs.
To verify Defects resolved, record the resolution evidence on the bug. Its Jira status must be one of the configured done statuses, and its Jira resolution timestamp must fall within the selected period.
Without complete review coverage, defect cards should remain unknown.
Check sprint reporting and time ranges
If you selected a board, inspect Features in this cohort by sprint.
Compare a closed sprint with Jira's completion history.
Expected: It counts feature tickets from this selected cohort. Ergatis delivered separately identifies delivery through the linked PRs.
Switch between Last 24 hours, Last 7 days, and Last 30 days. Metrics should follow the applicable delivery, defect, and sprint completion dates. Tickets may remain visible as evidence even when their delivery falls outside the report period.
Verify pause and export
Click Pause collection.
Expected: Collection changes to paused and the button becomes Resume collection. An already-running job may finish; Refresh sources still works manually.
Click Save report.
Expected: A pilot-report-<id>.json file downloads, and the page confirms that the report was saved. Inspect the JSON for the selected pilot, period, metrics, and source evidence.
Save a second report after a correction. The first downloaded report should preserve its original values and evidence.
Check frontend regressions
Test these before signing off:
You can also rerun the frontend checks:
cd web-admin
npm test
npm run lint
npm run build
The implementation report contains reference screenshots and detailed limitations. Passing this walkthrough validates frontend behavior and live sources; release verification still has the previously reported dependency-security blockers.