Suites & test runs
Define reusable suites and open test runs against them; post results, retry, and attach evidence to a run.
POST /api/v1/test-runsSS-53 · Test suites, device farm & quality gates
Run a suite; produce signed release evidence.
What it does
The test and device lab behind a release: suites and test runs, a real-device farm with reservations and sessions, performance budgets, accessibility audits, flaky-test quarantine and human-in-the-loop sessions. A quality-gate evaluation turns run results into a pass/fail verdict, and release evidence is retrievable per release for audit.
In-mesh base http://ss-53-testgrid.shared-services.svc.cluster.local:8053 · auth is an SS-01 bearer token + X-Tenant-Id on every /api/v1 route.
Sub-capabilities
Define reusable suites and open test runs against them; post results, retry, and attach evidence to a run.
POST /api/v1/test-runsReserve real devices and open device sessions, so a run executes on hardware, not just emulators.
POST /api/v1/device-farm/reservationsEvaluate accumulated results against a gate to produce the pass/fail verdict a release is gated on.
POST /api/v1/quality-gates/evaluateDeclare performance budgets and run against them; run accessibility audits with retrievable findings.
POST /api/v1/performance-runsDetect flaky tests and quarantine them so they stop blocking merges; open human-in-the-loop sessions for judgment calls.
POST /api/v1/flaky-tests/{id}/quarantineWorked example
A suite runs on the device farm, accumulates results, and a quality-gate evaluation produces the verdict that gates the release — with signed evidence retained.
/api/v1/suitesA reusable suite of cases.
/api/v1/device-farm/reservationsReal hardware, not just emulators.
/api/v1/test-runsRun the suite; post results as they land.
/api/v1/quality-gates/evaluateTurn accumulated results into a pass/fail verdict.
/api/v1/evidence/{release}Signed, retrievable evidence for audit.
Endpoint catalog
Real method + path from services/testgrid/openapi.json. The full contract, including ops endpoints, is the 27-operation surface.
/api/v1/suitesList test suites/api/v1/suitesDefine a test suite/api/v1/suites/{id}Fetch one suite/api/v1/test-runsList test runs/api/v1/test-runsOpen a test run against a suite/api/v1/test-runs/{id}Fetch one test run/api/v1/test-runs/{id}/resultsPost results into a run/api/v1/test-runs/{id}/retryRetry a run/api/v1/test-runs/{id}/evidenceAttach evidence to a run/api/v1/device-farm/reservationsList device reservations/api/v1/device-farm/reservationsReserve real devices/api/v1/device-sessionsOpen a device session/api/v1/hil-sessionsList human-in-the-loop sessions/api/v1/hil-sessionsOpen a human-in-the-loop session/api/v1/quality-gates/evaluateEvaluate results into a pass/fail verdict/api/v1/performance-budgetsDeclare a performance budget/api/v1/performance-runsList performance runs/api/v1/performance-runsRun against a performance budget/api/v1/accessibility-auditsList accessibility audits/api/v1/accessibility-auditsRun an accessibility audit/api/v1/accessibility-audits/{id}Fetch one accessibility audit/api/v1/flaky-testsList detected flaky tests/api/v1/flaky-testsRecord a flaky-test signal/api/v1/flaky-tests/{id}/quarantineQuarantine a flaky test/api/v1/evidence/{release}Retrieve signed release evidence/api/v1/usageTenant usage ledger + SS-10 delivery status/api/v1/infoService info + capability envelopeArchitecture placement
Continuous TestGrid & device lab does not reimplement the platform — it composes it. These are the services it leans on and the seams between them.
A green TestGrid gate is a natural pre-condition for a SkyForge auto-deploy on a release branch.
TestGrid results feed the gate-stack verdicts the Factory merge decision requires green.
Signed release evidence per release is exported as compliance evidence.
Device-farm and run usage is metered into the tenant usage ledger.