Back to product

Test Suite Manager for Jira

Zero-egress test management: author, run, trace, and report — without your data leaving Atlassian

Version 1.0 · Updated July 2026

On this page

1. Introduction

Test Suite Manager brings full test management into Jira without your test data ever leaving Atlassian. Test cases, cycles, plans, campaigns and requirements are native Jira work items; everything else lives in Atlassian-hosted Forge storage. There are no external services, no outbound calls, and no data egress — so the app qualifies for Atlassian's "Runs on Atlassian" program and inherits Atlassian's data residency and security posture.

You can author test cases as structured steps, Gherkin/BDD, or data-driven parameter sets; run them in a step-by-step Test Player with live status roll-up; trace significance-weighted coverage to requirements; import results from any CI; and report from a project Hub — all inside Jira, with a tamper-evident audit trail and compliance-grade governance.

2. Getting started

2.1 Installation

Install Test Suite Manager from the Atlassian Marketplace onto your Jira site. Because it is a Forge app, it runs entirely on Atlassian infrastructure — there is nothing to host and no credentials to configure.

2.2 Provisioning a project

Test Suite Manager stores test cases, requirements and cycles as dedicated Jira work types. An administrator provisions a project once; this creates the "Test Case", "Test Cycle", "Test Plan", "Test Campaign", "Test Case Execution" and "Requirement" work types and the links between them. All provisioning runs as the acting user, so Jira enforces your own permissions.

2.3 Where to find it

  • Issue panel — on any issue in a provisioned project: the tabbed Test Suite Manager panel (Test Cases, Run, Requirements, Coverage).
  • Project Hub — a project page with the summary, pass-rate trend and coverage.
  • Project settings — governance: audit trail, auto-defect policy, CI API tokens and the migration wizard.
  • Dashboard gadget — add "TSM Execution Summary" to any Jira dashboard.
  • Rovo — chat with the "TSM Test Architect" agent.

3. Authoring test cases

3.1 Creating a test case

Open the issue panel and use the Test Cases tab. Enter a summary and create the case; it appears in the list with its Jira key and type. Each test case is a real Jira issue, so it is searchable, linkable and permission-controlled like any other work item.

3.2 Steps, Gherkin & data-driven

  • Steps — ordered action / test-data / expected-result rows.
  • Gherkin/BDD — author a .feature-style body (Feature, Background, Scenario, Scenario Outline with Examples). It is parsed into executable steps, expanding outline examples with <param> substitution.
  • Data-driven — attach a dataset of parameter rows; step${param} placeholders are substituted to produce one iteration per row. Deterministic all-pairs generation can produce a compact covering set of combinations.

3.3 Versions & review

Editing a test case creates versions; a new version copies the latest steps forward as a draft, and approving freezes it. Executions reference a specific version, so historic runs always render the exact steps that were run. A review workflow moves versions through draft → review → approved.

3.4 Preconditions & call tests

Shared preconditions are authored once and attached to many cases. Call tests let a version "call" other test cases (Xray-style modular tests); at execution the called steps are inlined. Cyclic calls are rejected at author time.

4. Running tests

4.1 The Test Player

The Run tab lists the project's test cases with a Run button. Starting a run creates an execution from the case's latest version and opens the Test Player, where you set each step's outcome (PASS / FAIL / BLOCKED / SKIPPED). The overall execution status rolls up live as you record results.

4.2 Evidence & defects

Attach evidence (screenshots, logs, files, links or notes) to an execution or a specific step; file bytes are stored as Jira attachments (never outside Atlassian) and the app tracks a per-project storage rollup. Raise or link a Bug directly from a failed execution.

5. Organizing your suite

5.1 Cycles, plans & campaigns

Group work for a release with the hierarchy Campaign → Plan → Cycle → Test Case. A cycle collects cases into a runnable set (with progress and a "rerun failed" action); plans group cycles; campaigns group plans across a release, with a completion roll-up.

5.2 Collections & TQL

Static collections are named sets of test cases. Smart collections are defined by a query in TQL (Test Query Language) over fields such as type, format,folder and key, e.g. type = AUTOMATED AND format = GHERKIN. Membership is evaluated live; the preview shows matches before you save.

5.3 Baselines

A baseline is a named, point-in-time snapshot of the version each test case was at — immutable evidence once approved. Baselines can be compared to see what was added, removed or changed between two points in time.

6. Requirements & traceability

Link requirements to the test cases that verify them. The Requirements andCoverage tabs show significance-weighted coverage per requirement and the project coverage summary, and surface gaps (uncovered requirements). A traceability matrix rolls the whole picture up and can be exported.

7. Automation & CI/CD

7.1 Importing CI results

Import automated results in JUnit XML, TestNG XML, NUnit 3 XML or Cucumber JSON. Imports support a dry-run (summary without writing) and are idempotent — re-sending the same run with the same idempotency key won't import twice.

7.2 CI API tokens

In project settings, mint scoped API tokens for CI. The token is shown once at creation; only a hash is stored, so a leak never exposes a usable credential. Tokens can be revoked or rotated.

7.3 Auto-defect & change feed

An auto-defect policy can file defects for failing automated tests (off, on failure, or only on new failures — dedupe-aware). A poll-based change feed exposes an ordered, gap-free stream of activity for integrations, since Runs-on-Atlassian apps do not send outbound webhooks.

8. Reporting

8.1 The project Hub

The Hub project page shows the execution summary, a pass-rate trend with a sparkline, and requirement coverage — all read from your own project's data.

8.2 Test health & exports

Test-health analytics score each case from its execution history and flag FLAKY, SLOW, VALUABLE, UNUSED and STALE tests. Reports export to CSV, JSON and PDF (all generated on Atlassian, no external services).

8.3 Dashboard gadgets

Add the Test Suite Manager gadget to any Jira dashboard and configure it to show the execution summary, coverage or pass-rate trend for a chosen project.

9. Governance & compliance

9.1 Tamper-evident audit trail

Every governed change appends to a per-project, hash-chained audit trail. The Settings page shows the events with an integrity badge; a verification re-walks the chain and flags the first event if anything was altered or removed — something no other Jira test tool provides.

9.2 Roles & folder permissions

Assign roles (Test Lead, Tester, Auditor, or custom) that grant capabilities, and set per-folder VIEW / EDIT / DENY permissions that inherit down the folder subtree. These narrow, never widen, what Jira already permits.

9.3 E-signatures

A 21 CFR Part 11-style e-signature engine lets users re-authenticate with a private signing PIN (stored only as a salted hash) to sign an artifact with a recorded meaning. Each signature is cryptographically bound to what it signed.

9.4 Trash & data export

Deletes are reversible from a trash view (restore or permanently purge). Full project export and per-account (DSAR) export produce portable JSON of your Test Suite Manager data.

10. AI: the Test Architect

The optional "TSM Test Architect" Rovo agent runs on Atlassian's own Rovo AI — there is no external model call and no data egress, so the app keeps its Runs-on-Atlassian status. The agent finds coverage gaps, reasons over your test cases, and drafts test cases from requirements.

AI is consent-gated per project, records provenance and meters usage. The agent proposes — you review and save. It never persists test cases on its own.

Deterministic helpers need no AI at all: all-pairs (pairwise) dataset generation, flaky-test detection and near-duplicate detection.

11. Migrating from other tools

The migration wizard in project settings parses an exported test suite from Xray, Zephyr Scale or TestRail and runs a dry-run analysis — counts by format, total steps and warnings — so you can see exactly what will import before committing.

12. Privacy & security

  • Zero egress. No remotes, no external fetch, no web-triggers. Data stays in Atlassian (Forge SQL/KVS + Jira issues).
  • Acts as you. Every Jira read and write runs as the acting user, so Jira enforces your own permissions.
  • Personal data. Only Atlassian account IDs are stored (in audit, signatures, roles, assignees); a weekly privacy poller and account-export/erasure paths support GDPR.
  • Full policies: Privacy, Security, EULA.

13. Support

Questions or issues? Use the ChefStackz support portal at chefstackz.atlassian.net/helpcenter/CS or email support@chefstackz.com. Vendor hours are Mon–Fri, 9:00–17:00 EEST.