1. Introduction
Test Suite Manager brings full test management into Jira without your test data ever leaving Atlassian. Test cases, cycles, plans, campaigns and requirements are native Jira work items; everything else lives in Atlassian-hosted Forge storage. There are no external services, no outbound calls, and no data egress — so the app qualifies for Atlassian's "Runs on Atlassian" program and inherits Atlassian's data residency and security posture.
You can author test cases as structured steps, Gherkin/BDD, or data-driven parameter sets; run them in a step-by-step Test Player with live status roll-up; trace significance-weighted coverage to requirements; import results from any CI; and report from a project Hub — all inside Jira, with a tamper-evident audit trail and compliance-grade governance.
2. Getting started
2.1 Installation
Install Test Suite Manager from the Atlassian Marketplace onto your Jira site. Because it is a Forge app, it runs entirely on Atlassian infrastructure — there is nothing to host and no credentials to configure.
2.2 Provisioning a project
Test Suite Manager stores test cases, requirements and cycles as dedicated Jira work types. An administrator provisions a project once; this creates the "Test Case", "Test Cycle", "Test Plan", "Test Campaign", "Test Case Execution" and "Requirement" work types and the links between them. All provisioning runs as the acting user, so Jira enforces your own permissions.
2.3 Where to find it
- Issue panel — on any issue in a provisioned project: the tabbed Test Suite Manager panel (Test Cases, Run, Requirements, Coverage).
- Project Hub — a project page with the summary, pass-rate trend and coverage.
- Project settings — governance: audit trail, auto-defect policy, CI API tokens and the migration wizard.
- Dashboard gadget — add "TSM Execution Summary" to any Jira dashboard.
- Rovo — chat with the "TSM Test Architect" agent.
4. Running tests
4.1 The Test Player
The Run tab lists the project's test cases with a Run button. Starting a run creates an execution from the case's latest version and opens the Test Player, where you set each step's outcome (PASS / FAIL / BLOCKED / SKIPPED). The overall execution status rolls up live as you record results.
4.2 Evidence & defects
Attach evidence (screenshots, logs, files, links or notes) to an execution or a specific step; file bytes are stored as Jira attachments (never outside Atlassian) and the app tracks a per-project storage rollup. Raise or link a Bug directly from a failed execution.
5. Organizing your suite
5.1 Cycles, plans & campaigns
Group work for a release with the hierarchy Campaign → Plan → Cycle → Test Case. A cycle collects cases into a runnable set (with progress and a "rerun failed" action); plans group cycles; campaigns group plans across a release, with a completion roll-up.
5.2 Collections & TQL
Static collections are named sets of test cases. Smart collections are defined by a query in TQL (Test Query Language) over fields such as type, format,folder and key, e.g. type = AUTOMATED AND format = GHERKIN. Membership is evaluated live; the preview shows matches before you save.
5.3 Baselines
A baseline is a named, point-in-time snapshot of the version each test case was at — immutable evidence once approved. Baselines can be compared to see what was added, removed or changed between two points in time.
6. Requirements & traceability
Link requirements to the test cases that verify them. The Requirements andCoverage tabs show significance-weighted coverage per requirement and the project coverage summary, and surface gaps (uncovered requirements). A traceability matrix rolls the whole picture up and can be exported.
7. Automation & CI/CD
7.1 Importing CI results
Import automated results in JUnit XML, TestNG XML, NUnit 3 XML or Cucumber JSON. Imports support a dry-run (summary without writing) and are idempotent — re-sending the same run with the same idempotency key won't import twice.
7.2 CI API tokens
In project settings, mint scoped API tokens for CI. The token is shown once at creation; only a hash is stored, so a leak never exposes a usable credential. Tokens can be revoked or rotated.
7.3 Auto-defect & change feed
An auto-defect policy can file defects for failing automated tests (off, on failure, or only on new failures — dedupe-aware). A poll-based change feed exposes an ordered, gap-free stream of activity for integrations, since Runs-on-Atlassian apps do not send outbound webhooks.
8. Reporting
8.1 The project Hub
The Hub project page shows the execution summary, a pass-rate trend with a sparkline, and requirement coverage — all read from your own project's data.
8.2 Test health & exports
Test-health analytics score each case from its execution history and flag FLAKY, SLOW, VALUABLE, UNUSED and STALE tests. Reports export to CSV, JSON and PDF (all generated on Atlassian, no external services).
8.3 Dashboard gadgets
Add the Test Suite Manager gadget to any Jira dashboard and configure it to show the execution summary, coverage or pass-rate trend for a chosen project.
9. Governance & compliance
9.1 Tamper-evident audit trail
Every governed change appends to a per-project, hash-chained audit trail. The Settings page shows the events with an integrity badge; a verification re-walks the chain and flags the first event if anything was altered or removed — something no other Jira test tool provides.
9.2 Roles & folder permissions
Assign roles (Test Lead, Tester, Auditor, or custom) that grant capabilities, and set per-folder VIEW / EDIT / DENY permissions that inherit down the folder subtree. These narrow, never widen, what Jira already permits.
9.3 E-signatures
A 21 CFR Part 11-style e-signature engine lets users re-authenticate with a private signing PIN (stored only as a salted hash) to sign an artifact with a recorded meaning. Each signature is cryptographically bound to what it signed.
9.4 Trash & data export
Deletes are reversible from a trash view (restore or permanently purge). Full project export and per-account (DSAR) export produce portable JSON of your Test Suite Manager data.
10. AI: the Test Architect
The optional "TSM Test Architect" Rovo agent runs on Atlassian's own Rovo AI — there is no external model call and no data egress, so the app keeps its Runs-on-Atlassian status. The agent finds coverage gaps, reasons over your test cases, and drafts test cases from requirements.
Deterministic helpers need no AI at all: all-pairs (pairwise) dataset generation, flaky-test detection and near-duplicate detection.
11. Migrating from other tools
The migration wizard in project settings parses an exported test suite from Xray, Zephyr Scale or TestRail and runs a dry-run analysis — counts by format, total steps and warnings — so you can see exactly what will import before committing.
12. Privacy & security
- Zero egress. No remotes, no external fetch, no web-triggers. Data stays in Atlassian (Forge SQL/KVS + Jira issues).
- Acts as you. Every Jira read and write runs as the acting user, so Jira enforces your own permissions.
- Personal data. Only Atlassian account IDs are stored (in audit, signatures, roles, assignees); a weekly privacy poller and account-export/erasure paths support GDPR.
- Full policies: Privacy, Security, EULA.
13. Support
Questions or issues? Use the ChefStackz support portal at chefstackz.atlassian.net/helpcenter/CS or email support@chefstackz.com. Vendor hours are Mon–Fri, 9:00–17:00 EEST.