Managing test case status

  Work in Jama > Work in projects > Test management >

Managing test case status

Previous pageNext page Print this topic! Mail us feedback on this topic!

Include Unexecuted Test Runs in Status Calculations

The "Include Unexecuted Test Runs in Status Calculations" setting is under the Admin Organization Detail panel. This setting applies to test status values automatically calculated by Jama for Test Cases. By default, this calculation setting is checked for existing and new organizations, which includes unexecuted test runs in the calculation. Organizations that wish to only include executed test runs will want to uncheck this flag.

 

The following table walks through various scenarios around a specific Test Case used in a single Test Plan to illustrate the difference in calculations with the flag checked and unchecked.

Test Case Scenario

Test Case test status (with flag checked)

Test Case test status (with flag unchecked)

Cycle 1: Not Run

Cycle 2: Not Run

Scheduled/In Testing

Scheduled/In Testing

Cycle 1: Passed

Cycle 2: Not Run

Scheduled/In Testing

Passed

Cycle 1: Failed

Cycle 2: Not Run

Scheduled/In Testing

Failed

Cycle 1: Blocked

Cycle 2: Not Run

Scheduled/In Testing

Blocked

Not included in Cycle 1 or Cycle 2

Not Scheduled

Not Scheduled

Changing the "Include Unexecuted Test Runs in Status Calculations" setting does not retroactively get applied to existing Test Cases. The flag will apply any time the Test Case test status gets calculated in after the change. To trigger a recalculation of all Test Cases in a Test Plan, a user with sufficient permissions can archive and then un-archive the Test Plan. This two-step action will result in a re-calculation of the test status for all Test Cases in the Test Plan using the new calculation flag setting.

 

For a Test Case used in multiple active (un-archived) Test Plans, the roll-up calculation of the test status for that Test Case becomes more complex. The calculation will weigh the results of all active Test Plans against each other. If the Test Case is Failed in any active Test Plan, then the test status is Failed. Otherwise the roll-up calculation works like this:

Test Case is Blocked in any active Test Plan. Test status is Blocked.
Test Case is Passed in one or more active Test Plans. Test status is Passed.
Test Case is Scheduled/In Progress in any active Test Plan. Test status is Scheduled/In Testing.