CAPA effectiveness check: worked example and calculation

Review output and next actions

Effectiveness is not demonstrated by this population. Preserve the unfavourable and incomplete assessment. A later review must identify its new evidence and must not make this earlier review appear to have had it.

Define the operating population before calculating the rate.

Start from the effect the action was intended to produce. In this example the question concerns leakage during a defined pump operation. Select completed runs with comparable equipment and operating context. Record what qualifies as one event and how repeat observations during the same run are counted. If the event definition changes halfway through, the two populations are not automatically comparable.

Use the actual implementation evidence to place the change boundary. The date an action was approved or marked complete may not be the first run that used the revised procedure. If implementation was phased across equipment, retain that distinction. Keep runs with uncertain implementation exposure in an explicit unresolved or excluded population until the reviewer decides how to use them.

Calculate from totals and keep missing exposure visible.

For the combined post-change population, calculate 100 × (2 + 1 + 2) / (20 + 22 + 18) = 8.33 events per 100 runs. In general, use the combined event and exposure totals instead of averaging period percentages: an unweighted average gives a small period the same influence as a large one. Retain the counts behind the displayed rate so rounding never becomes the source of the assessment.

Attach the Python formula below to an Observed rate Number field on the same record as Eligible run count and Event count. These names must match the configured fields exactly. The formula leaves the rate empty when exposure is zero or missing, or the event count is unavailable or negative. Configure evidence and input-validity Checks separately; an empty calculation is not itself a publication Check. Do not substitute zero events per 100 runs when no runs have been observed. Missing event records are also different from confirmed zero events: reconcile the source population before treating an empty result as evidence of absence.

Keep the selected evidence or report population identifiable when the assessment is signed. A live dashboard is useful for finding new work, but its current values must not silently become the historical basis of an earlier decision. If additional runs are assessed later, retain the earlier population and a new assessment revision.

runs = entity["Eligible run count"]
events = entity["Event count"]
field.value = (
    None
    if runs is None or runs <= 0 or events is None or events < 0
    else 100 * events / runs
)

Separate an incomplete observation from an accepted effectiveness conclusion.

The illustrative target is not met by the available data: the observed rate is above 4 per 100 and only 60 of the planned minimum 100 runs are available. The review should report both facts. The authorised team decides whether further observation is justified, whether implementation needs investigation, or whether the action should be reassessed. Continuing to add runs until a favourable number appears is not a substitute for the original plan.

Do not automatically rewrite the original cause when an effectiveness check is unfavourable. Incomplete implementation, a changed operating population, a different event definition or an ineffective action can lead to different next work. Link each proposed explanation to evidence that could support or challenge it.

Audit which events and runs belong in the calculation.

Before aggregating the six periods, join each eligible run to the implementation state of its equipment and to the defined leakage events. Keep a dated extraction and a reviewed list of inclusions, exclusions and duplicate reports. A maintenance note and an incident report may describe one leakage event; two separate leaks during one run may instead be two events. Decide that from the event definition and source evidence, not from how many records the search returns.

The measure here is events per 100 runs. It is not the percentage of runs with a failure: if two distinct events occurred in one run, that run contributes two events but one exposed run. Do not switch between those measures before and after the action. Also compare equipment and operating conditions across periods; a shift to easier work could change the aggregate independently of the CAPA. The observed 8.00 and 8.33 rates alone establish neither statistical significance nor causation.

For the illustrative plan’s minimum of 100 post-change runs, the available 60 leave the planned observation incomplete. Record the current result now, the outstanding exposure and any recurrence requiring action under the plan. Collecting the remaining 40 runs can complete the exposure requirement; it cannot erase the five events already observed or justify changing the original acceptance criterion after seeing them.

Questions this example raises

Acceptance tests

Download these acceptance cases

These are proposed acceptance tests for the configured workflow. They describe expected behaviour to verify, rather than results of a customer implementation.

Review the evidence with neil

Review CAPA-071's effectiveness plan and implementation evidence. Reconcile the pre-change and post-change run populations, calculate leakage events per 100 eligible runs from the totals, and show exclusions or missing records. Compare the available evidence with the plan's exposure and outcome criteria. Prepare an assessment brief with separate conclusions on completeness, observed outcome and unresolved explanations; leave the effectiveness and closure decisions to the authorised reviewers.

Review the proposed output against these deliverables:

• A numerator-and-denominator table linked to the eligible runs and observed events.
• An explanation of the 8.00 versus 8.33 observed rates and the incomplete post-change exposure.
• A review brief listing the evidence needed to distinguish implementation, population and action-effectiveness questions.

Configure this process with Seal

Use a representative source record and an exception your current process struggles with. We can work through the record design, review responsibilities and connection requirements with you.