ROLE 2: Assess Performance Information:
Practice 2a.Test relevance or reliability: Test or certify performance measurement relevance, reliability, or both.
Kansas City, Missouri, City Auditor [Profile]
The City Auditor will test for both relevance and reliability of agency performance measures during the course of performance audits or of special studies of performance measures (see Practice 3a. Help choose measures or targets). While City Auditor staff test for many similar relevance and reliability factors as other offices, two of their steps are a bit different. In testing relevance, in addition to determining whether existing measures provide a valid, complete picture, auditors look for comparability with other organizations that provide the same service, and may recommend that an agency adopt measures commonly used by other local governments to enable both local and national comparisons. In addition to typical steps for testing reliability, auditors will talk directly with the people who collect the data to determine if they understand the reasons why they are supposed to gather data in a specific way. When personnel are just “doing what they’ve been told” to collect data without knowing why, auditors consider it a risk factor warranting further examination.
More information on how the City Auditor’s Office tests for relevance and reliability, including types of comparisons sought and sources of comparison data, and additional reliability testing steps [Example in 2004 Guide-PDF] |
