ROLE 2: Assess Performance Information:
Practice 2a.Test relevance or reliability: Test or certify performance measurement relevance, reliability, or both.
Nevada Legislative Auditor [Profile]
Nevada State Agencies are required by law to have performance measures in their budget (NRS 353). In 2001, in response to this requirement, the Legislative Auditor conducted an audit examining the reliability of measures considered important to the legislature and to public understanding of performance, based in part on the dollars appropriated to each program. The Legislative Auditor focuses on performance measurement reliability. Executive branch agencies receive guidance on what measures are relevant to report from the State Department of Administration’s Budget and Planning Division. The Legislative Auditor reviewed 35 measures across various state agencies for reliability, found that about half were not reliable, and made recommendations for corrective actions. Two Audit Division staff devoted approximately 2,400 total hours to this audit over an 11-month period. Audit Division staff followed a methodology for each measure selected that included interviewing agency personnel, documenting the measurement process, and assessing the adequacy of the control environment. They also evaluated the reasonableness of the agency’s methodology to measure each performance indicator. In one case, for example, the measurement method for a customer waiting time was found to be unsound because the agency tracked only one step of a three step process. Audit staff then examined documentation supporting the reported measure for completeness and tested calculations for accuracy. The acceptable margin of error between actual and reported performance varied depending on the nature of the measure. After completing testing procedures, auditors met with agency management to discuss their conclusions regarding the reliability of the selected performance measures. They also discussed any weaknesses noted in process design and controls including written procedures and supervisory review. More recently, some of the Legislative Auditor’s audits have included a review of the reliability of the performance measures presented in the Governor’s Executive Budget. As a result of this process, legislators are showing increased interest in agencies’ performance measures and the Department of Administration has established guidance to help ensure the reliability of performance measures.
Reports:
Highlights of Reliability of Performance Measures Used in the State's Budget Process, (January 2002) (PDF) |
