ROLE 2: Assess Performance Information:
Practice 2a.Test relevance or reliability: Test or certify performance measurement relevance, reliability, or both.
Austin, Texas, City Auditor [Profile]
Since 1985, the Office of the City Auditor (OCA), which reports to the City Council, has been reviewing City departments’ performance measures as part of performance audits, and did so on a citywide basis several times as part of system reviews from 1994 to 2002 (see Practice 1b. Audit performance management systems). Since 2002, a separate audit office that reports to City management, the Austin Corporate Internal Auditor, has been conducting formal performance measurement certification audits. However, when auditing departments and programs, OCA still may include audit objectives to check the reliability of performance information and the information systems that support reported performance measures. For example, an OCA audit in 2007 assessed the reliability of the City’s Water Conservation program’s reported water savings, and this included an assessment of the systems that produced the relevant measures.
In implementing reliability testing, OCA has made increasing use of software tools such as ACL to perform data mining and analysis of entire data universes with less reliance on sampling, or ArcGIS for testing the completeness and accuracy of geographic information that may be used in performance measurement. OCA has also developed guidance for testing data reliability (see “tools”), which documents what OCA would do to assess the reliability of information that supports reported performance measures.
Tools:
The following documents provide guidance on testing data reliability and are also available on the OCA website:
|
