Technical and Capacity Challenges
Enhancements in information systems environments and increasing sophistication of policy officials as users of performance data are usually good signs for improving performance management in a government entity. However, these kinds of improvements also create special technical and capacity challenges both for managers and auditors of government performance measurement. For example, these challenges can involve:
Identifying and correcting internal control deficiencies in collecting and maintaining reliable data as information systems become more complex and change: Modern enterprise information systems tend to use data from distributed sources, which increases management’s capacity to calculate, track, and analyze performance measures. But information from distributed sources is inherently difficult to keep up to date and accurate. Yet without reliable data, performance reports are useless to decision makers. While more powerful information systems increase the capacity to analyze and report on performance measures, they do not assure elimination of control deficiencies or increased data reliability. For example, in performing Practice 1b. Audit performance management systems, the Texas State Auditor found that extensive reliance on automated systems did not necessarily increase the reliability of reported results. Automated systems can also add complexity to the auditor’s task of determining the adequacy of controls, especially as the systems change.
Integrating cost accounting and performance measurement data to support efficiency measurement: Policy officials often want to know the efficiency, or unit cost, of services to help them in making resource allocation decisions. But many state and local governments lack cost accounting systems essential to linking program inputs and outputs or outcomes. Without system support, information to derive efficiency measures is much more difficult to routinely collect, analyze, and distribute to decision makers.
Relying on data from outside your government entity to benchmark performance: Policy officials often want to know how performance of their government compares with the performance of other entities. To develop performance comparisons with other entities, organizations need to be able to rely on measures and supporting data being produced from external sources, yet they are not likely to be in a position to check on the internal controls to ensure reliability of data from these sources. Here are two approaches currently used by some auditors to at least partly address this issue, and a potential future approach:
Some local government auditors have recommended that management report data for measures of common services used by multiple jurisdictions and shared through the Center for Performance Measurement of the International City/County Management Association (ICMA). The ICMA Center tests the “reasonableness” of data before reporting on common measures to member jurisdictions (e.g., checks for “outliers” or unusual changes in data, and asks for correction or verification), which is helpful but not the same as rigorous reliability testing. Auditors have also used such comparative data in performance reports.
Some auditors have used citizen survey questions and scales used by other jurisdictions, and available through common survey providers, to obtain comparative results on citizen satisfaction and perceptions. As illustrations, see:
A potential future approach could involve imposing a “certification” or assurance process across organizations to improve consistency and accuracy of reported performance information. The efforts of provincial auditors and the federal Auditor General in Canada to develop performance report assurance as a common provincial government accountability practice may provide a start in this direction. In the U.S., auditors collecting comparative information from other jurisdictions for external performance reporting (Practices 3b. Collect data and 5b. Report performance) could try to work collaboratively with auditors in the comparator jurisdictions to establish minimum reliability standards they would all try to influence their entities to achieve.
|
