Emerging Criteria Auditors Can Use
Some convergence has begun to emerge relating to criteria used for certain practices. Auditors are still experimenting with practices relating to performance measurement, but some commonly used criteria have begun to emerge, especially criteria for testing relevance and reliability of performance information, and criteria for performance reporting. As criteria to guide auditors’ assessments is an important part of many audit practices, convergence of criteria used by audit organizations can be an important part of the evolution of standard practice across the profession.
Criteria for Testing Relevance and Reliability of Performance Information
On a broad level, some convergence seems to be emerging on criteria for practice 2a: test relevance or reliability of performance information. Research in 2003-04 examined surveyed auditors who use this practice on the kinds of things they checked for when they test relevance and reliability. Their answers suggest strong convergence of relevance and reliability criteria at least at the broad level of key questions they try to answer by their testing. Click here for these questions and comparative results (DOC), published in 2004 in Auditor Roles in Performance Measurement by The IIA Research Foundation.
Commonly Used Criteria for Testing Relevance
Relevance criteria most strongly converges on alignment of performance measures with mission, strategy, or goals. There is also strong agreement on relevance criteria concerning whether measures provide users or stakeholders with performance information they need on time, and whether measures can be used to benchmark, compare, or trend performance.
Commonly Used Criteria for Testing Reliability
Reliability criteria most strongly converges on whether measure definitions are adequate and whether data supporting the measures are accurate and complete. Several Role 2 examples include more specific criteria and testing steps used by audit organizations.
Specific Criteria, Related Tools and Examples
See Tools for Practice 2a: Test Relevance or Reliability for specific criteria that have been used by several audit organizations whose practices are featured on this website. See Practice 2a for examples of how audit organizations test relevance and reliability. In addition to practice 2a examples, some of the examples for Practice 2b: Assure Performance Reports involve assessing relevance or reliability
Criteria for Performance Reporting
In both Canada and the United States, standards setting bodies and professional organizations have been active in projects involving principals, recommended practices, or suggested criteria that can be used by auditors for several practices of Role 2: Assess Performance Information, Role 3: Define or Measure Performance, and Role 5: Assist Elected Officials or Citizens.
Performance Reporting Principles and “Recommended Practice” in Canada
In Canada, the Public Sector Accounting Board (PSAB) of the Canadian Institute of Chartered Accountants (CICA) issued a “Statement of Recommended Practice SORP-2: Public Performance Reporting” in June 2006, and in May 2007, published Public Performance Reports: A Guide to Preparing Public Performance Reports (PDF) which can serve as criteria for auditors to use for Practice 2b. Assure Performance Reports. Years before this PSAB guidance was issued, common principles for performance reporting began to emerge in Canada, which in turn had started driving convergence on criteria for assuring external performance reports. The federal Auditor General and the Canadian Council of Legislative Auditors (CCOLA) stimulated learning among government auditors by forming, in 1997, the Performance Reporting and Auditing Group, which has been sharing practices in auditing performance information. Starting with a set of performance information and reporting criteria proposed by the CCOLA in 1999, discussions among auditors and sharing of practices led to performance reporting principles first issued by the CCOLA, later revised and issued by CCAF-FCVI (formerly the Canadian Comprehensive Audit Foundation) in 2002. The CCAF-FCVI principles were locally adapted and used as criteria by auditors in several provinces, including British Columbia and Quebec. The PSAB Statement of Recommended Practice is largely based on the CCAF-FCVI principles.
“Suggested Criteria” for Performance Reporting in the United States
In the U.S., the 2003 “Suggested Criteria for Effective Communication” of performance information published by the Governmental Accounting Standards Board (GASB) is a source for criteria for Practice 2b. Assure Performance Reports. For several years, the GASB criteria have been used in a review process by the Association of Government Accountants (AGA) to assess state and local government performance reports and issue “Certificates of Achievement” to reports that do well in following the criteria. Government auditors are among the volunteer reviewers used by the AGA in this process, suggesting that the GASB criteria can also be useful for Practice 2c. Support External Review processes, at least those involving external review of performance reports. In April 2007, the GASB voted to put consideration of issuing voluntary guidance on state and local performance reporting on its technical agenda, suggesting that a more formal pronouncement by GASB may be coming within the next year or two, which may involve a revision of the current suggested criteria based, in part, on reporting experiences of state and local governments since 2003.
Criteria Useful for Auditors Who Issue Performance Reports or Help Select Measures
Auditors who produce and issue external performance reports (Practice 5b. Report Performance) will find the emerging criteria on performance reporting in the U.S. and Canada useful, as well as those engage in Practice 2b: Assure Performance Reports. Also, as the Canadian principles and U.S. (GASB) suggested criteria include guidance concerning what kinds of information should be reported, so these emerging criteria should also be useful for auditors who provide advice on determining performance measures or performance expectations (Practice 3a: Help Choose Measures or Targets).
Specific Criteria, Related Tools and Examples
|
