One of the OPEN Annual Conference afternoon workshops was “What Program Evaluators Can Learn from How We Performancy Auditors Do Our Work” from Gary Blackmer, Oregon Audits Director from the office of the Oregon Secretary of State. He referred to the 2007 Government Auditing Standards definition of performance auditing, pointing out that information resulting from audits is intended by use of those charged with governance and oversight to improve program performance and operations, reduce costs, and facilitate decision making. How does this differ from program evaluation? Auditors are required to follow the Government Auditing Standards from the Government Accountability Office (also known as “The Yellow Book”). Auditors are organizationally independent from the entities being audited; they do not negotiate scope, objectives, or access to data; and they always produce public reports. He pointed out that auditors make it a practice to spend one-third of their time conducting assessment and developing their audit plan, one-third gathering evidence, and one-third producing reports of findings. (Audience members generally agreed that evaluators tend to spend less time than that on planning and reporting.)
Published audit reports are typically intended to provide a window into an organization–a portrait for the public–with referrals to separately published working papers that provide details about methodology, data, and analysis. The speaker observed that bad news doesn’t travel up very well and we don’t get rewarded for doing things wrong. Organizations often only want to be assessed on things that can be controlled, leading to an emphasis on process rather than outcomes. Auditors sometimes have a more accurate view of “reality” than management and, emphasizing that there is always room for improvement in every organization, provide “bad news” (ie, suggestions regarding changes) in doses that can be tolerated.
Handouts and powerpoints from this workshop will be made available at the OPEN web site.