Christine L. Cohen*

"... The five internal auditors have an average of 1.6 years experience with the County and an average of 3.7 years total audit experience." Response: Partially agree. The 1.6 years of experience with the County refers to County audit experience only, and, as of April 15, 2010, the five internal auditors had an average of 4.0 years total audit experience, not 3.7 years as stated by the Grand Jury.

through F-04. Agree.

"... Auditors are expected to obtain 80 hours Continuing Professional Education training per year in a combination of qualified at-work training and outside training courses." Response: Respectfully disagree. The 80-hour requirement is every 2 years, not annually.

"The personnel turnover rate in the IAD ranges from 25% to 50% per year for the prior three years..." Response: Respectfully disagree. How the Grand Jury calculated the turnover rate is not clear. For example, our calculations show that the turnover rate during FY 2008-09 was 14 percent (i.e., not within the range of 25 to 50 percent).

through F-17. Agree.

"...Larger agencies and departments are subdivided into 'Budget Units' for CSA Program purposes; there are a total of 52 CSA budget units for the County..." Response: Respectfully disagree. What the Grand Jury appears to be referring to is the number of program areas or divisions reportedly reviewed during the CSA, which may contain multiple budget units. Our records show that the number of budget units expressly reviewed by departments during the CSA totaled 125.

through F-20. Agree.

"Of the 26 budget units, 18 had submitted CSAs by the end of 2007..." Response: Respectfully disagree. The Grand Jury appears to be referring to the 26 agencies and departments, not 26 budget units. Of the 26 agencies and departments, 25 submitted CSAs by the end of 2007. The remaining department submitted their CSA on January 7, 2008.

through F-23. Agree.

"Budget units report concerns regarding the CSA process: "having to 'train and re-train' new auditors" • 3 Response: Requires clarification. Only two auditors performed work on the CSA validations, although the Grand Jury's statement inappropriately appears to imply a rotation of multiple auditors. In addition, use of the word "train" inappropriately appears to imply that the auditors assigned to the validations were not knowledgeable on audit or CSA, when this was not the case. As with every engagement, departments must inform the auditors on their processes so that the auditors can make a proper evaluation on the adequacy of those processes. "while budget units are encouraged to tailor the CSA guides to their own organizations, there is ٠ little practical direction from IAD in doing so" Response: Respectfully disagree. The few departments that requested assistance received personal training from the IAD. "validations may be performed on comparatively old information, requiring re-doing a self- assessment in order to complete the validation" Response: Respectfully disagree. The validation process did not require any "re-doing" of the self-assessments. The validation process evaluated how departments performed their self- assessments and evaluated whether internal controls were in place and operating effectively as departments claimed.

though F-31. Agree. Responses to Conclusions (Limited to those applicable) C-02. "The IAD audit staff continues to lack overall audit experience and experience with the County." Response: Respectfully disagree. Although we technically agree that IAD staff auditors do not have a large number of years of experience (if this is the basis for "experience" used by the Grand Jury), we have found that basic audit competencies can be obtained by quality auditors within the first few months of employment and within the first few engagements assigned. We believe the mark of a good auditor includes not only attainment of those competencies, but the ability to apply critical thinking within every step of the engagement. While years of experience may help to develop those critical thinking skills, other times it may not. Therefore, to generalize and imply that a lack of number of years of experience equates to a lack of competency is not appropriate. Also, professional internal audit experience as well as accounting related experience gained in other sectors is generally beneficial to conducting internal audits. Further, the collective experience of the IAD should be taken into account, which includes the experience of the Audit Chief Deputy. The collective years of total audit experience was over 30 years for the IAD team as of April 15, 2010. C-03. "The A-C has established no special training plan or an extra effort for training inexperienced auditors beyond the A-C's standard training budget. There appears to be no training plan directed specifically toward junior personnel, other than close supervision." Response: Respectfully disagree. Although auditing standards require 80 hours of formal CPE every 2 years, our records show that for the 2-year period ending December 31, 2008, the six collective IAD team members as of that date exceeded the required number of hours by an average of nearly 70 hours per auditor. Training of auditors also involves ongoing collaboration with the Audit Chief Deputy and other auditors, including weekly meetings with the Audit Chief Deputy, biweekly roundtable discussions among staff auditors, and monthly formal staff meetings. Further, depending on the experience level of new auditors, junior auditors are assigned as appropriate to more senior auditors with whom to work. In addition, new auditors are allocated up to 200 hours to orient themselves on IAD procedures and for on- the-job training. C-05. "Recruitment and retention of audit staff is suffering because HR and A-C have not updated auditor salary schedules to be competitive with those of other Southern California counties." Response: Partially agree. Recruitment and retention has improved in the short-term, but we agree that a long-term solution is needed. We have been pursuing such a solution with HR since 2006, offering a variety of salary studies, suggested re-classifications, and requested MOA changes as items for consideration. C-07. "... The CEO's reports incorporate the substance of the A-C's report, but the serial process invites CEO filtering, diminishes the perception of auditor independence, and may reduce BOS visibility." Response: Respectfully disagree. The CEO properly coordinates the periodic BOS status reports with the A-C and properly takes into account any comments by the A-C. C-10. "CSA validations have been scheduled and conducted two or more years after the date of a department self-assessment. The delayed validation can detract from the perceived importance of the program." Response: Respectfully disagree. The CSA validation process was designed to be conducted within the 3-year time frame of the CSA Program. During the validations conducted, departments did not disclose any significant changes in their systems of internal control since the date of the self-assessment. Also, the true value of the CSA Program does not lie with the A-C's validation process. The true value is obtained during the department's self-assessment of internal controls and the self-directing of internal control improvements. C-11. "With little feedback from IAD or County management upon completion of the CSA, individual departments may have filled out the questionnaire forms on schedule, but find out only during a much later validation, if ever, that the self-assessment may have been incomplete or superficial." Response: Partially agree with comment. As stated in our response to C-10, the CSA validation process was designed to be conducted within the 3-year time frame of the CSA Program. In addition, see our response to R-06 for improvements planned for the CSA Program. C-12. "The validation process has proven to be laborious and duplicative. With the high turnover in auditors, there is lack of auditor continuity from performance of the CSA to validation; additional time is also required on the part of the audited organization and the audit staff." Response: Respectfully disagree. Although true that the CSA process involves additional time of departments and audit staff, we disagree that the validation process is duplicative inasmuch as any audit process that involves confirmation of performance is seen as duplicative. In addition, as stated in our response to F-24, only two auditors performed work on the CSA validations, which is not indicative of serial auditors. Further, auditor continuity is not relevant "from performance of the CSA to validation" as the Grand Jury concludes because the CSAs are performed by the departments (as the term "self-assessment" suggests), not the auditors. C-14. "The A-C's IAD is presently understaffed and insufficiently experienced to lead effective agency/department CSAs and to perform validations in an efficient manner." Response: Respectfully disagree. As previously provided to the Grand Jury, the IAD has benchmarked its staffing levels with those of 32 other California counties in terms of the total number of county employees per auditor and the budget coverage per auditor. In each case, the total staffing of eight IAD members approximated the average across the state, which indicates to us an appropriate staffing level. In addition, we believe the staff who completed the CSA validations was sufficiently experienced and competent. The IAD is currently fully staffed and has the professional experience to effectively lead the CSA Program and to conduct independent and objective validations in accordance with the International Standards for the Professional Practice of Internal Auditing. C-15. "The CSA Program is not being executed timely or effectively." Response: Respectfully disagree. See our responses to F-24, C-10, C-11, C-12, and C-14. C-18. "The A-C is responsible for oversight and direction of the CSA Program with insufficient resources, but the responsibility for performance and execution belongs to County agencies and departments for which the CSA Program is a low priority. Consequently, the CSA Program lacks thoroughness and urgency. As currently conducted by the County, the CSA Program provides the illusion of safety without the substance." Response: Respectfully disagree. As stated in our response to the Summary above, the CSA Program is only one initiative within the County's internal control structure to promote public accountability. The CSA Program should not be misconstrued by the Grand Jury to provide absolute assurance of safety. We believe use of the phrase "illusion of safety" is inflammatory and a disservice to the County's efforts to promote good internal controls. In addition, as noted in the Grand Jury's Finding F-31, the resources allocated to the CSA Program were approximately 10 percent of direct hours and we have concluded that this is an adequate amount for CSA Program oversight and assistance. C-20. "Continuation of the CSA into a third triennium with no substantial changes in process, and with no enhancement of supporting audit resources, would be ill-advised." Response: Respectfully disagree. As stated in our response to the Summary and C-10 above, CSA is a best practice that derives much of its value from serving as a reminder to departments of their internal control responsibilities, providing tools for self-assessment, and requiring periodic reporting of self- assessment results. We also believe audit resources allocated to the CSA Program are sufficient as stated in our response to C-18. That being said, the IAD had previously identified areas where the CSA Program could be further improved, which is addressed in our response to R-06. C-21. "In the first three quarters of the FY 2009-10, the IAD had expended only 32% of its planned CSA hours for the year, for which understaffing appears to be the major contributor." Response: Requires correction. As stated in the Grand Jury's Finding F-31, the IAD spent 293 of 960 FY 2009-10 planned hours on the CSA as of March 31, 2010, or 30.5 percent, not 32 percent as the Grand Jury states. Responses to Recommendations