Illusion of Safety: Control Self-Assessment

The A-C IAD is authorized eight positions. The IAD has entered fiscal years (FY) 2008-09 and 2009-10 with five of the authorized positions filled. For FY 2009-10, two individuals remained from the previous year, three had been hired within the year.

As of April 15, 2010, IAD has seven positions filled: four Internal Auditor II, one Internal Auditor III, one newly-hired Fiscal Manager, and the Deputy Director. The five internal auditors have an average of 1.6 years experience with the County and an average of 3.7 years total audit experience.

The position of Fiscal Manager I/II (a supervisory position) was recently filled.

Until the fiscal manager position was filled, the IAD’s junior auditors received formal oversight only by the IAD director.

The A-C budgets 640 hours per year per auditor for “indirect time— training, staff meetings, administrative projects, etc.,” of which 280 hours represent standard vacation, sick and holiday accruals and eight hours represent “in-house training to increase staff knowledge of internal auditing techniques.” Auditors are expected to obtain 80 hours Continuing Professional Education training per year in a combination of qualified at-work training and outside training courses. ________________________________________________________________ 2 Illusion of Safety: Control Self-Assessment

The personnel turnover rate in the IAD ranges from 25% to 50% per year for the prior three years, as compared to the average turnover rate for the County as a whole of about 12%.

HR performs exit interviews of departing employees through the use of a form to be completed after the employee’s last day of work and to be returned by mail.

In reviewing mailed-in exit interviews of former internal auditors, HR has found nothing to indicate a possible problem with working conditions within the County or the A-C organization.

Classification Specifications (similar to job descriptions) and salary ranges for A-C internal auditors were adopted in 2001. The salary ranges have been modified only for general “cost of living” increases negotiated between the County and the Service Employees International Union. [Ref-04] Job Title Rate Range Experience: Annual Salary “Recruiting Standard” Internal Auditor I $41,340 - $52,916 None Internal Auditor II $47,997 - $61,436 18 months Internal Auditor III $52,797 - $67,580 3 years Internal Auditor IV $59,508 - $76,171 5 years Fiscal Manager I $68,288 - $90,206 5 years with 1 year supervisory Fiscal Manager II $73,364 - $96,912 5 years with 2 years supervisory

In April 2010, the County of Santa Barbara advertised in the Ventura County Star for an Audit Supervisor with a rate range of $84,701 - $103,402, requiring a Certified Internal Auditor certificate, three years experience, and one year lead work. [Ref-05]

HR compares salaries with seven Southern California counties (excluding Los Angeles County) for determining competitiveness for County salaries. HR has conducted no comparable survey of private- sector internal auditor salary and experience ranges.

In its report Controls Self-Assessment Restart, the 2007-2008 Ventura County Grand Jury recommended: “R-02 The A-C and the Human Resources Director should jointly conduct a comprehensive study to resolve staffing, recruiting and retention issues in the Audit Department, and provide the results and recommendations to the CEO and the BOS.” The CEO and the BOS combined response stated: ___________________________________________________________________________ Illusion of Safety: Control Self-Assessment 3 “Do not concur: the Auditor-Controller’s Office has six budgeted internal auditor positions . . . The recruitment opened 7/28/08. As of 9/02/08 they have 84 applicants . . . It is our office’s opinion that the current salary and classification status of the internal auditor series is appropriate . . . If they [Auditor-Controller] are unsuccessful in filling their vacancies at the end of this recruitment, we will work with the A-C to determine what other alternatives should be considered.” [Ref-05]

Since the time of the CEO and BOS response to the 2007-2008 Grand Jury Report, the A-C and HR have not undertaken a study to resolve staffing, recruiting, and retention issues in the IAD. Control Self-Assessment Status Reports

In 2009, the CEO added “Internal Control Policy” (Chapter VII(A) – 13) for inclusion into the Administrative Policy Manual, which policy provides: “The Auditor-Controller is also responsible for periodically reporting to the CEO and Board of Supervisors on the status of internal controls based on the results of department annual self-assessments.” (Att-01)

The A-C sent a detailed report on the 2007 CSA Program directly to the BOS on January 29, 2008.

Beginning April 1, 2008, the A-C IAD has been sending a summary report on the status of the CSA Program quarterly to the CEO within 10 days of the end of the quarter. Thereafter, with noted exceptions, the CEO has provided a quarterly CSA report to the BOS, typically 14 weeks following the end of the quarter. [Ref-06] Fiscal Quarter Date of A-C IAD Date of CEO BOS Action Ended Report to CEO Report to BOS Mar 31, 2008 Apr 1, 2008 (none) Jun 30, 2008 Jul 7, 2008 Sep 16, 2008 Receive and File Sep 30, 2008 Oct 7, 2008 (none) Dec 31, 2008 Jan 15, 2009 Mar 10, 2009 Receive and File Mar 31, 2009 Apr 6, 2009 (none) Jun 30, 2009 Jul 9, 2009 Oct 20, 2009 Receive and File Sep 30, 2009 Oct 6, 2009 Jan 12, 2010 Receive and File Dec 31, 2009 Jan 4, 2010 Feb 23, 2010 Receive and File Mar 31, 2010 Apr 5, 2010 May 11, 2010 Receive and File Control Self-Assessment Process

The County Internal Control Policy (Chapter VII(A) - 13) adopted in 2009 broadly outlines the scope of departmental internal control systems and control self-assessment. (Att-01) ________________________________________________________________ 4 Illusion of Safety: Control Self-Assessment

The County is organized into 26 agencies and departments. Larger agencies and departments are subdivided into “Budget Units” for CSA Program purposes; there are a total of 52 CSA budget units for the County. Each such budget unit is responsible for performing a self- assessment of internal controls within the organization. The A-C’s IAD issues instructions and guidance to the agencies and departments in performance of their own self-assessments.

The budget units perform the self-assessments, providing copies to the IAD. The IAD selects self-assessments for a complete agency or department for validation by its internal auditors.

The County has established a three-fiscal-year cycle (triennium) for performance of control self-assessments and validations for all budget units. The first CSA triennium went from July 2004 through June 2007; the current triennium ends June 30, 2010. Control Self-Assessment Performance

Of the 26 budget units, 18 had submitted CSAs by the end of 2007. By March 2010, all budget unit CSAs for the 2007–2010 triennium had been performed (except for two CSAs in one agency which were expected to be completed by June 30, 2010).

IAD began validation of CSAs in 2008. As of the date of this report, IAD has completed validations of four agency or department CSAs, and one more is in process. These five validation reports issued in 2009 and 2010 covered self-assessments initially performed in 2007.

Some budget units report having used the CSA process to test and improve organizational control mechanisms. In many other cases, however, the validation process revealed the CSAs to have been incomplete or superficial.

Budget units report concerns regarding the CSA process: • unclear tasks and objectives • having to “train and re-train” new auditors • CSA effort is unbudgeted within the departments, unplanned, and often costly • while budget units are encouraged to tailor the CSA guides to their own organizations, there is little practical direction from IAD in doing so • validations may be performed on comparatively old information, requiring re-doing a self-assessment in order to complete the validation • CSA is uncoordinated with the County’s Lean Six Sigma efficiency program • CSA validations re-visit areas subject to regularly scheduled fiscal audits, such as cash control, receipts, and disbursements ___________________________________________________________________________ Illusion of Safety: Control Self-Assessment 5 • it is difficult to discern a difference between a CSA validation and an audit

CSA validations and reports are conducted at the agency or department level without active participation or review by the office of the CEO.

Through the office of the CEO and senior management, the County sponsors an operational efficiency program entitled Lean Six Sigma1. In this program, all agencies and departments continuously evaluate their activities at all levels for efficiency, effectiveness, and necessity. Lean Six Sigma does not include an equivalent emphasis on the factors of control and risk.

County management depends on the CSA Program to guide agency and department evaluations of risk and control, but they do not see the need to oversee the CSA directly.

The CSA Program and the Lean Six Sigma program are conducted concurrently within County administration, but without reference to one another; one is directed by the A-C, the other, by the CEO.

As of March 2010, modest changes were contemplated by the A-C IAD in the structure or methodology of the CSA for the new 2010-2013 triennium. No requests or suggestions for changes or improvements in the CSA Program had been received from the CEO’s office.

Absent changes to the CSA Program, all budget units will be tasked with completing new control self-assessments for the new 2010-2013 triennium.

In the FY 2009-10 Audit Plan, the A-C allocated 960 direct auditor hours to CSA and validations, out of the planned 9,000 available hours. As of the end of March 2010, IAD had expended 293 direct auditor hours on CSA efforts in FY 2009-10. Conclusions C-01. The A-C audit function has suffered from high turnover and a shortage of staff for several years. (F-01) C-02. The IAD audit staff continues to lack overall audit experience and experience within the County. (F-02 through F-04) C-03. The A-C has established no special training plan or an extra effort for training inexperienced auditors beyond the A-C’s standard training budget. There appears to be no training plan directed specifically toward junior personnel, other than close supervision. (F-04, F-05) C-04. HR is losing the direct and less-structured insight of a face-to-face interview by conducting exit interviews by mail. HR has lost input from 1 Lean Six Sigma is the name of a County initiative to focus on continuous improvements in processes and practices. Lean is a management approach to maximize value while removing wasteful activities and practices. Six Sigma can be defined as a management approach that seeks to systematically apply scientific principles to reduce variation and eliminate defects in service offerings. http://www.ventura.org/rma/service_excell/ci/pse.html ________________________________________________________________ 6 Illusion of Safety: Control Self-Assessment an indeterminate number of former employees, all of which could provide useful insight into high-turnover situations. (F-06 through F-08) C-05. Recruitment and retention of audit staff is suffering because HR and A-C have not updated auditor salary schedules to be competitive with those of other Southern California counties. (F-02 through F-09, F-11) C-06. HR has not conducted a comparison of the County auditor salary structure against that of private firms, which could be likely sources of significant competition (or of recruitment) for capable and experienced internal auditors. (F-11 through F-13) C-07. The County’s Administrative Manual Procedure VII(A) - 13, Internal Control Policy, requires that the CEO and A-C make periodic CSA reports to the BOS, but the A-C’s reports are sent only to the CEO. The CEO’s reports incorporate the substance of the A-C’s report, but the serial process invites CEO filtering, diminishes the perception of auditor independence, and may reduce BOS visibility. (F-14 through F-16) C-08. That the CEO’s office takes eight to 14 weeks to prepare the quarterly CSA report to the BOS reflects a low priority or importance attached to the CSA by County management. (F-16) C-09. All County agencies and departments have participated in the CSA program this triennium to the extent of having performed self- assessments, and have complied with the IAD guidance. (F-17 through F-21) C-10. CSA validations have been scheduled and conducted two or more years after the date of a department self-assessment. The delayed validation can detract from the perceived importance of the program. (F-22) C-11. With little feedback from IAD or County management upon completion of the CSA, individual departments may have filled out the questionnaire forms on schedule, but find out only during a much later validation, if ever, that the self-assessment may have been incomplete or superficial. (F-22, F-23) C-12. The validation process has proven to be laborious and duplicative. With the high turnover in auditors, there is a lack of auditor continuity from performance of the CSA to validation; additional time is also required on the part of the audited organization and the audit staff. (F-24) C-13. With only five of 26 agencies or departments having completed the validation phase during the current triennium, the other 21 agencies or departments will be expected to again perform self-assessments for the 2010-2013 triennium, without necessarily having had independent, knowledgeable evaluation of their prior efforts. (F-24, F-25) C-14. The A-C’s IAD is presently understaffed and insufficiently experienced to lead effective agency/department CSAs and to perform validations in an efficient manner. (F-24) C-15. The CSA Program is not being executed timely or effectively. (F-24, F-25) ___________________________________________________________________________ Illusion of Safety: Control Self-Assessment 7 C-16. Agency and department managers view CSA as a necessary nuisance rather than as a potential tool for assessing and improving organizational management. (F-23, F-24) C-17. The Office of the CEO views the CSA Program with indifference, as an A-C program which may offer insights to the agency and department managers, but which does not require executive oversight or participation. (F-24, F-25) C-18. The A-C is responsible for oversight and direction of the CSA Program with insufficient resources, but the responsibility for performance and execution belongs to County agencies and departments for which the CSA Program is a low priority. Consequently, the CSA Program lacks thoroughness and urgency. As currently conducted by the County, the CSA Program provides the illusion of safety without the substance. (F-23 through F-28) C-19. The County is performing the CSA Program as an afterthought. There is no apparent effort to coordinate or integrate CSA with the County’s Lean Six Sigma management improvement program. (F-26 through F- 28) C-20. Continuation of the CSA into a third triennium with no substantial changes in process, and with no enhancement of supporting audit resources, would be ill-advised. (F-29, F-30) C-21. In the first three quarters of the FY 2009-10, the IAD had expended only 32% of its planned CSA hours for the year, for which understaffing appears to be the major contributor. (F-31) Recommendations