Good Job Overall; Disaster California Recovery Must be Addressed*

Based on the Grand Jury's survey of the Chief Information Officers of California counties with populations of one to three million, current costs of Information Technology in Orange County, per resident, appear to be comparable or lower than IT costs in California counties of similar population size. CEO/IT Response: Agrees with the finding. Results from the Grand Jury's survey of California Counties comparable to Orange County's population show that IT cost per resident for Orange County is approximately $49. The IT cost per resident for Riverside County is $90, Sacramento County is $63, and Santa Clara County is $41. In addition, 2.7% of Orange County's operating budget is for IT expenditures. This is compared to Riverside County at 4.0%, Sacramento County at 2.6%, and Santa Clara County at 1.8%.

IT project cost overruns do not plague the County. However, policies and procedures are not in place in the IT governance structure to adequately measure and evaluate achievement of benefits and goals of IT projects over their entire project life cycle. CEO/IT Response: Agrees with the finding. The County's current IT Governance model requires that for every IT project ≥ $150,000, a business case be developed and reviewed by the IT Investment Review Board. The Response to OC Grand Jury Report "OC Info. Technology Management: Good Job Overall; Disaster Recovery Must Be Addressed" standard IT business case template includes a section whereby the requester identifies "Project Performance Measures" and is asked to describe how these benefits/business outcomes will be measured. A 5-year cost plan is also submitted with the Business Case. For the projects that are approved and budgeted, CEO/IT reports on the status of these quarterly to the Board of Supervisors. During FY 14-15 CEO/IT will develop a method to engage the project sponsors to determine if the stated benefits and goals of the projects reported and completed are realized. This method will be presented to the IT Executive Council for approval and implementation. Recovery of IT resources and services will be critical to the functioning of vital County

services in the event of a catastrophic disaster event. Recent Disaster Recovery (DR) exercises for the CEO/IT datacenter have not been completed successfully. Thus, the costs, time, and possibly the ability to recover some or all datacenter operations after a catastrophic disaster event has not been determined or demonstrated. CEO/IT Response: Agrees with the finding. The County agrees that DR exercises where the CEO/IT infrastructure, the Agency/Department applications, and actual business processes are tested end-to-end have not occurred. CEO/IT maintains and routinely tests the DR warm site (currently Solano County) infrastructure. Application and business user transaction testing, which is the responsibility of Agencies/Departments, has not been routinely scheduled. It is also not the case, as was stated in the Grand Jury report, that "[m]ost large Agencies in Orange County use this service [the DR warm site] for some or all of their applications." Currently, only four County Agencies/Departments (Auditor-Controller, Probation, Registrar of Voters, and Social Services Agency) have elected to use the DR warm site for the recovery of critical applications. Twenty-six Agencies/Departments depend on the DR warm site for the recovery of email functionality. CEO/IT supports the recovery of six enterprise-level line of business applications via the DR warm site. In FY 14-15, CEO/IT expects to increase its efforts to secure Agency/Department participation in DR warm site testing. CEO/IT also intends to work with Agencies/Departments to establish recovery capabilities for additional critical applications. Agency/Department IT divisions need to be aware of the continuity requirements identified by their business users and to ensure that they have adequately prepared the technical environment to enable those requirements. Those same divisions also need to be Response to OC Grand Jury Report "OC Info. Technology Management: Good Job Overall; Disaster Recovery Must Be Addressed" aware of their roles and responsibilities as they pertain to DR testing. CEO/IT will work with Agencies/Department IT divisions in the coming fiscal year to build this awareness. IT best practices indicate that user satisfaction should be a key measurement of IT

services. The current contracts for managed services both specify that the County shall "conduct satisfaction surveys semi-annually...or more frequently." However, there are no consistent countywide policies, guidelines, or procedures for user satisfaction surveys of all IT services, including those provided by agencies/departments, and surveys are not taken or published on a regular basis. CEO/IT Response: Agrees with the finding. The new IT Managed Service contracts for Countywide IT services require that the vendors perform, at a minimum, semi-annual user satisfaction surveys. CEO/IT will work with Agencies during FY 14-15 to develop a policy and procedure to implement IT user satisfaction surveys for Agency and CEO/IT provided services based on standard service level requirements. This process will include the ability to gather, analyze, and evaluate the results of the surveys for reporting purposes. Improvement areas will be identified and addressed. The CEO/IT's project management methodology that is accessible through the County

Intranet site describes a traditional "Waterfall" approach to system development. Although still being used in the industry, Waterfall is a somewhat dated approach. More current Agile system development methodologies have proven very successful in several County agencies and are recommended in the IT industry as best practices. Use of the Waterfall approach may have been a factor in the failure of the PTMS development project; however, there are many factors that can contribute to the success or failure of system development projects, and the Grand Jury renders no opinion as to the fault or liability relative to any litigation. CEO/IT Response: Agrees with finding, in part. Due to pending litigation, CEO/IT cannot comment with respect to the potential impact of the waterfall methodology on the PTMS project. CEO/IT does agree, however, that the Agile software development methodology can be very helpful in developing software applications that meet user requirements. The traditional waterfall application development model is a sequential model that follows standard phases, including requirements definition, software architecture and design, writing all of the code, testing, and so on. The schedule is built to support this sequential process. It assumes that every (or most) requirement of the software can be identified before any design or coding occurs. Response to OC Grand Jury Report "OC Info. Technology Management: Good Job Overall; Disaster Recovery Must Be Addressed" The Agile development methodology is "iterative", every phase of development - requirements, design, etc., is continually revisited in smaller periods of time. When a team stops and re-evaluates the direction of a project every two weeks, there is time to steer it in another direction. Stakeholders have recurring opportunities to assess what is being done and provide input. Users can also see what is being designed and developed much sooner, improving communication and helping alleviate the risk of developers misinterpreting requirements or users missing key requirements. Leading teams to support the Agile life cycle requires a modification of the standard, sequential project management life cycle that the County uses today. During FY 14-15, CEO/IT will work with Agencies and staff that have implemented the Agile software development methodology to incorporate best practices into the standard project management methodology. An Agile methodology training plan will be developed for software development staff and those responsible for managing software development projects so that this methodology is adopted Countywide. Under the new managed services contracts, costs for these outsourced services will

be very predictable over the life of the contracts, as long as the County can predict, manage, and control volumes (data, transactions, service calls, etc.) over that period. CEO/IT Response: Agrees with the finding. In May 2013 the Board of Supervisors awarded a contract to SAIC to provide data center, service desk, desktop support and application services. In September 2013 the Board approved a contract with Xerox to provide end-to-end support of the County's voice and data network systems and to transform the County's existing architecture to a converged voice and data network. Both of these managed services contracts, (contracts based on stated service level requirements that vendors are required to meet), contain pre- negotiated pricing for various services over the life of the five-year contracts, plus two additional one-year periods. In general, the contracts have been constructed so that as the County consumes additional services (resource units), unit costs decrease. Costs related to these managed services contracts include one-time Transition and Transformation Fees, recurring monthly Management Fees, and recurring monthly Service Fees that are dependent upon service unit consumptions. In summary, costs to the County are predictable based on the County's rate of service consumption. Under the new outsourced contracts, consolidation and centralization of some IT

services will result in overall cost savings. It will also standardize the delivery of many services, allowing for more consistent governance and alignment with County strategies and IT guidelines. However, agencies/departments are concerned that additional centralization will result in higher costs and reduced levels of service Response to OC Grand Jury Report "OC Info. Technology Management: Good Job Overall; Disaster Recovery Must Be Addressed" for them. Centralization of some services may make more business sense than others, and this may vary by function and by business unit. CEO/IT Response: Agrees with the finding. The new outsourced managed services contracts have been constructed so that as the County consumes additional services through resource units, unit costs decrease. For example, under the managed services contract with SAIC, as the County brings on additional Agencies/Departments for desktop support, service desk or data storage services, the unit price for these services decreases as volume increases, resulting in overall cost savings to the County. These cost savings do not necessarily mean that the County is receiving cheaper services. Since the transition to SAIC as the new provider for Data Center Services in February 2014, CEO/IT has been onboarding additional Agencies for desktop support, service desk, data storage and application support. As more and more Agencies/Departments utilize services offered through these managed services contracts, the County will achieve standardized delivery of services. In addition, the managed services contracts are based on guaranteed service levels that the vendors are required to meet; failure to do so would result in pre-defined penalties. In general, as additional Agencies join enterprise IT service offerings, service volume increases, resulting in decrease in unit costs for all participants. The County will need to analyze centralization of IT services closely, by the type of services and business requirements, to see where it is most cost-effective and makes business sense. At the direction of the County Executive Officer, the County has established an IT Centralization Working Group that is being charged with completing a centralization feasibility study. (See response to Recommendation #8 for additional details). County Response to Grand Jury Recommendations CEO/IT should enhance the current format and guidelines for post implementation