(endorsed) March 10, 2022 The Honorable Theodore C. Zayner MAR 1 5 2022 Presiding Judge Clerk of the Court Santa Clara*

R1

When undertaking a new project, the County should add a requirement for a high-level feasibility analysis during the initial planning phase. In most cases, the analysis should include a survey of how other counties and organizations, with an emphasis on counties and organizations of a similar size solved the problem. If no organization has used the approach being considered, explicit risk mitigation steps should be added to the project plan, and schedules and budgets should be adjusted accordingly. Resources such as CalEQRO and other state or independent auditors should be fully utilized to assess these risks. The County should develop a plan to establish these practices by June 30, 2022. Response: This recommendation has been implemented. New and improved processes as well as organizational structures have been put into place by TSS, and all projects that have started since January 2022 have been following the new process. All TSS projects go through a Conceptual/Requirements Review and a Design Review performed by the Technical Architecture Team. The objectives of the review are to ensure the following: • Feasibility of design considering technological, financial, and organizational constraints

R2a

The County should develop a plan to require anyone serving in a project management role to have sufficient project management experience and/or certification for the nature of the project. The County should develop this plan by June 30, 2022. Response: This recommendation is already in place. TSS has always required that Project Managers meet the requirements outlined in the IT Project Manager job specification, which specifically requires a relevant Bachelor's degree and "Five (5) years of IT project management experience" or training and experience equivalent to the possession of a Bachelor's degree and "Seven (7) years of IT project management experience, including experience with Microsoft Office Suite of products." The IT Project Manager job specification can be found at: https://www.governmentjobs.com/careers/santaclara/classspecs/1292485

R2b

The County should develop a plan to require that each project have a single overall project manager who has relevant project management experience or certification. The plan should be established by June 30, 2022. Response: This recommendation is already in place. All TSS projects are staffed with a single project manager who possesses the necessary qualifications.

R2c

If a project requires additional project managers, then the County should require a clear organizational structure that facilitates coordination and decision-making. The County should develop a plan to establish these requirements by June 30, 2022. Response: This recommendation has been implemented. Netsmart was a complex program that involved a collection of multiple, interrelated projects each with its own assigned project manager. Going forward, for all such programs, TSS will assign an IT Program Manager (or equivalent) for overall program oversight. The program charter will include the team members and the organizational structure along with their roles and responsibilities to better facilitate coordination, decision-making and escalation paths. This is already in effect for new programs such as the Public Health Grant program, and the Netsmart Contract Provider Rollout with Netsmart.

R2d

The County should verify that project managers follow the best practices defined by the County's own Project Management Center of Excellence. The County should monitor all projects and verify that all required items listed in the Governance Gates Checklist are completed by the project manager and approved by the executive responsible for the project. The County should verify the completion of required items for existing projects and should develop a plan to monitor compliance with the County's own Project Management Center for Excellence criteria for current and future projects by June 30, 2022. Response: This recommendation will be implemented by June 30, 2022. The TSS Project Management Center of Excellence was established in August 2020. Since then, the County has put in place additional project management policies, processes, and tools. For all new projects, project managers are following the processes established by the Project Management Center of Excellence, and completing the required items listed in the Governance Gates Checklist

R3a

The County should develop a plan to require documenting user acceptance testing criteria at the start of a project. The testing should involve applicable clerical staff, with testing done in their work environment when possible. The schedule for this testing should account for staff availability. The County should develop this plan by June 30, 2022. Response: This recommendation has been implemented. A user acceptance testing plan was created and executed during the project execution phase for all new projects that have started since January 2022.

R3b

The County should develop a plan for situations where the user acceptance testing cannot be done in the users' work environment. This can, for example, call for releasing the system to a subset of a department's staff before releasing it to an entire department. The County should develop this plan by June 30, 2022. Response: This recommendation has been implemented. New projects will go through a Conceptual/Requirements Review and a Design Review performed by the Architecture Team. This includes the steps taken to address testing and organizational change management. The Architecture Team will provide advice on necessary testing to reduce risks. The Project Management Office, through the Transition Stage Gate, will ensure that prescribed tests have been conducted. Please see the response to Recommendation 1 above for more details.

R4a

To ensure that Procurement is prepared to negotiate with vendors, the County should develop a plan that requires an analysis of technical staff needed to complete and deploy the project. This plan should include escalation procedures to be followed when Procurement is not provided the information it needs. The County should develop this plan by June 30, 2022. Response: This recommendation will be implemented by June 30, 2022. The Procurement Director will prepare a negotiation plan in collaboration with project sponsors and other relevant departments. This will include analysis of technical staff needed to complete and deploy the project, where appropriate. Senior management of the County will be consulted if the Procurement Department is not initially provided the information it needs to prepare an effective and complete negotiation plan.

R4b

When the County intends to have County staff train their colleagues on how to use a new system, the County should first determine if staff is available to do the training. If staffing is not available, either additional consultant support must be included in the agreement negotiated with the vendor or the project schedule should be revised. The County should develop a plan for this scenario by June 30, 2022. Response: This recommendation will be implemented

R4c

The County should investigate the feasibility of negotiating vendor contracts that include a bonus for on-time and successful completion of all parts of an IT project, even if the vendor is not responsible for them. This would incentivize outside vendors to provide TSS project management with complete information about what is needed from all parties to bring projects to successful conclusions. The County should report on its investigation of this approach by June 30, 2022. Response: The County will investigate the feasibility of negotiating vendor contracts that include a bonus for on-time and successful completion of all parts of an IT project.

R5a

The County should conduct a risk assessment to identify threats to the objectives of the ongoing project. Potential responses to those risks should be captured in a risk management plan for this project. The County should develop a plan that evaluates the future risks of this project by June 30, 2022. Response: This recommendation has been implemented. For all new TSS projects that have started since January 2022, project managers are following the processes established by the Project Management Center of Excellence, and this includes a risk management plan. For the Netsmart rollout to the contract providers, the County is engaging an external management consultant to do a risk assessment looking at staffing, schedule, rollout plan, training, etc.

R5b

The County should develop a plan to ensure that contract providers are informed as decisions are made about what work needs to be done by the contract providers, what work will be done by the County, who will pay for the work, and when the work will be scheduled. The County should develop the plan by June 30, 2022. Response: This recommendation has been implemented. TSS met with the Business Department to develop a communication plan that addresses these concerns. The communication plan includes a RACI Chart (Responsible, Accountable, Consulted, Informed), to define roles and responsibilities, along with timelines specifying who is responsible for each task and when. The Project Manager is the single point of contact responsible for developing and managing this communications plan as well as sharing and communicating it to all relevant stakeholders. . .