Social Services Committee Independent Management Audit During the course of its investigations, the 2001/2002 El Dorado

1 The Department's protocols for social workers to determine if an immediate investigation of a report of child abuse or neglect is needed are inadequate and allow for too much judgement by intake social workers. Though limited, the protocols provide at least some assurance of social worker consistency in determining appropriate responses to referrals. But in a sample of 58 case files reviewed for this audit, documentation of the required decision-making protocols was not found in approximately 50 percent of the cases.

1 The Department of Social Services Child Protective Services division regularly produces management reports that concentrate on caseload levels, assignments and referral characteristics rather than outcomes. The performance measure indicators tracked are inconsistent throughout the division.

1 The Child Protective Services division (CPS) is organized into four units, with one supervisor responsible for each unit. The four supervisors report to a Program Manager who reports to the Deputy Director and Director of the Department of Social Services. The four units vary in size, number of functions and spans of control. The largest unit, the Adoptions unit, has 11.1 full-time equivalent positions, including vacancies, while the smallest unit, South Lake Tahoe, has 5.1 full-time equivalent workers. The Adoptions unit supervisor oversees adoptions social workers in Placerville and South Lake Tahoe and all CPS clerical staff in Placerville. No other supervisor oversees staff from two different functions and in two different offices.

1 Communications with top management of the Department of Social Services is considered poor by many staff social workers and office assistants. Many staff members feel there is no forum to express their concerns and problems and to make suggestions for improvements. This is particularly true in the South Lake Tahoe office as visits and meetings with staff there by the CPS Program Manager and the Director and Deputy Director are infrequent. Regular forums with open communications between staff and managers should be employed as a mechanism for improving CPS processes and services and to improve staff morale.

The Department's protocols for social workers to determine if an immediate investigation of a report of child abuse or neglect is needed are inadequate and allow for too much judgement by intake social workers. Though limited, the protocols provide at least some assurance of social worker consistency in determining appropriate responses to referrals. But in a sample of 58 case files reviewed for this audit, documentation of the required decision-making protocols was not found in approximately 50 percent of the cases.

The Department does not have a formal risk assessment tool to assist investigation social workers in determining whether a child should be removed from home. The current tool and documentation used by Department social workers, called the Investigative Narrative, does not include a standardized scoring system or other methods to ensure consistent interpretation of similar situations. As with the intake protocols, the Investigative Narrative does not appear to even be used in all cases. In 8 out of 27 case files reviewed, or 29.6 percent, the Investigative Narrative documents were not completed. In another 11 cases, or 40.7 percent, the document was filled out incorrectly.

The After Hours Intake function violates Department policies and procedures by not gathering background information from the Child Welfare Services/Case Management System on children who are the subject of telephone reports of alleged abuse and neglect.

As a result of these problems, Department management does not have documented assurance that decisions made by social workers in the intake and investigative processes are consistent and properly supported. This problem is reinforced for the After Hours Intake function by the limited availability of supervisors for consultation.

Structured Decision-Making is a system used by some counties to minimize individual variation in determining the level of response to initial reports of child abuse and neglect and in determining whether or not a child should be removed from their homes. By implementing at least some components of this system in El Dorado County, the Department will have greater assurance of consistency in its treatment of abuse and neglect allegations. In addition, the Department should require supervisors to be available on call by telephone to social workers assigned to After Hours for consultation and direction as needed. State law mandates that all counties provide initial intake and evaluation of risk services to all children reported to the County as being endangered by abuse, neglect, or exploitation. Every county is to maintain and operate a 24-hour response system and provide immediate in-person 10 (cid:9)(cid:9)(cid:9)(cid:9)(cid:9)(cid:9) Section 2: Lack of Standardized Assessment Tools responses by a county social worker in emergency situations in accordance with regulations of the department.' In response to this requirement, the El Dorado County Department of Social Services has developed its Child Protective Services Protocol, Criteria and Process for Accepting CPS Cases for Assessment, and Emergency Response Protocol included in the Department's policies and procedures manual. This protocol includes a form called the Emergency Response Protocol (Form EL 212) that is to be filled out by social workers for all initial calls alleging child abuse or neglect. When an initial call reporting suspected child abuse or neglect is received by the Department, the intake social worker has three response choices: • Conduct an investigation immediately; • Conduct an investigation within 10 days; or • Do not conduct an investigation or "Evaluate Out" the case. The Department uses a 3 day response to investigate allegations of abuse or neglect. However, a 3 day response time is not formal policy in the Department. To assist intake social workers in determining which of these responses is the most appropriate, the Department's emergency response protocols include a series of response guidelines to guide the initial intake social worker's decision. The form includes the following questions to assist social workers in determining what the initial departmental response should be. 1. Is there sufficient information to locate the family? 2. Is this an open service case with DSS and is the current intervention adequately addressing the problem described in this allegation? 3. Does the allegation meet one or more of the legal definitions of abuse? 4. Is the perpetrator a caretaker of the child or is there reason to believe that the caretaker was negligent in allowing or unable or unwilling to prevent the perpetrator having access to the child? 5. Are specific acts and/or behavioral indicators of abuse, neglect, or exploitation included in the allegations? 6. Does additional information obtained from collateral contacts or record material invalidate the report? ' California Welfare and Institutions Code (WIC) § 16054 Harvey M. RoseAccountancy Corporation (cid:9) Section 2: Lack of Standardized Assessment Tools 7. Does this report represent one in a series of previously investigated, unsubstantiated, or unfounded reports from the same party in which no new allegations or risk factors are revealed? The answers to these questions are intended to guide CPS social workers in determining which referrals justify an in-person investigation. If an in-person investigation is not necessary, Department policies and procedures state the intake social worker should provide a more detailed rationale regarding their decision why an in-person investigation should not be conducted. This Emergency Response Protocol form is included in the Department's CWS/CMS computer system so that social workers receiving an initial call reporting child abuse or neglect can start a case file and enter all information required into the computer system as the call is received. According to Department procedures, this electronic form is supposed to be transferred to the emergency response supervisor for approval for all cases that are evaluated out. While the questions asked in the Emergency Response Protocol form seem appropriate for conducting an initial investigation, the problem with the protocol is that it is fairly open-ended and allows for significant individual interpretation of facts and circumstances. It relies primarily on interpretation of the facts and situation by the social worker. Social workers are trained to assess such situations and professional judgement is always required in children's welfare cases. However, trained social workers are still subject to personal biases and preferences and two social workers can interpret the same situation very differently. While the nature of the work is such that some judgement will always be required, Department management should make every attempt to minimize personal biases and variations in staff decision-making. The room for individual interpretation becomes more pronounced in cases where the situation is not obviously dire but may be on the border between a 10 day investigation or "evaluating out" the case. For such cases, the Department's policy is as follows: "Criteria are to be liberally interpreted, which means where circumstances are marginal, we should open a case for investigation. It is preferable to err in favor of ensuring the child(ren)'s safety and the appropriate response time should be considered." Thus, the explicit goal of the procedure, for safety purposes, is to conduct more investigations than potentially necessary. Emergency response protocols used by some counties provide more structured guides that link certain responses with certain outcomes. For example, a history of two or more previous referrals may lead to a guideline to conduct an immediate investigation unless the intake social worker can provide information that proves this would not be necessary. The tools available to assist social workers in making decisions of whether to investigate allegations of abuse and neglect or the evaluation of risk vary across the state. A study by the University of California at Berkeley found: 12 Section 2: Lack of Standardized Asses sment Tools "Departments of Social Services are increasingly being challenged to determine which cases are reported to them are at the highest risk and most in need of services. One response to this challenge involves the development of screening procedures that distinguish levels of risk and need among cases that come to the attention of Child Welfare Services ... The employment of effective screening procedures ...can help not only to reduce disruptive legal intervention into families in situations when it is unwarranted, but also to insure procedural fairness - one-element of which involves consistency in the treatment received by similar cases. The systematic use of screening guidelines would help to promote consistency among decisions made by individual workers and among counties; it would also aid new workers in the field and offer workers ,2 and the state some degree of protection in an era of increased litigation. Some counties have chosen more structured guidelines such as the Structure Decision-Making (SDM) model, developed by the Children's Research Center of the National Council on Crime and Delinquency, a non-profit organization. One of the components of SDM, the Response Priority Assessment, is a series of decision trees that guides the intake social worker on what the response should be for the various types of allegations ranging from physical abuse to general neglect. A system such as this would provide for a more consistent approach to determining the appropriate responses to initial reports of abuse and neglect in El Dorado County and would provide better documentation justifying the decision reached by the intake social worker. Emergency Response Protocol not being used in many cases To determine the Department's compliance with its Emergency Response protocol, 58 randomly selected cases were examined in CWS/CMS to verify that proper documentation existed in each case. Even though this protocol has limitations, it does provide some documentation of the decisions made and is required for every case by Department policy. Cases were selected largely from 2001 and included referrals which resulted in immediate investigations, investigations within 10 days, and cases that were evaluated out. It should be noted that the sample was randomly selected but was not designed to be statistically significant. A more authoritative examination would require significantly more time than was authorized for this project. As shown in Exhibit 2.1, out of the sample of 58 cases, Emergency Response protocol forms were only fully completed 50 percent of the time. Thus, although the policy manual clearly states that the form should be used as a guide to making initial intake decisions, half the time the information needed to do so was missing. Gilbert, Neil, Karski, Ruth, and Frame, Laura. The Emergency Response System: Screening and Assessment of Child Abuse Reports. School of Social Work, University of California Berkeley, 1997, pp.l-2. Harvey M. RoseAccountancy Corporation 13 (cid:9) Section 2: Lack of Standardized Assessment Tools Exhibit 2.1 Number of Completed Emergency Response Protocol Forms in Case Files Number of Forms Percent Complete 29 50% Incomplete 29 50% Total 58 100% Source: CWS/CMS As stated above, Department policies and procedures call for the Emergency Response Protocol form to be reviewed on CWS/CMS and approved by the Emergency Response supervisor for all cases which are evaluated out. The Emergency Response unit does an excellent job of reviewing the referrals that are evaluated out. Based on our review, supervisor approval was documented in every case where the decision was to evaluate out. As shown above, the response protocol forms are not complete 50 percent of the time. However, examination of the 25 cases where the decision was to evaluate out show that only two of those 25 referrals, or 8 percent, had incomplete Emergency Response protocol forms and were approved by the supervisor. Thus, in instances where supervisor approval is not required, social workers are more likely to not properly complete the Emergency Response protocol in CWS/CMS. In 47 of the 58 cases reviewed, the response proposed by the intake social worker was approved by the supervisor, even when a completed Emergency Response Protocol form was not entered in to CWS/CMS. Thus, our review suggests that the required Emergency Response Protocol form is not used to guide all intake decisions, in contradiction of Department policies and procedures. Additionally, it appears that this form is viewed by some social workers as a form that has to be filled out as an after thought and not as integral part of the decision-making process. . Management controls are needed to ensure that all workers are making appropriate decisions and documenting them consistent with Department policies. A regular process of reporting social worker compliance with department policies is needed as is periodic review of randomly selected case files by Department management to ensure that decisions are appropriate, properly justified and documented and in compliance with Department policies and procedures. Supervisors are reviewing a majority but not all decisions by the intake social worker. Based on the analysis above, we recommend the Department implement the Response Priority Assessment component of the Structured Decision-Making system. This assessment should be completed on every referral placed with CPS. Use of this tool would ensure greater consistency in social worker decisions about which cases to investigate and when. The Response Priority Assessment component of the Structured Decision-Making system provides social workers with a series of decision trees on which to base the initial response decision. This system will ensure that social workers systematically apply similar criteria to every case and provide consistency across social workers at the two DSS offices in El Dorado County. The 14 (cid:9)(cid:9)(cid:9)(cid:9)(cid:9)(cid:9) Section 2: Lack of Standardized Assessment Tools SDM system provides a process to support, guide, and fully document assessments. One significant advantage of this system is that it brings accountability to decision-making and, based on the decision tree system, decisions require an explanation. Moreover, because the decisions are based on the decision tree system to guide the worker, consistency should increase. The Emergency Response Guidelines lacks guidance to guarantee consistency. Lack of a Formal Safety and Risk Assessment If the result of the intake worker's assessment is that an investigation should be conducted, the case is transferred to an investigative social worker. Under Welfare and Institutions Code Sections 309 and 16504, social workers must determine whether the children that are the subject of the allegations are in immediate danger of physical or sexual abuse, and whether the children should be removed or can remain safely in their homes. The social worker also determines whether there are any services that can be provided that would allow the children to. safely remain in their homes. To document the basis for this decision, investigation social workers are required to complete an Investigative Narrative, a one-page form with ten fields, to document any risk factors contributing to the social worker's decision whether or not the child should be taken into protective custody. Currently, the Department does not have precise procedures requiring the use of the Investigative Narrative. CPS management reports that the Investigative Narrative serves as the Department's risk assessment tool. While the State of California requires only a written narrative, the Department has added ten fields to further explain the investigation. The ten fields in the Investigative Narrative that should be complete are: Brief Narrative of the Investigation (includes details of who, what, when and where) • Child's age, vulnerability, physical and/or mental abilities (includes perpetrator's access to child) Child's behavior • Caretaker's Parenting skills/Knowledge (includes capacity for childcare; interaction with children, other caretakers; skill, knowledge; criminal behavior, mental health) • Strength / Weaknesses of Family support system (includes relationships, presence of parent substitute) • Caretaker's Substance / Alcohol Misuse • Environmental Condition of Home • Any services offered and result (includes directives/referrals given and family's response) 15 Section 2: Lack of Standardized Assessment Tools Need for Emergency Services, if any One problem with the Investigative Narrative risk assessment tool used by the Department to document investigations is that it does not offer a structured approach to guide the investigation social worker's assessment of risk into a decision. Instead, the form is primarily used by the Department's social workers to provide a description of their investigations. As such the documentation of why a child should be taken into protective custody is not as thorough or objective as it would be if a standardized risk assessment were in place. Without a formal safety and risk assessment the criteria social workers use to make removal decisions can vary significantly. Based on a questionnaire provided to social workers in CPS as part of this audit, 58 percent of social worker respondents disagreed with the statement that all social workers use the same criteria in deciding to remove children from their homes. Compounding the inadequacy of the Investigative Narrative as a risk assessment tool is that social workers are not properly using it. As shown in Exhibit 2.3, in 19 of the 27 cases reviewed where an in-person investigation took place, or 70.4 percent of all cases, the form is not properly completed. The definition of not properly completed is that the Investigative Narrative only contains a paragraph or two providing a description of the investigation rather than completion of all ten fields of information required on the form. Often, the paragraph narrative was only a few sentences in the cases reviewed. The Investigative Narrative was missing entirely in eight of the referrals examined. In these cases, it is impossible to retrace the steps of the social worker to determine the basis for the decision. For the eight cases where the Investigative Narrative was incomplete, a list of individuals contacted for the investigation by the social worker was provided in CWS/CMS, but this failed to provide beneficial information to determine how the social worker reached a decision. Exhibit 2.3 Use of Investigative Narrative by CPS Social Workers Number of Percent Narratives Each field complete 8 29.6% Fields Incomplete 11 40.7% Narrative not found 8 29.6% Total 273 100% As shown in Exhibit 2.3, 11 of the 27 Investigative Narratives in CWS/CMS were improperly completed, by not having each of the ten fields documented. The quality of the narratives ranged from one sentence to a very detailed account of the investigation to one that simply referred to a list in CWS/CMS of individuals contacted for the investigation. As with cases that are evaluated 3The sample number is 27 here because this is the next step in the child removal process. If the intake social worker determines that the referral does not meet the criteria of an in-person investigation that case is "evaluated out" and no further action is required by the Department of Social Services. Section 2: Lack of Standardized Assessment Tools out after initial intake, prior to the Investigative Narrative being completed by the investigative social worker, the Emergency Response Supervisor must review and approve the document. Of the cases reviewed, only eleven of the Investigative Narratives, or 40.7 percent, were not properly filled out yet they were apparently approved by the supervisors anyway. However descriptive the narrative form may be, it is still an inadequate assessment of the safety and risk of the child. According to Department management, the Investigative Narrative is designed more to close an investigation, especially in cases where no petition will be filed, and is geared to move the case along. This creates the impression from management to line staff that the Investigative Narrative is a form required to be completed more as an after thought to move the case along, rather than a tool which assists social workers and is an integral part of the decision-making process. In one case reviewed, the Investigative Narrative included only a brief one paragraph review of the investigation. The decision was to open the case and offer Family Maintenance services. However, three days later the case was closed, and the case file indicates that the case was opened in error. While mistakes can occur in any system, the lack of precision in the Investigative Narrative means that mistakes like this will be more likely to occur. Use of a risk assessment tool with a structured assessment mechanism would significantly reduce the potential for opening a case in error or failing to open a case when the child should be removed from the home. Need for a more structured and consistent approach to case decisions Based on the case file review conducted for this audit, it is clear that formalized Safety and Risk assessments tools would assist the investigative social workers in their decision-making and ensure greater consistency in case decision-making. The Department should implement the Structured Decision-Making (SDM) Safety and Risk assessment components as a complement to CWS/CMS as a means to increase consistency of investigations. This system, or components of it, are currently in use in at least 15 other California counties. The Safety Assessment component of SDM is designed to be used by investigative social workers during the initial in-person investigation of abuse and neglect referrals to determine when a child should be taken into protective custody. The Safety Assessment form contains a simple checklist and a narrative to formalize the decision-making process in CPS. These assessments will ensure that CPS staff assesses all cases based on a standardized set of criteria. In some counties, full integration of the Structured Decision-Making tools and the CWS/CMS system are not fully realized. Until full integration, many counties have implemented a paper version of assessment tools to complement CWS/CMS. Santa Clara County uses a paper version of the Structured Decision-Making system. The risk assessment tool in Los Angeles County is a paper version to complement CWS/CMS. In both Counties, social workers manually complete the forms and keep the document in the hard case file. The Department of Social Services could implement a paper version of the Structured Decision Making tool and manually track the results of these assessment tools and maintain records in the hard case file, similar to Santa Clara County. 17 (cid:9)(cid:9) Section 2: Lack of Standardized Assessment Tools Inconsistency in After Hours Intake Section 16504 of the Welfare and Institutions Code (WIC) states the County must provide a 24- Hour intake hotline, where referrals can be made. CPS maintains After Hours hotlines and staff at both the Placerville and South Lake Tahoe offices. Currently the Department does not have dedicated staff to operate the After Hours intake at either office. Staffing for After Hours is made up of workers who either volunteer or are assigned on a non-voluntary rotating basis. According to the Memorandum of Understanding with the union, social workers who work After Hours are guaranteed a minimum of two hours pay, plus an additional $1.20 per hour on-call. Additionally, the Memorandum of Understanding states "On-Call duty" means that an employee is assigned to work outside their normal work week and must remain available to be contacted by telephone and be ready for immediate call-back. Thus, the social worker is not in the office, but is accessible by telephone and ready to respond should a situation arise. The After Hours process is as follows: • The Department has a contract answering service that receives calls to the After Hours unit. At the beginning of the month CPS will send the answering services a monthly schedule of the social workers scheduled to work the After Hours shifts; and • When a referral is received by the answering service, the service will put the caller on hold and will either page or call the on-call After Hours social worker. At that point the social worker is connected to the reporter to begin the initial intake process. The After Hours intake process varies significantly from the intake process during normal business hours when CPS is fully staffed. For instance, in Placerville during normal business hours, the intake social worker will make the determination whether an in-person investigation is required, but they will not generally conduct the investigation. However, the After Hours intake social worker will not only make the decision whether to conduct an in-person investigation, but will actually perform the in-person investigation if an immediate response is required. Thus, when a social worker is conducting an immediate in-person investigation, the After Hours intake center does not have a social worker available to handle a second referral. According to the Emergency Response supervisor, those instances are rare, but they do happen. When these instances do occur, the answering service will take a message and the social worker will then call the reporter to begin the intake process on the second referral. According to the Department policies and procedures manual, when a child is removed from the home and placed in protective custody by After Hours staff, the written documentation of the incident should go to the Emergency Response Supervisor by 8:00 a.m. the next workday. The potential problem with this is that, without a supervisor checking the social worker's decision, a child could potentially be placed in protective custody or left in the home for as long as several days erroneously. 18 (cid:9) Section 2: Lack of Standardized Assessment Tools Additionally, unlike daytime hours, After Hours intake social workers have no direct supervision. During normal business hours, the intake social workers are positioned near the ER supervisor and can have consultation with their supervisor and fellow employees. However, social workers on After Hours duty work alone and do not have consultation with their supervisor. The social workers can call the supervisor if needed for informal consultation, but there is no guarantee that the supervisor will be available. When the supervisor is on vacation or unable to answer the phone the social worker is left to make these decisions on their own. Furthermore, during After Hours duty, social workers have no immediate access to CWS/CMS since the call is received at home. Although social workers have access to the Department building and CWS/CMS during After Hours, there is a delay to obtaining all relevant information regarding the case. This poses potential significant problems and violation of the Department's policy regarding factors to consider in determining if an in-person response is needed: "The decision whether or not to make an in-person response for all other referrals . shall be based on an assessment which shall include collateral contacts, a review of previous referrals and other relevant information to the extent such information or measures are necessary to conduct an assessment." Furthermore, the Department's "Child Protective Services Protocol" states that in-person investigation decisions by the intake social worker shall include a review of the child's "history and disposition of prior referrals." However, since the After Hours social worker does not have immediate access to previous referrals on CWS/CMS, the decision is made on only a partial picture. Additionally, the use of the Emergency Response Guidelines form in CWS/CMS is impossible for social workers to consistently answer correctly, since they do not have access to the past history of the child. Without access to CWS/CMS or paper files at the office, social workers cannot answer two of the eight questions that require further analysis. These questions are as follows: • Is this an open service case with DSS and is the current intervention adequately addressing the problem described in this allegation? • Does this report represent one in a series of previously investigated, unsubstantiated, or unfounded reports from the same party in which no new allegations or risk factors are revealed? Thus, when an After Hours social worker is making the decision to initiate an in-person investigation or not, the decision is made lacking historical information and without the ability to answer two of the key questions included in the Response Guidelines. In the sample of cases reviewed for this audit, instances were found where After Hours social workers received calls in which access to CWS/CMS might have changed the decision regarding if and how soon an in-person investigation should be conducted. We found instances where an After Hours referral had 10 previous referrals throughout the State of California. Based on the 4Child Protective Services Policy Manual. El Dorado County Department of Social Services. 1992. pg. 9. Harvey M. RoseAccountancy Corporation 19 Section 2: Lack of Standardized Assessment Tools current system these referrals would not have been known immediately by the intake social worker. Additionally, we found several After Hours cases where children had three referrals each. In particular, the child had three referrals and the response guideline protocol was incomplete. Information such as this could be critical to the decision made by the intake social worker. During normal business hours, the worker would have immediate full access to this information on CWS/CMS. To address the lack of historical and other information available to intake social workers for After Hours cases, CPS should implement a new category of a three-day response to give social workers more flexibility in making the in-person investigation decision. Unless it is obvious that an immediate investigation is needed, After Hours cases could be placed in this category rather than being evaluated out, pending access to CWS/CMS the next business day. This increased flexibility will come at no additional cost to the County. Additionally, CPS supervisors should be required to be on-call After Hours to provide consultation as needed to social workers. Taking these steps will ensure that After Hours decisions are consistent with the approach used during normal business days. Conclusion The initial screening and investigation risk assessment tools required by the Department for social workers to use are not sufficiently structured to ensure consistency of decision-making by different workers and at both Department offices. Nor do the tools provide adequate documentation justifying decisions reached. Unfortunately, they are not used at all in some cases and are only partially completed in others. Even with their limitations, these required forms provide some documentation justifying the decisions reached by the social workers. Cases are being approved by supervisors without these required forms completed in apparent contradiction of Department policy. The Department of Social Services needs more management oversight of worker compliance with required procedures and spot checking of case files to ensure that case decisions are adequately documented and supported. To further improve the level of documentation and justification of decisions reached, the Department should implement use of Structured Decision- Making tools to determine what referrals receive further investigation, how soon a response is needed, and to help social workers conducting investigations to determine if there is further risk posed to the child and whether to remove the child or not. In only 50 percent of the cases reviewed were the current initial intake response forms complete. In a review of a sample of Investigative Narratives, over 70 percent were either incomplete or missing. Since social workers are not accurately completing the Department's existing screening and investigation tools, consistency of intake responses and the decisions by investigators within the Department cannot be tracked. There are substantial problems with the process, ranging from lack of CWS/CMS immediate access to check previous referrals to a lack of consultation from supervisors. 20 (cid:9)(cid:9)(cid:9)(cid:9)(cid:9)(cid:9) Section 2: Lack of Standardized Assessment Tools

The Department of Social Services Child Protective Services division regularly produces management reports that concentrate on caseload levels, assignments and referral characteristics rather than outcomes. The performance measure indicators tracked are inconsistent throughout the division.

Department management needs to establish outcome goals for the CPS division and related measures to better evaluate CPS outcomes and individual employee performance. Measures such as number of families reunified after receiving services, number of children in stable placements, and recidivism should be regularly measured in addition to caseload levels.

Employee performance evaluations appear to be conducted reasonably timely for employees who have passed their one year probationary period. For employees on probation the number of evaluations conducted is inconsistent and their use in determining permanent employment is unclear. Employees on probation during FY 2000-01 and FY 2001-02 received anywhere from none to four evaluations during that period. Of eight probationary employees during the last two years who received satisfactory reviews, four were terminated and four were retained for permanent employment.

By implementing an employee performance evaluation system that explicitly explains job expectations for the first year of employment, includes in-person documented performance evaluations every three months during the first year of employment, uses a standardized appraisal instrument, includes outcome based performance measures in the appraisal instrument, and provides timely annual post probationary evaluations, the Department will have greater assurance of consistency and accuracy in its performance evaluations. Organizations, whether public or private, need to establish a set of goals and create an ongoing system of measuring organizational and employee outcomes. Increased staff accountability, improved problem solving ability, and, ultimately, better results for children are goals that all child welfare agencies should strive for. Without an accurate measurement of outcomes, it is difficult to determine the impact of the resources allocated on services and whether or not the agency is effectively carrying out the goals and objectives of management. To achieve an accurate measurement of performance, the Department must set measurement goals, employee outcomes must be measured related to these goals, and data must be utilized to measure performance. Employee evaluations are the source to quantify the performance of employees. Section 3: Performance Measurement Both processes of setting goals and establishing outcome measures can be beneficial because they require management to establish priorities and to allocate resources and to establish systems and processes that will lead to the intended results. The Department of Social Services, according to its web site, cites its mission goal as "to help people in social or economic crisis increase their ability to become as self-sufficient as possible." However, the Child Protective Services division does not have its own mission statement, but generally the goal of the division is to provide assistance to children who are victims of abuse, neglect or exploitation. Based on research and interviews, the Department does not have a consistent system in place for measuring the effectiveness across units within CPS. However, the CPS does a good job of collecting and reporting a variety of statistics about caseload activity levels, but not always outcomes and Department goals. The statistical management reports produced concern the type of referrals and their dispositions, current caseload levels, type of out-of-home placements, closed and active, number of adoptions, and number of children freed for adoption. The information provides management a tool to review and assess caseload and staff productivity, but they do not measure outcomes or the achievement of Department goals, such as the number of families reunified, number of children in stable placements and others. Inconsistencies exist in the performance measures used in the Child Protective Services division. The variances not only exist between the two Department offices, but also within CPS units in the Placerville location. While there are many similarities, such as caseload numbers, between the units there was not a consistent guide for tracking outcomes throughout the division. The Emergency Response unit in Placerville, we found there are more formalized performance measurements in place. The supervisor in that unit has created a series of reports and systems that can track performance within the ER unit. The performance measurements in that unit are as follows: Referral Count by Start Date Referral Count by End Date Voluntary Status Report Case Plan Start Date for Cases Opened Between two Dates Open Referrals with First Investigation Date Referral Performance Statistics Number of All Evaluate Outs Number of All Referrals upon Closing Number of All Voluntary Family Maintenance Cases upon closing Monthly Caseload Activity Report Caseload Summary After Hours Intake Log Examinations of the Ongoing and Adoptions units show each unit within CPS has different standards which are examined to determine performance measurement. According to the Ongoing Unit supervisor, that unit reports social worker caseload numbers on a monthly basis. In addition, the supervisor scans cases and a determination is based on the case plan, as some 24 (cid:9)(cid:9)(cid:9)(cid:9) Section 3: Performance Measurement case types require more staff time than others do. In the Adoptions unit some performance outcomes are measured, such as the number of adoptions, but the outcomes are not linked to the case approach that preceded it in the other units. Caseload Tracking Another critical role of the supervisor is to monitor achievement of key casework activities and outcomes on a case-by-case basis. Monitoring activities and outcomes enables the supervisor to track client progress as well as the caseworker's completion of essential casework functions. In addition, supervisors monitor achievement of casework activities and outcomes across caseloads. Finally, by monitoring the unit to determine if it is achieving its program goals, supervisors can identify trends necessary for planning purposes as well as areas in need of corrective action. One problem with caseload tracking in CPS is there is not a system in place to determine caseload numbers on a historical basis. Supervisors informed audit staff that caseload numbers change daily and there is not a systematic procedure to produce reports, which offer a historical perspective on caseloads. Even with the Monthly Caseload Activity Report, we were unable to get a historical assessment of assignments across units and of both DSS offices in the County. Analysis of the number indicated dramatic fluctuations in caseload numbers. One consistent theme emerged from discussion with upper management of the CPS division. The theme is that the performance measurements are not outcome based. Some interviewed felt outcome based performance measurements would be beneficial to the Department Outcome Measurement Systems in other Jurisdictions Jurisdictions throughout California have or are in the process of instituting outcomes-based performance measures for their child welfare service departments. Los Angeles and Contra Costa counties are among the California counties that have or are establishing outcomes-based performance measurement systems. The Los Angeles County Department of Children and Family Services, in its Strategic Plan 2000, has identified accountability as one of its key values, defining it this way: Accountability involves the belief that efforts to achieve an outcome will be made, that these efforts will be directed at a goal and will result in achieving that goal, that achievement can be demonstrated, and that someone is held accountable for the results. Based on this value, Los Angeles County distinguishes five broad outcome areas for all children in the county: • Safety and survival • Good health • Social and emotional well being Strategic Plan 2000, 2000; LosAngeles County Department of Children and Family Services 25 SPECIAL DISTRICTS (cid:9)(cid:9)(cid:9)(cid:9)(cid:9)(cid:9) Section .3: Performance Measurement • Economic well being • School achievement and work force readiness Of the broad goals listed above, each has its own specific outcomes tailored to the children in the Los Angeles County system, which includes performance indicators and data sources for each specific outcome. The data sources established in Los Angeles to track each performance indicator is generated by CWS/CMS. For instance, for social and emotional well being the desired outcome goal is to reduce the time a child spends in placement. To measure this outcome Los Angeles County used CWS/CMS data to determine the average amount of time a child spends in foster care, stratified by placement type. Data for this measure is available in CWS/CMS Contra Costa County's child welfare system established broad outcomes, strategies, performance measures, and data sources, similar to the structure of Los Angeles County's system. The outcomes are as follows: • Children's health and developmental needs are being met • Families are satisfied • Children are safe and remain in their own homes whenever possible • Children achieve permanency in a timely manner El Dorado County could replicate the outcomes and models in both Los Angeles and Contra Costa Counties. To establish an outcome based performance measurement system usually requires an investment of staff time and possibly funding for new or enhanced information systems. The Department of Social Services Child Protective Services division can create outcome based performance measures from data stored in CWS/CMS. While the system has its limitations, it also has the ability to produce some useful outcome and performance reports for management. For example, throughout the audit, CPS staff provided ample data to audit staff directly extracted from CWS/CMS. The Department filled numerous audit staff data requests by extracting the data from CWS/CMS. Examples of the data provided included number of Emergency Response Investigations; Active Cases; Cases Closed; Current Caseload; Referral Counts by Start Date; and Voluntary Family Maintenance Cases. Since CWS/CMS is an automated version of a case file, all data recorded about each child and family should be available from the system and able to be extracted and aggregated for outcome reporting. Some of these measures listed above, such as the number of families reunited at case closure and number of Social Workers per child, would require development of special reports that are not produced at this time, though they are all technically possible to produce. For example, recidivism data is recorded in individual case files and could be extracted to review family referrals to the Department subsequent to case closures, but this would require production of a new CWS/CMS based report. Production of any performance measurement report requires that all Social Workers input the data needed on to CWS/CMS. All staff do not consistently use the system at this time, making some data collection and analysis difficult to perform. 26 (cid:9)(cid:9)(cid:9) Section 3: Performance Measurement Employee Evaluations As a result of an organization's goals and objectives, performance measures should be tracked to ascertain if the stated purpose of the organization is met. Further, organizations must have a formal system to review employee performance to determine if goals are being met by employees consistent with organization-wide goals. Without evaluation there is no objective, quantifiable measurement of service or staff quality, no technique for fostering improvement and no system to determine if stated employee goals are achieved. Employee performance evaluations can foster improvement in worker morale and employee performance. With detailed employee evaluations, areas for individual employee improvement can be identified and goals set for improvement in those areas. Unless feedback is provided voluntarily by supervisors to their staffs, the absence of an employee evaluation system can encourage the status quo or even a decline in performance as it communicates the message that one's performance doesn't matter. Performance evaluations are a method of communicating and reinforcing an organization's goals and values such as efficiency and responsiveness to customers. An effective performance evaluation system should not be constructed or used as a punitive measure but as a proactive system for management to communicate its expectations to employees and for assisting employees to improve. Employee performance evaluations are crucial in the initial stages of employment when workers are in the socialization process of learning the details of the job. The Memorandum of Understanding (MOU) between El Dorado County and Public Employees, Local Unit No. 1 details the review process for the yearlong probationary period for new employees. The MOU identifies the probation review process as follows: • The employee will receive a written statement of expectations signed by the supervisor and employee. The supervisor shall retain the copy signed by the employee and provide a copy to the employee. • Not less than monthly the supervisor shall meet with the employee to review the employee's progress toward meeting the supervisor's expectations. The supervisor shall provide the employee with a written summary of the meeting. • The employee will acknowledge receipt of the summary of his/her progress by signing a copy of the summary. The supervisor shall retain the copy signed by the employee. • The employee shall be considered to have met expectations in any month in which the supervisor does not meet with the employees and provide them with a written summary of his/her progress. 27 (cid:9) Section 3: Performance Measurement • All written summaries, containing the employee's acknowledgement of receipt, shall be submitted to Human Resources with the appropriate forms for successful completion of probation or of the employee's failure to complete the probation period. In addition to this process, the Department maintains a more formalized employee evaluation process in which probationary employees are reviewed using the Department's standardized Employee Performance Appraisal form after six months and eleven months of employment. These more formal reviews are done in accordance with Merit Systems. Merit Systems is a contractor for the California State Personnel Board that satisfies the State of California's responsibility to ensure that personnel systems which cover county employees of federal grant- in-aid programs comply with federal merit standards. The employee evaluation process begins when an employee is hired. The Department gives the employee a description of the job and a list of essential functions related to the job. According to the Department, employees are given informal monthly reviews. These informal reviews are supposed to consist of a memorandum chronicling the employee's status and highlighting any issues regarding the employee's job performance. These reviews describe the nature of the meeting, and provide some feedback to the employee regarding their performance. The formal employee evaluation process starts with the supervisor completing a formal written evaluation of the employee. Once the appraisal form is complete the supervisor will sign and date the review. The employee will then review, sign and date the evaluation. The form, however, clearly states that the employee signature does not mean the employee is in agreement with the review. At this point, the Program Manager reviews the employee evaluation and signs the form. Unlike the employee signature, the upper management signature does indicate concurrence with the employee review. The Department's instrument to meet the Merit System requirement for a formal performance evaluation is the Employee Performance Appraisal. This form is only completed for the formal evaluations at the six-month review and end of probation. This appraisal is used to evaluate the performance of staff based on several criteria. A review of this document indicates that the evaluator must (a) identify or report on the probation status of employee, (b) identify an overall rating, (c) identify employee goals, and (d) report on the employee's performance using a list of performance factors, which includes job knowledge, output, compliance with rules, and initiative. A supplementary appraisal form, used only in South Lake Tahoe, indicates that the evaluator must (a) list items discussed during the review, (b) identify desired training and training attended, and (c) identify length of time with the Department. As part of this audit, verification of employee performance evaluations was conducted for 13 randomly selected CPS employees at pre and post-probationary stages and for all employees on probation during the previous three fiscal years. Several issues limited the immediate review of the employee evaluations. DSS management was reluctant to provide audit staff access to the employee performance reviews due to confidentiality considerations and outstanding litigation. The County Counsel's Office determined their office could conduct the review and provide the data to audit staff with names redacted so employee privacy would remain intact. Section 3: Performance Measurement Based on this review, the Department does a good job of providing yearly post probationary evaluations. Analysis of the performance evaluations of post probationary employees revealed the Department completed an employee appraisal on a timely basis for all but one of the 13 employees. The employee that did not have a yearly review had their performance evaluated each of the previous six years prior and three years after the missed evaluation. The evaluations reviewed were conducted more than one year after their last evaluations. Excluding the missed review, we found that in 8 of 26 yearly post probation evaluations for six employees, or 30.8 percent of all their evaluations, the Department missed the required yearly time period. On average the missed reviews were 58 days past the one-year period. The longest delay was 141 days more than a year. The shortest delay was 4 days past the one-year mark for a post probation evaluation. This review found significant problems with probationary evaluations. The formal evaluation forms, performed after six months and at the end of probation, are conducted on an inconsistent basis. The Department does a sufficient job of conducting employee evaluations for some employees, while other employees get performance evaluations on a sporadic basis, if at all. As Exhibit 3.1 illustrates, the Department conducted formal Merit Systems evaluations during probation on employees at various frequencies. A majority of probationary employees, 23 out of 30, or 76.7 percent, received between one and three formal evaluations during their one year probation. Two employees, or 6.7 percent, received four formal evaluations, an average of once every three months. Exhibit 3.1 Completion of Formal Employee Performance Reviews during Probation Number of Number of Employees . Percent Probationary Reviews _ 4 2 6.7% 3 7 23.3% 2 9 30.0% 1 7 23.3% 0 3 10.0% N/A 2 6.7% Total 30 100.0% Further, Exhibit 3.1 shows of the 30 employees in the random sample, five employees, or 16.7 percent, did not receive a formal performance evaluation at either the six-month point of employment or at the conclusion of probation. Two of these employees are not applicable since each employee has been with the Department less than six months. However, of the three remaining employees without a formal review, one has worked for the Department for more than 540 days without a formal appraisal of work quality. Additionally, we found a second employee employed at the Department for over 375 days with no formal evaluation conducted, although 29 Section 3: Performance Measurement the Department policy is that these reviews should happen at the 6-month point of employment and again at the end of probation. Moreover, these two employees moved from the probation period of employment to post-probationary timeframe of employment without a formal review. In an examination of nine probationary employees where formal probationary reviews were completed using the Employee Performance Appraisal form, we found minimal consistency in how the results of the formal evaluation were used. Of these nine cases, we found that only one employee was given ratings above standard, the equivalent of above average. The remaining eight employees were all rated as meets standard, which is the equivalent of average or satisfactory, for all of their formal reviews during probation. The findings of the eight employees are presented in Exhibit 3.2. Exhibit 3.2 Probation Results for Eight Employees who Met all Standards in Their Formal Evaluations Result of No. of Average Overall Rating Percent Probationary Employees Number of Reviews Of Review Period 4 50% 1.50 Meets Standards Retained 4 50% 1.25 Meets Standards Terminated 8 100% 1.38 As Exhibit 3.2 demonstrates, based on the formal evaluations half of the employees were terminated at the end or during their probationary period even though their overall ratings were the same as those who were retained. This demonstrates that either these formal evaluations are not documented well or the meaning of the evaluation elements are not consistently understood by all supervisors and the Program Manager who reviews all evaluations. Moreover, of the four terminated employees, three received a recommendation for a step increase in pay in their written evaluation forms. One of the forms with a recommendation for a step increase is dated just 17 days before the employee was terminated. Inconsistencies exist in the informal review process as well. Since conducted on a monthly basis, the informal review process should be a way for the Department to address employment issues and problems to the employee in an effort to obtain improvement. However, a review of the informal evaluations for the eight employees listed in Exhibit 3.2 showed that the Department conducts these reviews on an inconsistent basis. Some employees receive multiple informal performance evaluations, while others do not receive any informal written evaluations. As Exhibit 3.3 illustrates, based on our sample of eight employees with the same ratings in the formal evaluations, the Department conducted informal evaluations during probation on employees at a range of frequencies. One employee received eight informal written performance evaluations. However, three employees never received any informal performance evaluations. Section 3: Performance Measurement Exhibit 3.3 Completion of Written Informal Employee Performance Reviews during Probation Number of Reviews Number ofEmployees Percent More than 4 1 12.5% 4 1 12.5% 3 0 0.0% 2 2 25.0% 1 1 12.5% 0 3 37.5% Total 8 100.0% As previously stated, the Memorandum of Understanding between the union and County states that employees shall be considered to have met expectations in any month in which the supervisor does not meet with the employees and provide them with a written summary of his/her progress. However, in the four cases where the employee was terminated at the end of the probation period, informal evaluations were not always conducted on a monthly basis. Of the four employees terminated during probation, one employee received four informal evaluations, one employee received two, one received one and the fourth employee received no informal evaluations. Without feedback in the form of regular performance evaluations, employees may not be encouraged to improve, especially at the initial stages of employment. It is possible that employees are performing above expectations, but unless that is acknowledged by management a decline in performance is possible as the absence of evaluations communicates the message that one's performance doesn't matter. Based on the sample of performance evaluations reviewed, many employees at the Department have not received an evaluation within the first month of employment. Of the four terminated employees, only one received an evaluation within the first month of employment. The remaining three terminated employees received their initial evaluations at 163 days, 198 days, and 220 days. In addition to the frequency of the evaluations completed on an inconsistent basis, the level of depth in the written memorandum varies considerably. Some of the memorandums are detailed descriptions, listing positive and negative attributes of the worker, as discussed during the informal review. However, a majority of reviews provide minimal detail. A majority of the informal reviews are a list of issues both positive and negative regarding the employee's performance, but detail is minimal and there is not a standard form or consistent criteria used in this informal review process. Without consistent and detailed performance evaluations, it is unclear if Child Protective Services supervisors are doing a sufficient job of documenting communication with employees regarding their performance. Based on informal performance evaluations it is uncertain whether the communication between the supervisor and the employee of these issues is adequate. The 31 Section 3: Performance Measurement Department provided documentation of CPS supervisor notes making comments of discussions regarding employment issues, but these are very brief comments explaining the issue. Moreover, these comments, along with the performance evaluations, occur on an inconsistent basis. As a result of these inconsistencies, the Department should formalize the review process so employees all receive the same number of reviews and that the reviews are done on a consistent form, whether an informal or formal evaluation. Department management should strengthen their evaluation process for probationary employees. Once an employee accepts an offer of employment, the supervisor should conduct an intake interview. The purpose of the interview should be a discussion of the job requirements and a clear statement of the employee's goals, objectives, and critical job tasks. This statement should also include any specific training or other development needs agreed to by the employee and supervisor at the time of hire. The basis of this intake interview should be the approved job description and a clarification of the Department's goals and objectives of the employee during the probationary period. A written statement of the meeting should be developed and signed by both the employee and the supervisor. At a minimum, within the first three months the employee and the supervisor should conduct a documented performance review with a formal evaluation instrument that is based on the outline of job expectations during the probationary period. Analysis of the formal probationary evaluation instrument found employees terminated during or at the end of their probationary period had the same ratings as those who were retained, and some were recommended for a salary increase weeks prior to termination. This indicates that supervisors and the Program Manager who review all probationary evaluations do not have a consistent understanding of the review process or evaluation criteria. To address this problem the Department should implement a policy to have the Deputy Director review all probationary evaluations to ensure consistency between supervisors and the Program Manager. Additionally, using a formal performance evaluation instrument for all evaluations will address the concerns that similar reviews lead to dissimilar outcomes. The performance evaluation form should be refined and the performance measures should be expanded. Additional categories, such as Quantity of Work, Quality of Work, Accountability, Dependability/Reliability, Job/Technical Knowledge, Working Relationships, and Judgment should be implemented to provide employees with additional feedback on performance. The overall ratings should be increased to, at minimum, a five point scale, which includes the following: Well Above Standard, Above Standard, Meets Standard, Below Standard, and Well Below Standard. Additionally, Department management should ensure that post-probation reviews should be conducted annually. These efforts should reduce any uncertainties that may arise during the employee performance review process. Conclusion The Department of Social Services Child Protective Services unit does not have a formal system in place to track its outcomes. The various units of CPS track caseload, but not outcomes. Some other counties in California have established or are establishing outcome based performance measurement systems. CPS has this capability through the CWS/CMS computer system. 32 (cid:9)(cid:9)(cid:9)(cid:9) Section 3: Performance Measurement Without tracking and closely monitoring key outcome measures, CPS and Department management is at a disadvantage in terms of monitoring its performance, identifying existing or potential problems, making necessary changes to improve performance, and measuring the results. The Department is not consistent in its approach to employee evaluations. The Department should conduct formal performance evaluation at least twice during the probation period, but in 33 percent of the time fails to properly do so. In addition, the Department has an informal performance evaluation where a memo is utilized to document employee status. Based on a review of these informal evaluations, 75 percent of employees receive fewer than two of these evaluations. This demonstrates that the Child Protective Services division is not properly documenting employee evaluation.

The Child Protective Services division (CPS) is organized into four units, with one supervisor responsible for each unit. The four supervisors report to a Program Manager who reports to the Deputy Director and Director of the Department of Social Services. The four units vary in size, number of functions and spans of control. The largest unit, the Adoptions unit, has 11.1 full-time equivalent positions, including vacancies, while the smallest unit, South Lake Tahoe, has 5.1 full-time equivalent workers. The Adoptions unit supervisor oversees adoptions social workers in Placerville and South Lake Tahoe and all CPS clerical staff in Placerville. No other supervisor oversees staff from two different functions and in two different offices.

There is one social worker supervisor at South Lake Tahoe but no on-site representatives of upper management. The CPS Program Manager is located in Placerville and is supposed to routinely visit the South Lake Tahoe office but this does not happen on a regular basis. An Administrative Services Officer position is assigned to the South Lake Tahoe office with responsibility for clerical staff and facility management. For social work staff at South Lake Tahoe the absence of an on-site manager results in less access to upper management on CPS issues and less opportunity to address and resolve social worker staff concerns and problems.

The Department should reorganize to address imbalances in supervision levels and the gap in management presence at South Lake Tahoe by: 1) removing oversight of adoptions staff at the South Lake Tahoe office from the Adoptions supervisor in Placerville to the South Lake Tahoe CPS supervisor; 2) reclassifying the Administrative Services Officer position at South Lake Tahoe to a Program Manager, with responsibility for all program staff at that facility. Child Protective Services division supervisors have a variety of responsibilities and perform an essential role in day-to-day operations of the division. As previously stated in the Introduction to this report, the Child Protective Services (CPS) division is comprised of four units: Emergency Response; Ongoing; Adoptions; and, South Lake Tahoe. Each unit has its own supervisor responsible for overseeing their unit's staff. All of the supervisors report to the CPS Program Manager. Fragmentation of Supervision Two of the four supervisors oversee staff in a single functional area at the Placerville office. The South Lake Tahoe supervisor manages all CPS staff at that office in all CPS functional areas except adoptions. The fourth supervisor is responsible for adoptions social worker staff in Placerville and South Lake Tahoe and all CPS clerical staff in Placerville. A review of the Department's organization and staffing charts and documents reveal differences in the assignments and number of staff reports to the supervisors of the different units. The number of social workers and office assistants the supervisors are overseeing varies substantially, and consequently, so does the volume and complexity of their workloads. To assess these differences, we used organizational charts provided for each unit to determine how many positions report to each supervisor and how many levels of staff there are in each unit. The findings are presented in Exhibit 4.1. Exhibit 4.1 Overview of Supervision by Unit Within Child Protective Services Emergency Ongoing South Lake Adoptions Unit Response Unit Tahoe Supervisor Supervisor Supervisor Supervisor Adoptions ER Continuing Clerical overseens Continuing Placerville 4.64 # SW Staff 10.06 7.0 66-5 5.05 #OA Staff 11.14 Placerville Location of South Lake Placerville Placerville South Lake Staff Tahoe Tahoe Location of South Lake Placerville Placerville Placerville Supervisor Tahoe Source: Compiled from Department of Social Services Organizational Chart and interviews with CPS staff. As Exhibit 4.1 shows, the supervisors within CPS lack consistency in their spans of control. Based on the data provided above, CPS supervisors vary in the number of staff they supervise. The numbers supervised range from a high of 11.14 in the Adoptions unit to a low of 5.05 in South Lake Tahoe. Further examination of Exhibit 4.1 shows there is variance not only in the number of employees supervised but also the number of functions and the number of offices where supervised staff are located. Only the Adoptions and South Lake Tahoe supervisors oversee two functions each, the other supervisors are responsible for one function each. Of all the supervisors, only the Adoptions supervisor is responsible for staff in both Placerville and South Lake Tahoe. All of the other supervisors oversee staff in a single location. While there are many opinions and recommended standards regarding supervisor to staff ratios for child welfare agencies, in many organizations, a ratio between 6 and 15 is fairly typical. The Department is within this range though on the high side for the Emergency Response and Adoptions units. The ratios in El Dorado County are similar to those in Santa Clara County. The Emergency Response unit in Santa Clara County supervisor to staff ratio is 8, while in the Continuing unit is it 10. Though the Emergency Response supervisor is responsible for almost as high a number of positions as the Adoptions supervisor (10.06 for Emergency Response vs. 11.14 for Adoptions), the Emergency Response supervisor oversees only one function at one location. The Adoptions Supervisor oversees two dissimilar functions in two locations. The Adoptions supervisor must not only divide time between office assistant and social worker staff in Placerville, but she is also responsible for the adoption social workers in South Lake Tahoe. The difficulty for a supervisor of properly supervising staff performing different functions is compounded in this situation by having staff in two locations. The consequence of this structure is fragmented supervision and often a supervisor who is unavailable to provide proper guidance to the staff. The Department has recognized the problem of having the Adoptions supervisor oversee both clerical and social worker staff. The Department is attempting to add an Office Assistant Supervisor position to provide direct supervision of clerical staff. This ongoing effort would remove office assistant supervision from the Adoptions supervisor and lower the span of control to supervision of only 4.64 full-time equivalent of only adoption social workers. The proximity of the supervisor to line staff has a direct and significant impact on the quality of the supervision. As defined by one study on this subject: The supervisor is the link between the front line of services delivery and upper levels of administration. The supervisor brings the resources of the organization into action at the front line, the point of client contact. Simultaneously, the supervisor communicates information from the unit to upper management, which enables agency administrators to plan and allocate resources. The CPS supervisor has two overarching roles - building the foundation for and maintaining unit effectiveness and developing and maintaining individual staff capacity. The ability of CPS to achieve the goals listed above by the Department of Health and Human Services study is doubtful based on the current organizational structure. The problem with achieving the optimal goals provided above is that without direct supervision on site, it is difficult to achieve unit effectiveness. Because of the lack of proximity between Placerville and South Lake Tahoe the levels of supervision varies dramatically. While the supervisor makes a concerted effort to provide as much guidance and supervision as possible, deficiencies clearly exist in the level of supervision. Critical Nature of Supervision As previously discussed in Section 2, the lack of supervision for After Hours cases is a significant potential liability for the Department. Additionally, supervision in general for CPS is perceived as a critical element to provide services effectively and efficiently. Supervisors must have the expertise and experience needed to provide consultation and guidance to workers in Morton, Thomas and Salus, Marsha. Supervising Child Protective Services Caseworkers. United States Department of Human and Health Services. 1994. 37 decision-making and to teach new skills to workers. The Child Welfare League of America (CWLA) wrote supervision in child protective services is critical to effective service delivery and staff and professional development. Moreover, CWLA further wrote that competency and qualifications of supervisors is critical to assure that caseworkers provide services, engage families from a helping perspective, and follow agency policies and procedures. In short, supervision in child protective services is critical to effective service delivery and staff and professional development. Supervisors should be able to take a step back from the daily operations of line staff and offer critical evaluations of decisions and assist staff to make a decisive decision regarding a case. Based on the survey responses from employees, CPS supervisors were praised as a beneficial and a useful source of information. As shown in Exhibit 4.2, the survey responses are overwhelmingly favorable toward CPS supervision. Exhibit 4.2 Survey Responses Regarding Supervision Disagree Agree My supervisor is almost always available to 14.3% 85.7% help me with m work. My supervisor provides useful guidance to me 5.5% 95.5% in m work. Source: Compiled from survey responses of CPS staff. Based on Exhibit 4.2, when the supervisor is available to provide guidance, social worker and office assistant staff feel the guidance is constructive. Furthermore, the finding that 86 percent of workers felt supervisors are always available to assist in work demonstrates that supervision is generally adequate in CPS. However, the fact that 14 percent felt that supervisors are not available demonstrates that there is some room for improvement. As mentioned previously in Section 2, supervision can stem the potential threat of liability. The risk of liability stems from negligence resulting in an error by a staff member, or when an employee does not follow the Department's policies or procedures. Moreover, Recommendation 5.2 in Section 5 recommends the Department overhaul the policies and procedures manual, thus, creating policies and procedures which should result in more consistency. However, until that happens direct supervision is critical to ensure that staff understands current Department policies to keep the risk of litigation to a minimum. The importance of supervision goes beyond the supervisors with direct contact over line staff. The supervision continues up the chain of command to include program managers and even the Director of the Department. The role of the Program Manager is critical to offer support and guidance to the supervisor, just as the supervisor should provide to the line staff. While the Program Manager has a visible presence in the Placerville office, that does not appear to be the z Child Welfare League of America, "CWLA Standards of Excellence for Services for Abused or Neglected Children and Their Families." Revised Edition, 1999. 38 case in the South Lake Tahoe office. The problem is compounded by the fact that according to the Program Manager she has not had on on-site visit to the South Lake Tahoe office since January 2002, at the time of this report some four months without an on-site visit to that office. In the past, a full time Program Manager was assigned to oversee all programs at the South Lake Tahoe office3. This position was reportedly eliminated due to difficulties in managing the broad range of programs at that office. A position with the level of authority greater than any in the office at this time is appropriate as it would provide on site management presence and the ability to more quickly resolve program management issues. In interviews and surveys, staff at South Lake Tahoe reported weaker communications with upper management than reported by staff in Placerville, as demonstrated by the survey responses presented in Exhibit 4.3. Exhibit 4.3 Survey Responses Regarding Supervision Agree- Disagree- Agree- Disagree- South Lake South Lake Placerville Placerville Tahoe Tahoe The CPS Manager regularly communicates with staff about 87.5% 0.0% 12.5% 100.0% Department activities andpolicies Communications between the Department's top management and 73.3% 0.0% 26.7% 100.0% CPS are very good Source: Compiled from survey responses of CPS staff. Consistent and regular contact by the Child Protective Services Program Manager with CPS staff on a monthly basis in an open forum and meetings would provide South Lake Tahoe staff with a management representative who could respond to program issues raised by staff and resolve problems more quickly than under the current structure. Moreover, to ensure that the meetings take place and the Program Manager is holding meetings on a monthly basis, the meetings should be documented and their occurrence reported to the Board of Supervisors every 6 months and if they have not occurred, explanations for why not should be reported. This will ensure that the meetings happen on a consistent basis. Moreover, the meetings would address issues and would the Program Manager would provide consultation and problem solving authority to the South Lake Tahoe staff. Conclusion The Child Protective Services organizational structure contributes to inconsistency in the level of supervision a worker receives due to variances in the number of employees, functions and office locations overseen by CPS supervisors. Supervision is accepted as a critical element in CPS, however, the current structure is organized in such a way that some staff receives less 3Besides CPS, Department Special Investigations, Income Maintenance, Employment Services, and Staff Services are also located at the South Lake Tahoe office. 39 (cid:9)(cid:9)(cid:9)(cid:9) supervision than others. The organizational units within Child Protective Services vary in size, number of functions and spans of control exercised by the supervisors. The Adoptions supervisor oversees 11.1 clerical and social worker full-time equivalent positions (FTEs) in two locations while the South Lake Tahoe supervisor oversees 5.1 FTEs in one location. This structure makes it impossible for all staff to receive a similar level of supervision and guidance. Though the CPS Program Manager is responsible for overseeing staff in both Placerville and South Lake Tahoe, the South Lake Tahoe office receives substantially less attention than the Placerville staff and irregular on-site visits.

Communications with top management of the Department of Social Services is considered poor by many staff social workers and office assistants. Many staff members feel there is no forum to express their concerns and problems and to make suggestions for improvements. This is particularly true in the South Lake Tahoe office as visits and meetings with staff there by the CPS Program Manager and the Director and Deputy Director are infrequent. Regular forums with open communications between staff and managers should be employed as a mechanism for improving CPS processes and services and to improve staff morale.

The CPS polices and procedures manuals are not up to date and omit some key areas of operations such as how the CWS/CMS computer system should be used for case intake and processing. Incomplete, out of date or missing policies and procedures could lead to inconsistencies in staff approaches to case work. Most staff surveyed reported that the Department's rules and regulations are not clear or consistently enforced. The manuals are now all hard copy paper documents and could be placed on the Department's computer system for easier updating and access by all staff.

Many staff members believe that more and different types of training are needed to ensure greater consistency in approach by staff. A mentor program for new staff is one approach suggested by staff which Department management could implement on a pilot basis and assess its costs and benefits to see if it should be replicated throughout the CPS division.

The Department does not have a formal written policy or formal reporting mechanism for client and family complaints. Such a policy is needed to ensure consistency in responses to complaints and to ensure that management is kept informed of all complaints and staff responses and correct ive actions. For staff and supervisors to perform in a manner that will lead to the consistent and desired outcomes for an organization, management must first clearly and accurately define policies and procedures. Second, management must communicate these policies and procedures to staff. Then, adequate resources and supports for staff to perform successfully must be provided. Finally, and most important, the organization must maintain a capacity to obtain feedback and measure and report the degree of success it has achieved in following the policies and to determine if the policies are effective. Deficiencies, in various forms, were identified in each of these elements at Child Protective Services. Communications The employee survey conducted for this audit showed that many CPS employees perceive communication between management and staff as a problem. Exhibit 5.1 shows the responses from social workers and office assistants to statements regarding communications within CPS. Section 5. Communications As the Exhibit clearly shows, line staff feel there is a problem with communications in the Department, particularly with top management. Exhibit 5.1 Survey Responses Regarding Departmental Communication Agree Disagree Communications between Department's top 41.7% 58.3% staff and CPS staff are very good Top Management of the Department are very 50,0% 50.0% supportive of CPS Top Management of the Department is very 33.3% 66.7% responsive to suggestions from staff I can influence matters above me 16.7% 83.3% Source: Audit survey of CPS social worker and office assistant staff Only 16.7 percent of the survey respondents believe they have the ability to influence matters above them. Overall, 83.3 percent of office assistants and social workers felt they have little or no influence over Department matters. Further, 66.7 percent of the survey respondents believe that top management is not responsive to suggestions or concerns. Of greater concern is that half of all survey respondents, 50 percent, felt that Department management was not supportive in general of the Child Protective Services division and only 41.7 percent believe top management and CPS staff communications are very good. These perceptions are not conducive to high staff morale or a motivated work force. Child Protective Services management conducts staff meetings in Placerville bi-weekly. These meetings are attended by the CPS Program Manager, supervisors and line staff. A review of staff meeting minutes demonstrated that the CPS staff meetings are primarily a forum for management to update staff on new Department developments, changes in polices and related matters. The Department Director and Deputy Director do not attend these meetings. While CPS should be commended for its efforts to keep staff informed, the meeting minutes demonstrate that staff is not provided with an opportunity to express concerns and grievances or make suggestions for program improvements at these meetings. Department staff indicate that forums to provide such opportunities do not occur on a regular basis. Regular staff meetings with the CPS Program Manager, supervisors and staff do not take place at the South Lake Tahoe office as they do in Placerville. The managers based in Placerville may discuss issues and concerns with South Lake Tahoe supervisors, but there is minimal, if any, group communication and feedback between the Program Manager and South Lake Tahoe line staff. Without regular contact with management, many of the South Lake Tahoe staff feel they do not have a place to air grievances, complaints or make suggestions for improvements. One technique for improving communications between management and line staff would be to set aside time during the regularly scheduled meetings where employee concerns and suggestions can be freely expressed directly to upper management. As such, the Director of the Department 42 Section 5: Communications and/or the Deputy Director should attend these meetings which should take place on a regular basis in both the Placerville and South Lake Tahoe offices'. It is of equal importance that top management respond to issues, concerns, and suggestions on a timely basis. Many employees expressed frustration in their written comments on the audit survey over the lack of response from management toward their suggestions for improvements made in the past. Furthermore, employees often felt when a response came from management it was long after the employee suggestion was made if a response came at all. The Department should have strong support systems in place to provide workers with open forums to discuss and deal with dissatisfactions and frustrations, and suggestions for how to make constructive improvements. The forums should not become a series of sessions where workers continually complain about the Department but should be directed toward identifying and solving problems. The discussion above is not meant to imply that upper management should immediately implement any and all suggestions made by staff. But the process will be perceived as meaningless if management does not respond to staff suggestions and grievances within a reasonable amount of time, even if the decision is to not implement the staff suggestion. Responses to staff suggestions should be communicated directly to employees by upper management, including the Director, through as few layers as possible. By taking timely actions to address employee concerns, even if the actions are not what employees suggested, management would demonstrate responsiveness to staff concerns. Regularly communicating how and why decisions have been made to staff would also help improve morale. Updating the Policies and Procedures Manual Child Protective Services possesses extensive policies and procedures manuals that provide guidance to social workers and staff regarding Department operations. However, the manuals are lacking some key policies and procedures such as reference to the Department's CWS/CMS computer system and policies and procedures for office assistant support staff. Without documentation of all key areas of operations, the chances increase of certain activities being inconsistently applied throughout CPS. This current state of Department policies and procedures is consistent with findings of the FY 1999-2000 El Dorado Grand Jury which reported: The Grand Jury requested and received a copy of the El Dorado County CPS Policy and Procedures manuals. Analysis indicated that the manuals contained many outdated or undated documents, documents whose origin could not be determined, unsigned documents, and documents that referred to manual record keeping operations which had long since been replaced by computerized operations. This dilapidated state appeared to have been in existence for several A recommendation in Section 4 calls for routine visitation by the Program Manager to South Lake Tahoe so that upper management is represented at that office and to provide a means of resolving staff concerns and issues. One of the appropriate roles of this position would be conducting staff forums as described above. Section 5: Communications years. In May, 2000, the Grand Jury observed that these manuals had been professionally updated. Further, it is noted that the Department of Social Services (DSS) has initiated other corrective actions, including initiation of periodic internal audits to ensure compliance with state requirements. The Department disagreed with the previous Grand Jury finding wholly, stating "first, the outdated manuals were state regulations that are outdated. Second, the professionally updated manuals were prepared by DSS." However, during our review of CPS policy manuals we found that some formal statements were outdated and appeared to be produced by the Department. Many of the policies that appear outdated involve some of the most important responsibilities facing the Department and social workers such as use of CWS/CMS in processing cases. The Department's policies and procedures regarding the overview of CPS appears to be dated 1992. The manual was written prior to the full implementation of CWS/CMS by the State. Thus, Department policies and procedures regarding the intake and screening process do not make any reference to CWS/CMS. As demonstrated in Section 2 of this report, documentation of intake processes could not always be found on CWS/CMS, indicating inconsistent application of Department policies and procedures. As discussed below in more detail, CPS does not have formal written policies regarding client complaints. Although, management provided a detailed oral description of how the client complaint process works, without a formal written policy in place regarding how to address concerns raised against the Department, responses to complaints have the potential to be inconsistently handled. Department management acknowledges these lapses in the policies and procedures, however, they do not feel it is necessary to rectify these deficiencies. Clerical staff perform a vital role within CPS but currently are working without any formal policies regarding their roles and duties. According to staff, an effort is underway to complete a formal clerical policies and procedures manual but it is not yet complete. Thus, clerical staff are not provided with a training manual when new to the job and must learn their duties primarily through on-the-job training without written materials to use as references. Samples, training and written policies are particularly important for the Department's court report procedures, which are very specific and must comply with the requirements of the State Welfare and Institutions Code. In some counties, clerical staff has specific written guidelines of their duties and responsibilities. In Los Angeles County, for example, a clerical policies and procedures manual deals with many issues similar to those faced by El Dorado County's clerical staff. For instance, Los Angeles County has specific policies and procedures regarding how to handle instances of a mail referral containing allegations of child abuse or neglect. Los Angeles County policies also include court procedures ranging from processing proof of service notices for Welfare and Institutions Code 366.26 hearings to entering Court results into CWS/CMS. The results of the employee survey for this audit demonstrate staff concerns regarding Department policies and procedures. Exhibit 5.2 presents responses from social workers and 44 Section 5. Communications office assistants regarding Department policies and procedures and enforcement of these procedures. Exhibit 5.2 Survey Responses Regarding Department Rules and Regulations Question Agree Disagree Department rules and regulations are clear 22% 78% Department rules and regulations are 18% 82% consistently enforced Source: Audit survey of CPS social worker and office assistant staff Nearly 8 out of 10 survey respondents, or 78 percent, stated that rules and regulations in the Department are not clear. Further, 82 percent of survey respondents stated that regulations are inconsistently enforced. This demonstrates a combination of incomplete written policies and procedures and/or lack of consistent implementation. One reason staff may feel policies and procedures are inconsistent and unclear is because the Department does not present them in a standardized format. At least three different formats are used to explain and illustrate current policies, each with different levels of information. By comparison, Los Angeles County has very formal policies and procedures regarding their entire Child Protective function all presented in a consistent format. Each policy is given a formal issue date, the employees the policy is applicable to, associated documents, and relevant Code sections. Additionally, each policy provides a detailed description of the adopted policy and provides a step-by-step detailed description for guidance of the procedure. For some of El Dorado County's CPS policies and procedures, the format includes subject, adoption date, revised date, reference and a detailed explanation of the policy and procedure. However, this format is the exception rather than the rule in the Department's written policies and procedures. A standardized format with the data elements listed above should be used for all procedures to provide staff with improved guidance and clarity in performing their job duties. Improvements to Department policies and procedures should include making the manual available on-line to ensure that all staff members have access to the most up-to-date versions of all policies and procedures and that they are applied consistently throughout CPS. In written responses to the audit questionnaire, numerous staff expressed concern that each staff member did not have their own copy of the manual. An on-line version would be particularly useful for ensuring that the South Lake Tahoe staff has access to any updates or revisions to Department policies and procedures. All staff should be oriented and trained on the use of the on-line policies and procedures manual as soon as it is fully updated. When new policies or procedures are implemented, supervisors should be directed to discuss the changes with all unit staff at unit meetings rather than distributing copies to staff. That way accountability for remaining current with management approved policies and procedures would rest with the supervisor, rather than the line staff person. Section S: Communications Training One example of a staff concern that could be addressed through the recommended staff forums is training. Throughout the audit process of interviewing line staff, supervisors and managers, concern about training was a common theme. Without exception, those that commented on employee training indicated that training is not done on a regular basis, and/or more training is needed but, for various reasons, is not taking place. Training is necessary to ensure supervisor, social workers and office assistants have the specialized skills and knowledge required to effectively provide the functions of CPS. Of the employees that responded to a survey conducted for this audit, 70 percent stated that not all CPS staff had received adequate training to perform their jobs. Many social workers believe that ongoing training was a key weakness in Child Protective Services. The main concern voiced by staff at all levels was that the training received by new employees of CPS was not sufficient given the complexity of the job. In discussions, some employees suggested that a mentoring program for new employees could provide new staff with a more useful hands-on training in the various functions performed by the Department. A staff forum for exchanging ideas and considering improvements in operations would be a place to discuss ideas such as this. Department management could then consider the costs and benefits of a mentoring program or other training enhancements and respond to staff within a specified time frame concerning whether or not the idea could be implemented. Key to management's decision should be an assessment of the costs and benefits of any proposals with benefits expressed in measurable terms such as reduced turnover, improved family outcomes, or reduced delays in processing cases. Client Complaints The Department does not have written policies and procedures regarding complaints filed against the CPS division or its social workers. Department management reports a single procedure for responding to all complaints filed against Child Protective Services. According to management, CPS requests that grievances be formally submitted to the Department in writing. Once a complaint is received, the Program Manager reports that she and the supervisor will develop a response and, in some cases, the complaint will be investigated. When the response is finalized, it is provided to the complainant in writing. The Program Manager and the supervisor responsible for the unit determine the final outcome. In response to a request by auditors for copies of documentation regarding complaints against any CPS social workers or CPS practices, the Department provided copies of seven complaints and all documentation prepared in response to the complaints, all of them dated from January to December 2001. The nature of the complaints against CPS included transfer of cases between two social workers, visitation complaints, placement complaints, and inadequate case plan therapy. One file contained a letter stating that the Department had failed to respond to an original complaint, sent 133 days earlier. 46 (cid:9)(cid:9)(cid:9)(cid:9) Section 5: Communications Several of the complaint files contained documentation of the follow-up investigation conducted by the Department in response to the initial complaint. The thoroughness of the investigation, based on the documentation provided, ranges from extremely detailed to very brief. One letter chronicles the specific details of the investigation and makes it clear the Department indeed investigated the grievance. However, the letters explaining the details of the follow-up investigation make it difficult to determine the Department's response. The Department should document and implement a formal consistent policy regarding how CPS responds and documents formal complaints. The Department should create a centralized, computerized database detailing the grievance and the response to said complaint. The computerized database should be accessible to County Counsel staff to conduct a periodic review to determine if the Department is handling complaints properly. A regularly produced summary report, such as quarterly, should be provided to the Department Director by the CPS Program Manager reporting the date of each complaint received, date the internal response was initiated, nature of internal response (e.g., full investigation, employee discipline, etc.), and date of written response to complainant. The Department should institute formal procedures governing when and how complaints are investigated, as follows: • The complainant shall make a formal complaint in writing and the Program Manager and/or unit supervisor shall have initial contact with the complainant within 10 working days; • The investigation and final decision shall be rendered by Child Protective Services within 10 working days once contact is made; • The final decision shall be signed off by the Director of Social Services or the Deputy Director of Social Services; and • The Department has the authority for formal disciplinary action if the investigation finds the staff violated State law or DSS policy. A benefit of this review process and a centralized database is that CPS will have access to documentation currently unavailable to determine any trends in grievances. The review and the database could provide a mechanism to implement further policy and procedure changes to minimize the number of complaints. Conclusion The employee survey conducted for this audit showed that communications between management and staff is not perceived positively by most Department employees. Specifically, most line staff believe upper management is unresponsive to their needs and that staff has no ability to influence matters above them within the Department. A widely held belief of CPS staff is that Department policies and procedures provide inadequate guidance for job tasks. Analysis of the Department's policies and procedures manual found that complete formal policies do not exist for some key aspects of CPS, such as grievances and clerical duties and responsibilities, and. that existing policies and procedures are presented inconsistently and without key information such as the date the policy became effective. Many of the existing policies are old 47 (cid:9)(cid:9)(cid:9)(cid:9) Section 5: Communications and outdated. Our analysis found that child removal policies were written prior to the implementation of CWS/CMS, and the manual makes no reference to the program. Many employees expressed frustration at the level of training, especially relating to clerical support and new hires. Child Protective Services does not have written formal policies governing grievance procedures. Even where information was presented in the actions taken by the Department, the standard response or evidence of ample investigations could not be consistently documented.