San Mateo County Grand Jury
• 2017-2018
Project Management of the Caltrain Modernization Program Issue | Summary | Glossary | Background | Discussion |
⚠️ Translation Notice: This content has been automatically translated. The original English text is the official version. Translation may contain errors.
⚠️ Este contenido ha sido traducido automáticamente. El texto original en inglés es la versión oficial. La traducción puede contener errores.
Note: Missing finding numbers detected: F2
Findings and Recommendations 11 findings
F1
The CBOSS PTC project had several management failings as noted by the American Public Transportation Association (APTA) report. Among the Caltrain management issues noted by APTA were: No commonly approved project schedule between Caltrain and Parsons Lack of a strong in-house technical team to oversee the project No commonly agreed upon definition of some final deliverables or how to test them Lack of a working escalation method to resolve issues raised between Caltrain and Parsons
Related Recommendations (1)
R6
TECHNOLOGICAL: EFFECTIVENESS OF DESIGN In the view of the panel, animosities between the prime contractor and project management oversight present an impediment to resolving outstanding technical issues. PTG appears to have appropriate technical resources to complete CBOSS requirements for Caltrain running on Caltrain tracks. It is apparent that TASI resources have not been fully engaged during the course of the project. RE
F3
The CalMod project schedule process appears coordinated and integrated among the main contractors, consultants, and CalMod. The duties of the consulting firm of Gannett Fleming include schedule coordination.
No recommendations for this finding
F4
While membership in the Change Management Board is not completely settled, the board is evaluating and approving changes as they occur.
No recommendations for this finding
F5
CalMod’s Risk Assessment Committee (made up of representatives from funding partners and CalMod management) meets monthly to identify risks and mitigation measures. Progress in mitigating risks enumerated in the Program Risk Register is reviewed at this meeting and on an ongoing basis by the person employed as Risk Management Lead.
No recommendations for this finding
F6
CalMod appears to have an experienced Chief Officer and senior managers in place.
No recommendations for this finding
F7
Quality control and quality assurance processes appear to be functioning as designed.
No recommendations for this finding
F8
Caltrain has adequate management processes in place to implement a project of this scale.
No recommendations for this finding
F9
Most CalMod project information is available to the public, but it is not readily accessible to members of the public who do not know exactly where to look for it. It is difficult for the public to understand if the CalMod project is on budget and schedule without access to easily decipherable summary reports.
Related Recommendations (1)
R1
Caltrain should publish an interactive dashboard focusing on overall project schedule, budget, and deliverables that is readily visible on the CalMod homepage (calmod.org). The dashboard should include links to the supporting data in the Monthly Progress Reports and other places. This recommendation should be implemented
F10
Transportation agencies such as the Virginia Department of Transportation and LetsGoCT! use interactive project status dashboards to keep the public informed about their large and complex projects and budgets.
No recommendations for this finding
F11
The Monthly Progress Reports delivered to the PCJPB show total current expenditures compared to the overall budget. In particular, they show “Cost this Month” and “Cost to Date.” but do not show a budget timeline of current expenditures to planned expenditures on a monthly basis. Graphs showing draw downs on CalMod’s $315 million contingency fund that are provided to the PCJPB do not include a narrative explanation. Without a variance analysis and narrative, it is difficult for members of the public to understand why a deviation from the budgeted amount occurred.
No recommendations for this finding
F12
The summary information provided to the public regarding scheduling milestones in the body of Monthly Progress Reports is insufficient. The public cannot tell from that summary if CalMod is on schedule when the published milestone chart and the critical path charts in that summary have entries that have gaps as long as two years. The appendix to each Monthly Progress Report does contain more detailed schedule information in the form of a summary Master Program Schedule, but understanding it requires some knowledge of Gantt charts to fully understand the information.
Related Recommendations (1)
R3
A high-level CalMod project schedule should be published every month showing the progress of the project against the planned timeline. The schedule should have quarterly milestones so that the public can determine if the overall project is on schedule. This schedule should be included in the Executive Summary and Schedule sections of the Monthly Progress Reports. This recommendation should be implemented
Additional Recommendations 3
These recommendations are not explicitly linked to specific findings.
-
R2CalMod should publish an explanation of how total project spending is tracked against the planned budget. Spending more or less than budgeted should be explained and a brief explanation of how the budget will be returned to plan (if possible) should be included. This information should be appended to the Monthly Progress Reports. This recommendation should be implemented
-
R4TECHNOLOGICAL: INTEROPERABILITY DESIGN VIABILITY It appears that the current status of software does not support interoperability with tenant and host railroads and that configuration management of the versions of system software control is lacking. It does not appear that interoperability will be included with the October 2016 revenue service demonstration. RE
-
R5TECHNOLOGICAL: EFFECTIVENESS OF DESIGN The logic for the fiber optics design and distribution was well planned and can be leveraged for future revenue. The control center design incorporates state of the art technology, and is well laid out and labeled. The On Board Computer (OBC) has approximately 10,000 lines of code which is considered a small system that promotes maintainability and robustness. However, in reviewing the open software defects list, it appears that at least one of the defects is said to crash the OBC. As noted by the current defect list, there are multiple communication network issues that remain to be resolved. The panel notes that the security layer of the software is an older application that is vulnerable to cryptographic intrusion. RE
Conclusions 1
-
CL1 Page 21The Grand Jury began its investigation into whether Caltrain had put in place the processes necessary for effective and efficient management of the almost $2 billion CalMod program based in part on the reported Caltrain management issues in the CBOSS PTC project. The Grand Jury wanted to know if those issues might also be present in CalMod. It also examined the management processes put in place implement a project of this scale. The Grand Jury also explored the extent to which interested members of the public could easily determine the status of the CalMod project, including whether the project is on time and on budget. In the Grand Jury’s view, Caltrain has hired a Chief Officer and senior managers with the education and experience necessary to manage the program. The Grand Jury also believes that reasonable processes and procedures are in place to properly manage this mega project. However, the Grand Jury is not auditing the project or speculating on its success. The Grand Jury is aware that a 140-page Monthly Progress Report and Quarterly Update within an almost 300-page agenda packet and others like it presented to the PCJPB leaves readers, especially the public, drowning in data but starving for information.123 Moreover, Monthly Progress Reports are difficult for the public to find because they are not prominently highlighted on either the Caltrain or CalMod websites. 120 “It all comes down to transparency,” Frederick P. Salvucci, Boston Globe, October 3, 2010. http://archive.boston.com/bostonglobe/editorial_opinion/oped/articles/2010/10/03/it_all_comes_down_to_transpare ncy/. 121 Virginia A. Greiman, Megaproject Management: Lessons on Risk and Project Management from the Big Dig, Project Management Institute (Hoboken, New Jersey: John Wiley & Sons Inc, 2013), chapter 3. https://books.google.com/books?id=zZrw3wXT8TEC&printsec=frontcover#v=onepage&q&f=false. 122 “Justice Louis D. Brandeis,” Louis D. Brandeis Legacy Fund for Social Justice, Brandeis University, accessed May 22, 2018. https://www.brandeis.edu/legacyfund/bio.html. 123 “Agenda Peninsula Joint Powers Board (May 3, 2018),” Caltrain, accessed June 1, 2018. http://www.caltrain.com/Assets/__Agendas+and+Minutes/JPB/2018/2018-05- 03+JPB+FINAL+MEETING+PACKET.pdf. 2017-2018 San Mateo County Civil Grand Jury 21
No Responses Found 1
Government entities assigned to respond to this report. No response documents have been linked in our database.