CSS Tracking, Monitoring, and Evaluation Project
The Community Services and Supports (CSS) Tracking, Monitoring, and Evaluation System Project has been initiated by the California Mental Health Services Oversight and Accountability Commission (MHSOAC) to assess the CSS component of the Mental Health Services Act (MHSA) and to consider how all aspects of this system, including client outcomes, can best be tracked, monitored, and evaluated. The project involves the development, piloting, and preliminary use of a data collection and reporting system that will track, monitor, and evaluate the full CSS component. This data collection and reporting system will be designed to enable providers, counties, and the State to understand the clinical and functional status of clients within individual CSS programs/services, and determine whether clients are enrolled in appropriate services. In essence, this system should provide client-level outcomes and data that speak to the appropriateness of a client’s current level of care and characteristics of that care.
CSS is the largest component of the MHSA and includes client and family driven services that focus on wellness and integrated service experiences for clients and families, as well as providing services for traditionally un-served and underserved populations. The CSS component includes Full Service Partnerships (FSP), which are designed to provide comprehensive services to the highest-need clients in the system (e.g., those with severe mental illness/emotional disturbance who have co-occurring histories of homelessness, incarceration, and/or institutionalization) as well as services for those who may not qualify for FSP services.
To date, a larger focus has been placed on evaluating outcomes and services for clients served through FSP programs than for clients receiving less comprehensive services from other CSS programs. This project represents MHSOAC’s goal to expand the evaluation focus to include clients who receive less comprehensive services than FSP clients. Throughout the project, consideration of currently used methods to track, monitor, and evaluate both FSP and non-FSP clients will be necessary due to the continuum of care that is offered to all clients served through the CSS component.
As part of this project, lessons learned via the development and piloting of a new system for the full CSS component and initial evaluation of services for non-FSP clients will be used to generate policy implications and recommendations for next steps that would facilitate further development and full implementation of a statewide system to track, monitor, and evaluate the full CSS component. The project will also inform policy recommendations regarding current CSS-related statutes and regulations that may hinder the ability to properly place and serve clients within service settings that are appropriate for their needs and current level of care.
This evaluation represents another step toward continuous assessment of the MHSA and the broader public community-based mental health system that is focused on accountability and quality improvement and guided by MHSA values and principles. The ultimate goal of this project is to increase our ability to understand and improve upon the quality of services offered via the CSS component and the system that supports these services.
The overarching questions to be addressed during the course of this project include the following:
- What statewide methods should be employed to ensure that providers, counties, and the State can track, monitor, and evaluate the status of adults who are receiving CSS services in order to determine the efficacy and appropriateness of those services?
- What policies, practices, systems, and infrastructure should be created and/or modified in order to better track, monitor, and evaluate adults who are receiving CSS services?
- How effective are services for adults who receive less comprehensive services than what is provided via Full Service Partnerships? (i.e., how effective are services for non-FSP clients?)
- What policies, practices, systems, and infrastructure should be created and/or modified in order to better serve adults within the CSS component? (i.e., what new and/or revised methods may further our understanding of CSS client clinical/functional status, current client level of care/service, the most appropriate level of care/service, as well as our ability to provide the most appropriate level of care?)
In order to address the above listed questions, the evaluation team is working collaboratively with an Evaluation Advisory Group comprising clinicians, providers, county staff, contractors, state staff, persons with lived experience, and end users of the tracking, monitoring, and evaluation system to:
- Determine what should be measured for adult behavioral health outcomes, how and when it will be measured and by whom, while integrating data that is already collected to minimize burden on staff and clients. Also, maximize the clinical usefulness of outcomes tracking to enhance people’s recovery.
- Develop and implement an adult behavioral health outcomes tracking and monitoring system that meets the needs of various stakeholders by providing meaningful data to evaluate the impact of mental health services.
- Create policy and practice recommendations for future directions in evaluating mental health services by performing a preliminary evaluation of some MHSA mental health services using the new data collection system.
Through this project, the MHSOAC expects to complete the initial development and piloting of a system, as well as a preliminary evaluation of adult clients being served via CSS in less comprehensive services than FSPs. In addition, an assessment of the policies, regulations, and guidelines that pertain to the CSS component will be carried out in order to better understand potential limitations of current policies and practices that may hinder the provision of effective and appropriate services for CSS clients.
Stakeholder feedback gathered on the initial development, piloting, and evaluation of the system will be used to build the foundation for a future statewide system. A successful statewide system would enable providers, counties, and the State to continuously track, monitor, and evaluate CSS component performance for clients in FSP and non-FSP programs. A successful system may also enable clients, family members, and the general public to better understand the performance of individual providers, as well as the performance of the overarching statewide public community-based mental health system. This tracking, monitoring, and evaluation system can then be used to improve both the quality of services offered to adult clients, and the system through which the services are offered. Paramount to this effort is the creation of data-driven recommendations for improving the capacity of the CSS component to promote positive outcomes in clients, including proper movement of clients through the systems of care and the receipt of appropriate treatment/services.
This project will result in five reports (the project’s deliverables), listed below. The expertise and assistance of the Evaluation Advisory Group will be an essential part of the development of all five of these reports:
1. Report of Proposed Tracking, Monitoring, and Evaluation System.
2. Report of Proposed Implementation Plan to Pilot the system in a sample of providers/counties.
3. Report of Proposed Research Design and Analytic Plan to evaluate the efficacy of non-FSP CSS services.
4. Report of Pilot Evaluation Results.
5. Report of Policy and Practice Recommendations for how to Improve upon current CSS services, evaluations, and systems.