The Community Services and Supports (CSS) Tracking,
Monitoring, and Evaluation System Project has been initiated by the
California Mental Health Services Oversight and Accountability
Commission (MHSOAC) to assess the CSS component of the Mental Health
Services Act (MHSA) and to consider how all aspects of this system,
including client outcomes, can best be tracked, monitored, and
evaluated. The project involves the development, piloting, and
preliminary use of a data collection and reporting system that will
track, monitor, and evaluate the full CSS component. This data
collection and reporting system will be designed to enable providers,
counties, and the State to understand the clinical and functional status
of clients within individual CSS programs/services, and determine
whether clients are enrolled in appropriate services. In essence, this
system should provide client-level outcomes and data that speak to the
appropriateness of a client’s current level of care and characteristics
of that care.
CSS is the largest component of the MHSA and includes
client and family driven services that focus on wellness and integrated
service experiences for clients and families, as well as providing
services for traditionally un-served and underserved populations. The
CSS component includes Full Service Partnerships (FSP), which are
designed to provide comprehensive services to the highest-need clients
in the system (e.g., those with severe mental illness/emotional
disturbance who have co-occurring histories of homelessness,
incarceration, and/or institutionalization) as well as services for
those who may not qualify for FSP services.
To date, a larger focus has been placed on evaluating
outcomes and services for clients served through FSP programs than for
clients receiving less comprehensive services from other CSS programs.
This project represents MHSOAC’s goal to expand the evaluation focus to
include clients who receive less comprehensive services than FSP
clients. Throughout the project, consideration of currently used methods
to track, monitor, and evaluate both FSP and non-FSP clients will be
necessary due to the continuum of care that is offered to all clients
served through the CSS component.
As part of this project, lessons learned via the
development and piloting of a new system for the full CSS component and
initial evaluation of services for non-FSP clients will be used to
generate policy implications and recommendations for next steps that
would facilitate further development and full implementation of a
statewide system to track, monitor, and evaluate the full CSS component.
The project will also inform policy recommendations regarding current
CSS-related statutes and regulations that may hinder the ability to
properly place and serve clients within service settings that are
appropriate for their needs and current level of care.
This evaluation represents another step toward
continuous assessment of the MHSA and the broader public community-based
mental health system that is focused on accountability and quality
improvement and guided by MHSA values and principles. The ultimate goal
of this project is to increase our ability to understand and improve
upon the quality of services offered via the CSS component and the
system that supports these services.
The overarching questions to be addressed during the course of this project include the following:
- What statewide methods should be employed to ensure that
providers, counties, and the State can track, monitor, and evaluate the
status of adults who are receiving CSS services in order to determine
the efficacy and appropriateness of those services?
- What policies, practices, systems, and infrastructure should be
created and/or modified in order to better track, monitor, and evaluate
adults who are receiving CSS services?
- How effective are services for adults who receive less
comprehensive services than what is provided via Full Service
Partnerships? (i.e., how effective are services for non-FSP clients?)
- What policies, practices, systems, and infrastructure should be
created and/or modified in order to better serve adults within the CSS
component? (i.e., what new and/or revised methods may further our
understanding of CSS client clinical/functional status, current client
level of care/service, the most appropriate level of care/service, as
well as our ability to provide the most appropriate level of care?)
In order to address the above listed questions, the
evaluation team is working collaboratively with an Evaluation Advisory
Group comprising clinicians, providers, county staff, contractors, state
staff, persons with lived experience, and end users of the tracking,
monitoring, and evaluation system to:
- Determine what should be measured for adult behavioral health
outcomes, how and when it will be measured and by whom, while
integrating data that is already collected to minimize burden on staff
and clients. Also, maximize the clinical usefulness of outcomes tracking
to enhance people’s recovery.
- Develop and implement an adult behavioral health outcomes
tracking and monitoring system that meets the needs of various
stakeholders by providing meaningful data to evaluate the impact of
mental health services.
- Create policy and practice recommendations for future directions
in evaluating mental health services by performing a preliminary
evaluation of some MHSA mental health services using the new data
Through this project, the MHSOAC expects to complete
the initial development and piloting of a system, as well as a
preliminary evaluation of adult clients being served via CSS in less
comprehensive services than FSPs. In addition, an assessment of the
policies, regulations, and guidelines that pertain to the CSS component
will be carried out in order to better understand potential limitations
of current policies and practices that may hinder the provision of
effective and appropriate services for CSS clients.
Stakeholder feedback gathered on the initial
development, piloting, and evaluation of the system will be used to
build the foundation for a future statewide system. A successful
statewide system would enable providers, counties, and the State to
continuously track, monitor, and evaluate CSS component performance for
clients in FSP and non-FSP programs. A successful system may also enable
clients, family members, and the general public to better understand
the performance of individual providers, as well as the performance of
the overarching statewide public community-based mental health system.
This tracking, monitoring, and evaluation system can then be used to
improve both the quality of services offered to adult clients, and the
system through which the services are offered. Paramount to this effort
is the creation of data-driven recommendations for improving the
capacity of the CSS component to promote positive outcomes in clients,
including proper movement of clients through the systems of care and the
receipt of appropriate treatment/services.
This project will result in five reports (the
project’s deliverables), listed below. The expertise and assistance of
the Evaluation Advisory Group will be an essential part of the
development of all five of these reports:
1. Report of Proposed Tracking, Monitoring, and Evaluation System.
2. Report of Proposed Implementation Plan to Pilot the system in a sample of providers/counties.
3. Report of Proposed Research Design and Analytic Plan to evaluate the efficacy of non-FSP CSS services.
4. Report of Pilot Evaluation Results.
5. Report of Policy and Practice Recommendations for how to Improve upon current CSS services, evaluations, and systems.