Skip to Main Content
NY.gov Portal State Agency Listing Search all of NY.gov
Ann Marie T. Sullivan, M.D., Acting Commissioner
Governor Andrew M. Cuomo

Clinic Quality Improvement Initiative
A Partnership Between Clinic Providers and OMH to Improve the Outcome of Our Services

Powerpoint Version | View Adobe Acrobat Version | Download Adobe Acrobat Reader

Slides:
Our Evolving Environment
OMH Vision: A Transformed System
An Overview of the Workshop
What is Quality Improvement?
What is Continuous Quality Improvement?
How Quality Improvement Began
Quality Improvement and Healthcare
Variations on the Approach
The Shared Elements of Continuous Quality Improvement Approaches
Quality Improvement is an Orientation and Attitude
Core Principles of Continuous Quality Improvement
Overview of a CQI Program
Setting Priorities
What is a Project or Initiative?
Shared Core Method of Quality Improvement Approaches
Quality Improvement: Plan
What is a Performance Indicator?
Quality Improvement: Do
Quality Improvement: Check
Quality Improvement: Act
PDCA is a Cycle
Evidence Based Practice
Our Partnership for Quality Memorandum of Agreement
Key Dates
Key Dates New York City
Highlights of MOA
First Year
Second Year
Third Year
Practical Demonstration
The Wide Variety of Possibilities (Three Examples)
Example 1: Relevance to Mission and Clinical Importance
What were the indicators?
North Shore Child and Family Guidance Center Baseline Data
What was the desired result?
Objectives
What is Engagement?
Evidence Based Engagement Interventions
Empirically Supported Telephone Engagement Strategy
How Was it Assessed?
Plan
Do
Check
North Shore Child and Family Guidance Center
North Shore
North Shore’s Experience
Act
Lessons and Challenges
More Information
More Contact Information
Example 2: Mission Relevance and Clinical Importance
What were the indicators?
Rensselaer County Baseline Data
What was the desired result?
What was the model?
Plan
Do
Data
Examples of what services people received other than outpatient therapy
Check
Act
Lessons Learned
Contact Information
Information on the Model
Example 3: Mission Relevance and Clinical Importance
Indicators
Desired Result
Plan: The Model
Stages of Recovery
Substance Abuse Treatment Scale
Four Basic Skills of Clinicians
Plan
What is a Strategy?
The Clinic’s Strategy
Strategy: Learning and Practicing
Do
Do: Starting Point: Assessment
Do:
Check
Act
The Quality Improvement Plan
Quality Improvement Plan Template
Mission/Vision: Scope of Services
Leadership and QI Committee
Goals and Objectives
Things to Consider in Selecting a Performance Indicator
Description of Performance Indicator
Assessment Strategies
Approach or Model Used
Contact Information

As the first decades of the 21st Century unfold, what changes do you expect in the provision of mental health services?

Our Evolving Environment
  • We expect far more definitive research on what is effective in the treatment and rehabilitation of mental disorders
  • Providers will need to be able to accomplish the transition from research to practice
  • Providers will need to use objective tools  to assess client needs as well as the process and outcome of care
OMH Vision: A Transformed System
  • Actively combats stigma
  • Values quality
  • Continuously improves
  • Measures success by measuring individual recovery
  • Adopts evidence based practices
  • Tailors evidence based practice combinations to the needs of individual clients
  • Stresses adequate housing, employment and social integration
An Overview of the Workshop
  • What is Quality Improvement
  • Overview of the MOA
  • Completion of the QI Plan
  • Three Examples of Quality Improvement Initiatives
What is Continuous Quality Improvement?

A quality management model whereby healthcare is seen as a series of processes and a system leading to an outcome. QI strives to make changes in the structural and process components of care to achieve better outcomes.

How Quality Improvement Began
  • Shewhart and Bell Labs
  • Deming and the Japanese Auto Industry
  • Industry in the 1980’s
  • Healthcare in the 1990’s
  • The Enduring Principles
  • Method: Plan, Do, Check, Act
Quality Improvement and Healthcare
  • Added element of the client
  • Passive vs. active: Individuals are empowered
  • Medical Errors
  • Outcome of Care
  • Basing Practice on Evidence
Variations on the Approach
  • CQI  Continuous Quality Improvement
  • TQM Total Quality Management
  • Six Sigma Engineering Origins
  • PDCA  Plan-Do-Check-Act cycle
  • BSC  Balanced Scorecard
Quality Improvement is an Orientation and Attitude
  • We understand our work as processes and systems.
  • We are committed to continuous improvement of processes and systems
Core Principles of Continuous Quality Improvement
  • Customer Focus
  • Recovery Oriented
  • Employee Empowerment
  • Leadership Involvement
  • Data Informed Practice
  • Using Statistical Tools
  • Prevention over Correction
  • Continuous Improvement
  • Participation and Communication at all levels
Overview of a CQI Program
  • Essential Program Aspects
    • Provide a structure through which the core organization functions are evaluated and improved
    • Core functions will be defined by the Mission, Vision, and Values of the organization
      • Examples of core functions
      • Outcomes: client safety, clinical outcomes, client satisfaction
      • Process: client flow, fiscal issues
    • Core functions operationalized for data collection purposes
      • Examples of operationalized functions
      • Outcomes: med errors, suicide attempts, satisfaction survey data
      • Process: wait list latency, no show rates, medication costs
    • Evaluation of the functions achieved through analysis of collected data
    • Improvements accomplished through projects/initiatives

Where do projects and initiatives come from?

  • Internal
    • Unacceptable variation in key indicators
    • Management initiatives
    • Client complaints
    • Incidents
  • External
    • Literature, e.g. Evidenced Based Practices
    • Benchmarking – comparing organizations results to other, like organizations
    • Regulatory agencies, changes in law/standards
  • Internal and external factors will be reviewed by QI Committee (and others – Board of Directors, etc)
  • Projects/initiatives will be started based on results of prioritization process
Setting Priorities
  • Always more improvement opportunities than can be effectively addressed
  • Set Priorities based on:
    • Relevance to mission
    • Clinical Importance: High volume, high risk, problem–prone
    • Expected impact on outcome of care
    • Available resources and cost
What is a Project or Initiative?
  • A planned activity, often involving a group of people, with a specific goal or expected outcome
  • Quality improvement is about doing something based on our priorities
  • Requires a planned and systematic approach
Quality Improvement: Plan
  • Select the project
  • Understand and clarify the process
    • Data
    • Flowcharting
    • Brainstorming
    • Fishbone Diagram
    • Develop a Plan of Action

Plan the action

  • Plan the pilot test of the action
  • Include in the plan a measure of performance
What is a Performance Indicator

A quantitative tool that provides information about the performance of a process

Quality Improvement: Do
  • Collect data
  • Analyze and prioritize
  • Determine most likely solutions
  • Test whether our action really works before we make it a routine part of our daily operations
Quality Improvement: Check
  • At the end of the pilot period, determine whether the action has had the desired effect.
  • Is the modified process stable?
  • Did the process improve?
Quality Improvement: Act

If the action works:

  • Make it part of routine operations
  • Continue to gather data to make sure you are holding the gains

If the action does not work:

  • Return to the Plan stage
  • Use the test to plan a better action
PDCA is a Cycle

It is not about one single dramatic action, but about trying things to see if they work.

Remember, life is a series of experiments.

Evidence Based Practice
  • A special QI method:  Systematically copying a process or system that works better
  • Care of psychiatric disorders is an increasingly research based activity
  • The Challenge:  Transfer of knowledge
  • A formal rather than informal activity
    • Approach fidelity.
    • Objective assessment.
Key Dates
  • November 30, 2005:  MOA due to OMH in order to receive increase retroactive to 4-1-05
  • January 15, 2006:  CQI Plan due to OMH
  • February 1, 2007:  First Annual Evaluation due to OMH
  • February 1, 2008:  Second Annual Evaluation due to OMH
Key Dates New York City
  • November 30, 2005: MOA due to OMH in order to receive increase retroactive to 4-1-05
  • January 15, 2006: CQI Plan due to OMH
  • April 30, 2007: First annual evaluation due to OMH
  • April 30, 2008: Second annual evaluation due to OMH
Highlights of MOA
  • Does not require providers to adopt one particular framework
  • Does require that certain key principles guide any QI effort
  • Requires participation in the implementation of Child and Adult Integrated Reporting System (CAIRS)
    • Creates measurement opportunities
First Year
  • Educate Leadership and Staff on principles of QI, Performance Measurement and Evidenced-Based Practices (EBP)
  • Create an Administrative structure to perform QI activities through the designation of a Quality Improvement Committee
  • Develop a Quality Improvement Plan
  • Implement at least one measure of the process or outcome of care
Second Year
  • Include QI Plan in new employee orientation
  • Develop and use a second performance indicator
  • Document use of data from indicators in decision-making
  • Survey individuals served, families and staff about their perceptions of care
  • Complete an annual QI Evaluation
Third Year

Collaboration with OMH on an evidence-based practice in the area of medication therapy to be determined.

  • Selecting potential indicators
    • Brainstorming
    • Nominal Voting Techniques
The Wide Variety of Possibilities (Three Examples)
  • Improving Engagement of Youth and Families
  • Reducing the Wait List
  • Integrated Treatment (An Evidence Based Practice)
Example 1: Relevance to Mission and Clinical Importance

Improving the Engagement of Youth and Families in Clinic Services

  • Two thirds of children in need of mental health services do not receive services. (US Public Health Service, 1999, 2001)
  • Only 20% of all children in a study sample of clinics in New York State completed the episode of care in which they were enrolled. (Lyons, 1999)
  • No show rates can be as high as 50%. (Lerman and Pottick, 1995)
What were the indicators?

# 1 The show rate for the first intake appointment for all new evaluations of children and adolescents:

  • # who kept first intake appointments
  • # scheduled appointments

# 2 Attendance at any scheduled clinic appointments subsequent to the first kept intake appointment for all new evaluations:

  • # of attended clinic appointments (excluding first kept intake appt.)
  • # scheduled clinic appt. (excluding the first kept intake appt)
North Shore Child and Family Guidance Center Baseline Data
  • 91% of children and families keep intake appointments
  • 66% of children and families attend clinic appointments subsequent to the first intake appointment
What was the desired result?

By implementing an evidence based training program related to improving the engagement rates for clients, the clinic would treat and retain more children for the duration of their episodes of care.

Objectives
  • Improve the show rate for the first intake appointment for all new evaluations of children and adolescents
  • Improve attendance at any scheduled clinic appointments subsequent to the first kept intake appointment for all new evaluations
What is Engagement?
  • A process that begins with an individual being identified as experiencing mental health difficulties and ending with that individual receiving mental health care (Laitinen-Krispiijin et al, 1999, Zawaanswijk et al, 2003)
  • Has been divided into 2 specific steps: initial attendance and ongoing engagement (McKay et al 1996,  1997, 1998). Rates of service engagement can differ at each and warrant specific consideration.
Evidence Based Engagement Interventions
  • Reminders reduced missed appointments by as much as 32% (Kourany et al, 1990, McLean et al 1989, Shivack et al, 1989 and Sullivan)
  • Intensive family-focused telephone engagement intervention associated with 50% increase in initial show rates and a 24% decrease in premature terminations (Szapocznik, 1988, 1997)
Empirically Supported Telephone Engagement Strategy

Focus during the initial telephone intake or appointment call

Relies on an understanding of child, family, community and system level barriers to mental health care

Family-Focused Telephone Engagement Intervention:

  1. Clarify the need for mental health care
  2. Increase caregiver investment and efficacy
  3. Identify attitudes about previous experiences with mental health care and institutions
  4. Problem Solve, Problem Solve, around concrete obstacles to care
How Was it Assessed?
  • Staff received training in EBP engagement techniques Oct 04
  • In Nov 04, agency began collecting data on indicators 1 and 2
  • Data was tracked for 9 months
  • Results compared pre (business as usual in Oct 04) and post (following the engagement intervention)
Plan
  • Agency received training in the evidence based engagement intervention (Oct 04)
  • Following the training, agency identified a quality improvement team, developed a plan to incorporate the EBP into their operations, and a plan to track the effectiveness of the intervention over nine months using the indicators agreed upon
Do
  • In November 2004, agency implemented the engagement strategy and began collecting data
  • Data was collected each month for nine months and analyzed for trends and patterns
Check
  • Course corrections were made when it appeared that the results did not support the original intent
  • North Shore reviewed data and discussed findings in monthly conference calls and quarterly meetings with other clinics participating
North Shore

Performance Indicator #2

Performance Indicator chart
North Shore’s Experience
  • Fewer issues with dropout (Indicator #1). They started high (91%) and maintained  that rate over all
  • The real success came with Indicator #2. They retained a significantly higher rate of clients after the collaborative than before
  • Moved from 66 to 84 percent
Act

Revised process was fully implemented

  • Data collection continues to ensure that the positive results are maintained over time
  • Staff learned from each other
Lessons and Challenges
  • Collect only the data that is tied to the improvement you want to make. (example of what was collected for this in your packets)
  • Keep it simple and Non–Burdensome. (Most clinics collected data by hand)
  • Make sure the findings are communicated and that leadership knows about the QI project. It is part of the overall agency management framework.
  • Don’t take shortcuts. Don’t skip the PDCA.
  • Call your colleagues.
  • Compare results across sites in an agency.
More Information

Contact:

Mary McKay, Ph.D. Professor of  Social Work
Mt. Sinai School of Medicine, Dept. of Psychiatry
MaryMckay@mssm.edu
212–659–8836

Marcia Fazio, Director, External Review Unit
New York State Office of Mental Health
Office of Quality Management
518–474–3619

More Contact Information

North Shore Child and Family Guidance Center

Regina Barros, Director of Clinical Services
Andrew Malekoff, Associate Director
480 Old Westbury Rd.
Roslyn Heights, NY 11577
(516) 626–1971 ext. 330
Example 2: Mission Relevance and Clinical Importance

Reducing the Wait List

  • Rensselaer County Mental Health Department determined that individuals were unable to access its clinic services in a timely manner. (Problem prone and High risk)
  • There was no way to differentiate people who needed counseling from those who needed other types of services. (Mission relevance)
  • Information gathered during the intake call was often inaccurate. (Problem prone) 
  • The result was that consumers were waiting 2-3 months for a clinic appointment. (Problem prone, High risk)
What were the indicators?
  • Number of calls received
  • Number of appointments made
  • Number of individuals who did not show for treatment
  • Wait time
  • Cost per appointment
Rensselaer County Baseline Data
  • 760 calls received
  • 760 appointments made
  • 14 no shows each month
  • 2–3 month wait time
  • $133,000 cost for 760 appointments
What was the desired result?

The county wanted to reduce the wait list, improve the response time and provide services based on what its customers really needed.

What was the model?
  • A team of senior clinicians was formed. They have the confidence to make decisions. 
  • Clinicians were trained to ask the right questions. Who is bothered? What is bothering them? How much and in what way? Client, Referral Source, Clinician?
  • Individuals who called the clinic for help received a call back from a senior clinician immediately.
  • Clinicians spent as much time as necessary over the telephone to determine what the caller needed.
  • Clinicians were prepared to make multiple calls to referral sources if necessary.
  • Clinicians were careful about the language that they used. Instead of saying “What do you expect to get out of treatment”? They asked “Why did you call?”
  • Callers were called back in 2 weeks to ensure that the suggestions they were given were helpful. 
Plan
  • The county decided that the wait time for clinic services was not acceptable
  • Senior staff implemented a model for answering the phone differently when consumers call for services that had proved successful in another part of the county’s organization
Do
  • Senior Clinicians were trained in the model, assigned days of the week and answered all calls for their particular day
  • Calls were tracked on a monthly basis and outcomes were monitored for the year to ensure that the intervention was having the desired effect
Data

Before

  • Calls received = 760
  • # Appts made = 760 (100%)
  • No–Show Rate = 14/mo
  • Wait time to appt = 2–3 mo
  • Cost 175. for 760 appts = $133,000.

After

  • Calls received = 760
  • # Appts made = 427 (56%)
  • No–show Rate = 4/mo
  • Wait time to appt =0 creeping to 1.5 mo
  • Cost $175. for 427 appts = $74,725. Savings to county = $58,275.
Examples of what services people received other than outpatient therapy
  • Transportation
  • Respite services
  • Parenting Classes
  • Flexible Funds to pay for basic needs: clothing, phone
  • Specialty Services: Bereavement, Sexual Offender Treatment
  • Advice
  • Education of the Referral Source (person doesn’t need therapy, they need some other service)
Check
  • Course corrections were implemented when problems were identified. (e.g. Team members were replaced when it appeared their practices were inconsistent with the model.)
Act
  • Successful strategies were reinforced during staff meetings
  • Adherence to new protocols monitored over time
Lessons Learned
  • Not everyone who calls a clinic needs outpatient therapy. 43% of the callers in this county needed basic problems solved.
  • Not all clinicians were cut out for this model and some were removed from the team.
  • It was important to use Sr. Clinicians who were confident making decisions on the spot, speaking to judges, etc. Sr. Clinicians had a 30% appointment rate vs. 60% appointment rate for less experienced staff.
  • Any person who really wanted to come in for therapy was provided an appointment even if the clinician had assessed that it was not necessary.
Contact Information

Rensselaer County Department of Mental Health
Katherine Maciol, Commissioner
Aaron Hoorwitz, Ph.D. Chief Psychologist
Ned Pattison Government Center
Troy, NY 12180
(518) 270-2807

Information on the Model

Developers:
Rensselaer County
Department of Mental Health

(518) 270-2807
Aaron Hoorowitz
Chief Psychologist

Donna Bradbury
Principal Court Consultation Specialist

Sara Thiell
Court Consultation Specialist

Example 3: Mission Relevance and Clinical Importance
  • Integrated Dual Disorder Treatment: The Opportunity
  • A clinic found that a significant percent of their clients were experiencing alcohol / substance use problems along with mental illness. (High volume/Problem prone)
  • Among this group were the most difficult to treat effectively. (Problem Prone)
  • Staff varied in their approach to these clients with no consensus on what was most effective. (High Risk)
  • “Why is my client not getting better?” (Problem prone)
  • OMH identified Integrated Treatment as an “evidence-–based practice” with resources available. (Mission relevant)
Indicators
  • The number of dually diagnosed individuals assessed initially and at 6 and 12 month intervals
  • The number of dually diagnosed individuals progressing through treatment
Desired Result

A significantly improved treatment outcome for dual disorder (mental illness and substance use) clients as measured by changes in stage.

Plan: The Model
  • Treatment is most effective if the same clinician or team helps the client with both substance abuse and mental illness
  • People who recover from substance abuse disorder go through a stepwise process that can be described in stages
  • At different stages, different types of treatment are helpful
Stages of Recovery

Change

  • Precontemplation
  • Contemplation
  • Preparation
  • Action
  • Maintenance

Treatment

  • Engagement
  • Persuasion
  • Persuasion
  • Active Treatment
  • Relapse Prevention
Substance Abuse Treatment Scale
  1. Pre–Engagement
  2. Engagement
  3. Early Persuasion
  4. Late Persuasion
  5. Early Active Treatment
  6. Late Active Treatment
  7. Relapse Prevention
  8. In Remission or Recovery
Four Basic Skills of Clinicians
  • Knowledge of substances of abuse and how they affect mental illnesses
  • Ability to assess substance abuse
  • Motivational counseling for those not ready to acknowledge substance abuse
  • Integrated substance abuse counseling for those motivated to address their problems
Plan
  • Commitment:  We are serious about doing this and will stick with it
    • Leadership and staff
    • Learning, measuring and assessing
  • Fidelity to the practice:  We want to do this right
  • Strategy
  • A starting point and 12 month objectives
What is a Strategy?
  • Avoiding the Britney Spears wedding
  • The larger picture
  • A set of related and planned actions which make a coherent whole designed to accomplish a major goal
The Clinic’s Strategy
  • Obtain and maintain leadership and staff commitment
  • All clinic staff will become competent in the skills of IDDT
  • Clinical staff will carefully review all their clients, further assessing and making a substance use disorder diagnosis as appropriate
  • Clinical staff will rate the stage of treatment of all clients with a substance diagnosis every six months
  • Ratings will be used to assess individual progress and aggregated to guide staff training and assess outcome
  • Staff will complete a self-rating of fidelity to the IDDT approach every six months
Strategy:  Learning and Practicing
  • Staff meet for an hour weekly using the “Integrated Dual Disorders Treatment Workbook” as a resource
  • Focus on developing skills most relevant to the stage of recovery of most clients
  • Later, as indicated by the data, focus on the most difficult stage transitions
  • Bring in outside training resources to focus on treatment challenges
Do
  • Classify all clients who meet criteria for substance abuse or dependence on the Substance Abuse Treatment Scale
  • Develop a profile of clients on the Scale
  • Classify clients at the time of each treatment plan review
  • Track and evaluate progress of individual clients and dual disorder clients as a group
Do: Starting Point: Assessment

Two Stages

  • Identify clients who meet the criteria for substance abuse and dependence
  • Classify clients on the Substance Abuse Treatment Scale
Do:

Initial Rating Database

substance abuse treatment Scale Ratings

Initial Rating of Clinic Patients

Initial Rating of Clinic  Patients

Check
  • Aggregate an initial, 6 month and 12 month rating of all IDDT clients
  • Complete staff training using the IDDT Workbook
  • Analyze stage of treatment of IDDT clients to help focus training needs and issues
  • Analyze progress of IDDT clients as a group
  • Analyze difficult stage transitions

Example of a Client’s Progress

Progress of client 1886

Progress of IDDT Clients

Mean Ratings of IDDT Clients

Percent of Clients Progressing Through Each Stage

Stage Number Counted Percent Progressing
Pre-Engagement 2 100.00%
Engagement 9 66.67%
Early Persuasion 11 63.64%
Late Persuasion 18 33.33%
Early Active Treatment 14 50.00%
Late Active Treatment 10 60.00%
Relapse Prevention 9 44.44%
Act
  • Actions were based on the initial assessment of clients
  • Since most of the patients fell into the stages from engagement to active treatment, training efforts focused on the staff skills relevant to these stages
Quality Improvement Plan Template
  • Meets the requirements of the MOA
  • Optional
  • Sections to be completed
    • Mission, Vision and Scope of services
    • Leadership and QI committee
    • Goals and objectives
    • Selection and description of indicator
    • Assessment strategies
    • Approach or model to be used
Mission/Vision: Scope of Services

Section 1 of the plan

  • Describe program philosophy
  • Provide basic descriptive information including:
    • Description of individuals served
    • Catchment area
    • Type of services
    • Size of the organization
Leadership and QI Committee

Section 2 of the plan

  • The Quality Improvement Committee
    • Membership issues
    • Responsibilities
    • Meeting frequency
  • Critical role of leadership support
  • Sharing of findings with stakeholders
Goals and Objectives

Section 3 of the Plan

  • Long term core goals of any quality improvement program
  • Objectives
    • Related to selected goals
    • Specific to the clinic
    • Measurable
    • Expected completion within 12 months
    • A basis for the annual evaluation
Things to Consider in Selecting a Performance Indicator

Section 4 of the Plan

  • Mission of the Clinic
  • Clinical importance: High Volume, High Risk, Problem Prone
  • Outcome
  • Available resources and cost
Description of Performance Indicator

Section 4 of the Plan

  • A quantitative tool that provides information about the performance of a clinic’s processes, services, functions or outcomes
  • Data collection
  • Assessment frequency
Assessment Strategies

Section 4 of the Plan

  • Describe assessment methods and tools to turn data into information
    • See Appendix A
    • Memory Jogger in packet
Approach or Model Used

Section 5 of the Plan

  • Various models exist
  • PDCA is described
Contact Information

Central and Western New York
Thomas Cheney
(315) 426-3608

Long Island
James Masterson
(631) 761-2077

New York City and Hudson River
Alan McCollom and Ellen Smith
(718) 862-3324