Guidance Document for Child and Family Clinic-Plus

New York State Office of Mental Health
May 2006

View Adobe Acrobat Version | Download Adobe Acrobat Reader

TABLE OF CONTENTS

1. Background on the Need for Transformation of Clinic Treatment
2. Overview of Child and Family Clinic-Plus

3. Appendices

I. Background on the Need for Transformation of Clinic Treatment

In 2003, the President’s New Freedom Commission on Mental Health released its report entitled “Achieving the Promise: Transforming Mental Health Care in America.” The document characterizes the current mental health system as a patchwork-relic, the result of disjointed reforms and policies, with poor access to the tools that children and their families need to succeed.

There is ample national evidence that our past efforts have not led to a transformation of services for children or their families. In what was perhaps the largest epidemiological study of its kind, Kessler et al shows that the average age of onset for serious mental illness in adult populations is fourteen (14), and that both identification of the disorder and initiation of treatment are delayed for approximately six years time. Delayed access to care for youngsters in the 14-20 year old range not only exposes them to the consequences of the disorder, but additionally, results in deferment of developmental milestones, rendering these children vulnerable to serious social, academic, and emotional difficulties during the most formative period in a young life. These findings cry out for a public health approach to identify those in need and to intervene at the earliest possible opportunity.

There are also extremely vulnerable populations that are in need of transformed mental health services, specifically children in foster care and those in the special education system. The nationally renowned Northwest Foster Care Alumni Study, released by Casey Family Programs examined the long term effects of foster care on adults who were now between twenty and thirty-three years of age. This report stated that 54 percent of those formerly served in foster care had an Axis I diagnosis of major mental illness by the time that they reached 24 years of age.  In addition, the rate for individuals formerly within the foster care system diagnosed with Post Traumatic Stress Disorder (PTSD) was double that of returning United States combat veterans. In a New York State Education Department longitudinal study of educational outcomes, children with serious emotional disturbance demonstrated significantly lower scores in every domain studied when compared against all other special education subpopulations. These studies represent the need for target interventions for high-risk, vulnerable populations.

Clinic treatment has been the foundation of the public mental health system for over thirty years. Each year, nearly 100,000 children and families are served in clinic treatment. This presents New York with a unique opportunity to demonstrate the impact that a transformation in State policy, financing and regulation can make.  The structure and financing of the program have remained constant and have not kept pace with findings generated by decades of scientific study in the recognition, diagnosis and treatment of childhood mental illness. Currently, clinic services are very structured, designed to be delivered within an office based setting and require children and families to self-identify.

To effectively address the mental health needs of children and their families in a timely manner, services need to be readily available and provided in a larger variety of settings, like the home.  In order to achieve this shift in service provision, the Office of Mental Health recognizes the need for changes to be made to current clinic service structure and funding to improve access to effective and flexible services. Building on the knowledge that early and effective intervention increases the likelihood of positive outcomes; the Office of Mental Health also recognizes the need to systematically identify childhood mental illness early through screening activities and to improve services by incorporating evidenced-based practices.  Additionally, the President’s New Freedom Commission’s goal to address disparities in mental health services must be considered. These disparities are readily seen through the lenses of culture, race, age and gender.  The opportunity to reduce these disparities in the children’s mental health system is within our grasp. When taken together, these actions are expected to result in the transformation of the children’s mental health system that is currently serving the children and families of New York State.

II. Overview of Child and Family Clinic-Plus

While a child may have been identified by his or her family, friends, teachers or clergy as in need of help, too many are still growing up with untreated mental illness. In spite of our knowing that recovery from mental illness is possible and best when identified early, emotionally disturbed children continue to be undetected and underserved.This barrier is included in a majority of families’ personal stories, as they painfully recount how their child’s situation worsened and they did not know where to go for help.

Numerous research studies document the lack of adequate identification and treatment for children with serious emotional disturbance. Whether you look to the 1999 Report of the Surgeon General on Mental Health or the Kessler et al epidemiological study, the magnitude of undetected mental illness in our children is staggering and the results of mental illness can be devastating.

1. 1 out of 10 children has a serious emotional disturbance. ¹
2. Only 30% of children age 14 and older with emotional disturbance graduate with a standard high school diploma.²
3. Among all disabilities, emotional disturbance was associated with the highest rate of school dropout.
4. Suicide is the 3rd leading cause of death among children and adolescents. ³

Many factors contribute to the lack of early recognition of emotional disturbance in children. The stigma associated with mental illness is a major factor for many families. Parents from various cultural and economic backgrounds differ in the degree to which they identify child behavioral and emotional problems as disturbed, believe that professional intervention will help, or have the ability to seek treatment for their child. As a nation, we also have not accepted the need for targeted efforts to recognize vulnerable children in need of mental health services. Pediatrician and Family Practice Physicians offices, as well as, schools are major settings for the recognition of emotional disturbance in children and adolescents.  Yet trained staff is limited, as are options for referral to specialty care.

Poverty has also been associated with both dropping out of services and shorter lengths of treatment. The relationship between underutilization of mental health services and poverty is especially significant for minority children and families. As a group, more Hispanic/Latino and African American children discontinue mental health services before goals are met than Caucasian children do. Language differences, geographic accessibility and cost are common barriers to treatment.

The majority of children being treated for emotional disturbance in New York receive their care in clinic programs. Services are theoretically-based and provided in office settings.  For example, a mother may present at the clinic with her teenage son, describing a home situation that is in chaos and daily difficulty in navigating the transition from school to home. The therapist in the clinic will meet with the family, complete an assessment and intervene using therapeutic and skill building techniques. All of this will occur in an office. While this approach works for some, many families find an insurmountable gap between what they hear in the clinic visit and what they need to do at home.

While there has been a significant effort to provide services in school and community-based locations, most services are still provided in a clinic-based setting.  In addition, due to the demands for services and pressures to generate revenue, providers have had difficulty keeping up with current evidenced-based research on early recognition and the engagement and treatment of emotionally disturbed children and their families. Child and Family Clinic-Plus calls for more aggressive community education, improved access, training and clinical development in effective treatment models, and short term in-home skill building and support.  

The Office of Mental Health recognizes the need to be more proactive and systematic in detecting emotional disturbance and engaging children and families that have been previously underserved in treatment. This initiative creates the opportunity for OMH licensed clinics to apply for designation as a Child and Family Clinic-Plus. The Clinic-Plus program will be where:

• Young people struggling in the community with unrecognized emotional disturbance will be identified;
• Families will find the staff and services culturally relevant, engaging, and focused on their needs and strengths;
• Treatments that have been shown to work will be readily available,
• Interventions, learning and skill building occur in the natural environments of the child and family.

To further illustrate the processes of Clinic-Plus, Appendix A provides examples of experiences and outcomes possible with and without a Clinic-Plus program.  Consistent with this vision, each Child and Family Clinic-Plus will: collaborate with the local mental health government agency to conduct systematic early recognition activities for the identified priority populations; demonstrate skill in engaging families in treatment; offer a range of evidence-based treatments that are individually determined and family focused; and will provide a constellation of support services in the home and community that lead to skill mastery for the child and family. Each Clinic-Plus will be licensed by the Office of Mental Health as an outpatient clinic and will receive targeted Medicaid and State Aid enhancements.

Child and Family Clinic-Plus providers will provide the following:

• Broad-based screening in natural environments
• Comprehensive assessment
• Expanded clinic capacity
• In-home services
• Evidence Based Treatment

1. Service Components

Community Education
This is the first area of fundamental change for the treatment of children and families. Building upon the public health approach, Child and Family Clinic-Plus providers, in conjunction with the local mental health department, will provide education to increase community awareness of childhood social-emotional growth and development. Targeted sessions will be provided to families, schools and the community-at-large using various tools, such as SPEAK and You Can Ask, to encourage the early recognition and treatment of emotional disturbance in our young people. These dialogues will include specific information on screening for emotional disturbance. The goal of community education is the establishment of a partnership with the community to overcome cultural, economic and stigma related barriers to receiving mental health services. Through community education efforts by the local mental health department and the Clinic-Plus provider, locations for screening activities will be identified and protocols established to implement a community-based public health approach to early identification of childhood emotional disturbance.  Community Education services will also encompass technical consultation around screening, including consent procedures, training and logistics.

Screening
The local mental health department will prioritize the highest need population(s) that will be routinely screened (examples include: Infant Mental Health, Early Intervention, Children or Adolescents in Schools, DSS Preventive Services, Family Practice/General Pediatric Practices). On a regular basis, Child and Family Clinic-Plus programs will screen children in the targeted population to identify those with an emotional disturbance who will benefit from a more comprehensive assessment.

Continuous and systematic identification of childhood emotional disturbance in the general population will ensure early detection of children in need and will promote prompt intervention to ensure achievement of key developmental milestones is not delayed. This will be accomplished through the following screening activities:

• Each program will annually: (1) receive from the local mental health department or identify in conjunction with the department the subpopulation(s) that will be the focus of their early recognition activities; (2) identify how and where screening will be provided; and (3) estimate the annual number of children who will participate in the screening service.
• All screening activities must be tailored to the language and developmental needs of the target population and be sensitive to all cultural variations.
• Each Clinic-Plus program will continuously use valid tools to identify children early who have emotional disturbance.
• The instrument used must have supporting documentation that it is a valid tool. (The material contained in Appendix B provides support to providers in selecting locations for screening, ages for screening and screening tools that have research supporting their effectiveness (OMH Child and Adolescent Early Recognition Matrix.)
• Screening services must always be voluntary and with the active consent of the Parent/Guardian. Active consent will be obtained using the OMH Active Consent for Child and Family Clinic-Plus Early Recognition (See Appendix C).
• Written approval for community-based screening must always be obtained from the administrative entity responsible for the operation of the screening location.
• All staff performing screening services must be qualified (Clinic-Plus Screener Qualifications – See Appendix E) and trained thoroughly in the administration, scoring and interpretation of the instrument that is being used.
• The confidentiality of all participants must be maintained throughout the process of screening as well as for any records generated in screening and subsequent referrals.  Findings of the screening will not be shared with the host site (i.e. school, day care center) without the parent’s prior, written permission.
• A plan must be in place for the immediate assessment and referral of a child who is identified with emergency/acute mental health needs.
• A process for the notification of parents of the results of early detection services must be in place in accordance with the timelines set forth under Guardian/Parental Notification.
• A plan for engaging children identified as in need of a comprehensive assessment and their families must be in place.
• A local telephone number must be made available for families to be able to ask questions about screenings, assessments and services under Clinic-Plus; or to follow up after any service has been received.

Guardian/Parental Notification
The process of parental notification is dependent upon the level of need of their child.  Where screening has indicated a need for a follow-up assessment, the parent must be notified within a week (7 days) by telephone or mail.   If by mail, providers are required to follow-up by telephone to ensure services were pursued and/or all questions about their child’s needs are answered.  Parents of children with no identified need for follow-up should be notified that their child was screened and that no follow-up is required at this time by the end of two weeks (14 days) from the date of the screening.  If the screening does not indicate the need for a comprehensive mental health assessment, but does indicate an area of concern, the letter to parents should include a list of possible resources and referrals that they may contact for help.  On the rare occasion that it becomes evident that a child is in need of emergency services, parents must be contacted immediately to help them access necessary services.  A local telephone number should be provided to parents should they have questions about the screening process or to follow-up after the screening has taken place.  This number should be made available at times convenient for parents and responses to inquiries should be provided by individuals with knowledge of and access to information about the provider’s Clinic-Plus program.

Child and Family Engagement
Throughout the processes of community education and screening, a crucial element for success is the active and continuous engagement of the child and family.  Families are often wary about screening and fearful when a mental health screening indicates the need for treatment. Educating families on the purpose and positive outcomes of screening; supporting and encouraging children identified through screening and their families to follow through with comprehensive assessments; and engaging families in continuing with treatment and support services is all part of the Child and Family Clinic-Plus program.  This can be accomplished through a variety of ways: the development and dissemination of fact sheets, pamphlets and brochures; through follow-up and confirmation telephone calls used to motivate and engage families; and the use of staff that have skills in or are trained on how to engage children and families.  These approaches to family engagement are to be used throughout the screening, assessment, and treatment process to maintain involvement and promote successful outcomes for the family.  Language assistance needs (interpretation and translation) should be taken into consideration during the development of any educational materials or engagement strategies to assure accessibility by all youth and families.

Clinic-Plus sites will be required to develop a plan for child and family engagement based on proven and effective practices. For example, Dr. Mary McKay of the Mount Sinai School of Medicine has developed formal protocols for the implementation of an engagement intervention that is used to improve initial contact with the family and youth to increase retention over time. One application of the engagement intervention, which specifically focuses on inner-city families, targets empirically defined attitudes toward mental health care, while simultaneously helping parents gain confidence in their ability to bring their child to a mental health appointment.

To ensure children and families have received the services they needed and wanted, it is important to qualitatively measure success through child and family satisfaction surveys.  Such surveys are a critical component to an agency’s continual quality improvement planning, as well as, strategic planning for programmatic needs and population trends.  To measure the true impact of clinic transformation, it is important for designated clinics to provide opportunities for children and families to provide feedback on their experiences.  Therefore, Child and Family Clinic-Plus designated clinics, will be required to conduct satisfaction surveys for both youth and their families. 

The Office of Mental Health has developed a “Family Assessment of Care Survey” and a “Youth Assessment of Care Survey,” based on common satisfaction measures utilized by SAMHSA (Substance Abuse and Mental Health Services Administration).  Rather than require ongoing surveys, designated Clinic-Plus providers will be asked to survey all clients who have received treatment services for at least one month’s time during any two consecutive week period during March to April 21, 2007.  Surveys are confidential and will be returned to the Office of Mental Health for evaluation.  The results will be sent back to providers and relevant local mental health departments.  Instructions and guidelines will be provided to Clinic-Plus providers once they have been designated.

2. Effective Treatment Begins with Comprehensive Assessments

Families consistently request more rapid access to clinical services. Widespread attention to evidence based practices raises the challenge from merely providing a “quick” clinic appointment to delivering effective treatment that is integrated with skill and support services available from their first point of contact. Each family brings unique traits, talents and resources.

A comprehensive assessment is a critical step in the process of fully identifying the needs and strengths of a child and their family. All Clinic-Plus programs will be required to conduct a comprehensive assessment prior to admission.  A comprehensive assessment includes: a comprehensive, diagnostic psychiatric formulation, evidenced-based psychometric scale assessments, and a thorough interview with the family to gain insight into their strengths, needs and hopes for the future.  In recognition of this, each Child and Family Clinic-Plus will receive an enhanced rate for up to three (3) pre-admission assessment visits.

Diagnostic Formulation
A comprehensive psychiatric evaluation is necessary to gather all important information relating to the child, their functioning and environment to make the most accurate diagnosis and determine applicable treatment interventions.  To assist with determining what elements are to be included in a comprehensive assessment, the American Academy of Child and Adolescent Psychiatry (AACAP) has developed specific parameters for conducting mental health evaluations.  The parameters are differentiated by Infants and Toddlers, and Children and Adolescents. The most common elements of comprehensive, diagnostic evaluation include:

• Description of behaviors present (i.e., when do the behaviors occur, how long do the behaviors last, what are the conditions in which the behaviors most often occur)
• Description of symptoms noted (physical and psychiatric symptoms)
• Effects of behaviors/symptoms as related to the following:

• Psychiatric interview
• Personal and family history of emotional, behavioral, or developmental disorders
• Complete medical history, including description of child’s overall physical health, list of any other illnesses or conditions present, and any treatments currently being administered

Based on the initial assessments and interviews, additional evaluation may be necessary to accurately diagnose and make appropriate recommendations for treatment.  Further assessments may include:

• Educational history and assessments
• Speech and language assessments
• Laboratory tests, in some cases (may used to determine if an underlying medical condition is present), including the following: 

Evidenced-based Psychometric Assessments
As with all Clinic-Plus interventions, use of evidenced-based practices and treatments are critical to ensure efficacy and positive outcomes for children and their families.  In addition to developing parameters for conducting mental health evaluations, the AACAP also recommends a range of valid psychometric rating scales that aid in the diagnostic process for children and adolescents. For example, these may include such scales as: the Beck Depression Scale, Connors Rating Scale (ADHD) and the Child Behavior Checklist.  More information about the AACAP practice parameters can be found at http://www.aacap.org/clinical/parameters/index.htm.

Another example of an evidenced-based assessment tool is the (Voice) Diagnostic Interview Schedule for Children (V-DISC).  The V-DISC is a computer operated, self administered, voice-activated assessment tool that is used to identify youth at risk for psychiatric conditions.  Due to the confidential and self-reporting nature of its application, it has proven to be highly reliable and accurate. 

Comprehensive Interview
The interview with family members not only provides needed context for consideration of the scientific elements of the assessment, but lends critical insight into the families’ strengths, hopes and needs for the future. Through a comprehensive interview, therapists/intake workers will gain important knowledge about the family’s assets, available support systems, culture and resources.  This awareness will help them to develop recommendations that speak to the uniqueness of the family and address their strengths, hopes, and needs in a family-centered manner. The interview is also a fundamental opportunity to engage and motivate families into seeking help.  It is a time for developing rapport and reinforcing hope for families who may be skeptical of treatment or treatment providers.

Treatment Recommendations and Referrals
Based on a comprehensive diagnostic evaluation, therapists will develop a clinical formulation and differential diagnosis that will be used to determine intervention or treatment recommendations.  Interventions will vary based on the uniqueness of the family and may include a broad range of services from mental health treatment or referrals to non-mental health service providers as dictated by the assessment.

It is important for the Clinic-Plus provider to be able to support and advise the family through any and all treatment recommendations or outside referrals.  A thorough knowledge of all community resources and services is necessary for making appropriate referrals.  Clinic-Plus providers must be able to provide accurate contact information with an address and telephone number of any referral source that is made.  If a family chooses to receive mental health treatment with another provider, written consent of the family is required to transfer assessment reports.

For reporting purposes, Clinic-Plus providers are required to complete “Minimum Data Set Client Level Measures” for ALL children receiving a comprehensive assessment.  This data set can be completed through the Child and Adult Integrated Reporting System (CAIRS) or through a direct data transfer (see Appendix G) to the Office of Mental Health.

Admissions
Upon admission to a Child and Family Clinic-Plus program, providers must complete a Child and Adolescent Needs and Strengths (CANS) survey, which will be used as a measure comparing the client’s pre-intervention status with their status upon completion of services.  The individual client’s progress and outcomes will be identified as a measure of success.  As a planning tool, the CANS is helpful in developing an individualized treatment plan for clients to ensure their needs are addressed, while also recognizing the child’s and family’s strengths.

Persons completing the CANS must be certified to use the tool.  Individuals can become certified in one of two ways: (1) complete training led by a Certified CANS Trainer, or (2) complete the web-based CANS training on-line through the Office of Mental Health webpage (Link to be announced).  A list of CANS trainers is available in Appendix F.  To make arrangements for training, providers should contact the trainers listed directly.

3. Supporting Children and Their Families Through In-Home Visits

Treatment within the Child and Family Clinic-Plus program is focused on the current, well-defined targets identified in the comprehensive assessment. The Clinic-Plus program recognizes the clinical significance of in-home, hands-on learning to children and their families. Support, redirection and positive reinforcement are basic elements of behavioral change and skill acquisition. Within Clinic-Plus, providers are encouraged to increase their provision of in-home services by receiving an enhanced rate of reimbursement for up to nine (9) visits per year with the child and family in their home/community.

Definitions related to In-Home Visits:

Duration:

at least 50 minutes of direct service to the child/family

Location:

the home of the child/family or other relevant sites where the child/family regularly interact.

Frequency/Rate:

There may be up to 9 in-home visits per year, per child reimbursed at an enhanced rate

Staff:

Professional Mental Health Staff

Activities:

Application of therapeutic approaches expressed in the goals of the child treatment plan. Examples could include: Facilitation of communication within the family or with key community resources, skill building, solution-focused role modeling, and pragmatic family therapy.

The intent of in-home visits is to ensure that clinical interventions work within the real-life environment of the child and family. Interventions and skills are reassessed continuously for effectiveness.  Assessment of caregiver strengths measured in the CANS (Child and Adolescent Needs and Strengths) survey can be used to help clinicians formulate their plans for skill building and modeling with the family.  The goal of in-home visits is to facilitate the development of insight and the creation of skills that can be generalized across various contexts.

In-home visits require that staff going into the home are well trained and experienced in engaging families and working with their strengths to support skill building.  Key competencies for staff performing in-home visits include:

• Parent engagement skills
• Expertise in skill development and teaching  
• Strengths-based redirection
• Cultural competence
• Knowledge of community resources
• Knowledge of safety training

• Promoting Efficacy Through Evidenced-based Practices

Evidenced-based Treatment Interventions
Mental health services research has demonstrated that some specific treatment approaches are effective in improving outcomes for individuals diagnosed with serious mental illness. Called evidence-based practices, these interventions are rooted in reliable scientific inquiry, and supported by a body of evidence.  They have demonstrated effectiveness in improving outcomes in areas relating to wellness (e.g., physical health, self-esteem, symptom management, and behavior management) and community integration (e.g., in home, in school, involved in extra-curricular activities). Adherence to specific population, outcome, and implementation standards is essential to producing positive outcomes consistent with research.

Child and Family Clinic-Plus programs will see families through the treatment phase with clinical interventions that are proven to be effective. The OMH Evidence-Based Treatment and Diagnosis Matrix(Appendix D) provides each program with a crosswalk for the selection of treatment approaches based upon diagnosis.  This will enable providers to choose the most appropriate intervention based on the individual diagnosis of the child that is research-based and proven to be effective. The use of evidenced-based practices is critical to ensuring positive outcomes for children and their families.

Cultural Competence activities need to be imbedded within all stages of development, implementation and evaluation of evidence-based practices. Readiness for implementation needs to include skill development and policy guidance to ensure clinical and administrative practices are responsive to the diversity of the population served.

Professional Development Practices
In order to implement the use of evidenced-based practices in a Clinic-Plus environment, agencies must provide their staff with opportunities for professional development.  Professional development activities include the following: formal clinical trainings, workshops, in-service trainings, and regular clinical supervision and case reviews.  Staff competency in evidence based practices should be regularly monitored by the agency.  In the Clinic-Plus Annual Plan, each program will identify their professional development strategies for the ensuing year and how they will utilize evidenced-based practices in their provision of treatment approaches.  Demonstration of professional development and staff competency can be determined by:

• Clinical documentation in the client record regarding diagnosis and evidenced-based treatment approach to be utilized.
• Documented completion of formalized training in specific evidence based practices.
• A professional development program that includes an annual plan and documentation of staff participation.

One opportunity clinics will have for accessing training for staff in evidence-based practices is through the Evidenced-Based Treatment Dissemination Center (EBTDC).  The EBTDC was developed to provide a sustained clinical training model in evidence-based treatment protocols and in specialized consultation to support the organizational changes necessary to transform the way in which mental health services are delivered. This initiative will ensure that scientifically proven treatment approaches are available to front line clinicians on a statewide basis.

            2. Financing of Child and Family Clinic-Plus

Child and Family Clinic-Plus is an initiative that is intended to transform the delivery of clinic treatment services throughout New York State. The critical task for planning this initiative is to balance the need for standardization in a model with an individual County or community’s ability to address the varying needs of subpopulations of children.  For example, a general screening of high school students will yield a smaller percentage of youth in need of assessment than a general screening of high risk children in foster care.  To address both of these imperatives, the Office of Mental Health has established a series of uniform program components and financial assumptions for reimbursement rates, admission rates and utilization of model service components.  These elements define the standard Clinic-Plus program and are the basis for the Clinic-Plus Calculator.

The Office of Mental Health is developing an easy-to-use, web-based tool that will help each local mental health department/provider see the financial impact of the programmatic vision that they have created for their local Child and Family Clinic-Plus program(s). The program elements of Child and Family Clinic-Plus are supported through a combination of State Aid and Medicaid enhancements as follows:

Designating Child and Family Clinic-Plus Programs
Each local mental health department will identify the number of Clinic-Plus programs that will be designated in their area.  The number and size of the Clinic-Plus program will drive the financial model that will help to determine: numbers of screenings, numbers of comprehensive assessments, the anticipated increase in clinic admissions capacity, and in-home services.  Generally, screening goals (initial number of screenings allocated) for each county/borough are based on local population data; the larger the county, the more screenings they will be able to conduct.  For the purposes of Clinic-Plus designation and funding distribution, screenings are defined in terms of “units.”  One unit of screening is approximately 1,000 screenings.  The minimum size for a Clinic- Plus program will be 1 screening unit.    

3. Annual Plans and Performance Contracting

Annual Plan
Each year, Child and Family Clinic-Plus providers will be required to submit an Annual Plan to their local mental health department.  The mental health department will combine all annual plans for a county-wide submission to the NYS Office of Mental Health.  The annual plan will detail providers’ intentions for implementing Clinic-Plus services from screening to in-home services.  Annual Plans will include the following elements:

• Target population to be screened;
• Community Education Plan and Community Education;
• Plan for addressing language and sensory needs of children and families;
• Screening instrument(s) that will be used;
• Annual Projected # Screened;
• Screening plan (personnel, location, coordination with other Clinic-Plus providers, etc.);
• Child and Family Engagement Strategies;
• Annual Projected # of Comprehensive Assessments (Medicaid and Non-Medicaid);
• Comprehensive Assessment plan (staff conducting assessments, approaches, etc.);
• Diagnostic and psychometric assessments to be used;
• Annual Projected Increase in Admissions;
• Plan for ensuring cultural congruence;
• Plan for use of Evidenced-Based Treatments;
• Staff Development Plan;
• Annual Projected In-home services (Medicaid and Non-Medicaid);
• Plan for providing In-home services; and
• Budget and budget narrative.

Annual plans should not only include anticipated data elements, but describe how Clinic-Plus will be implemented and how providers plan to uphold the values and treatment expectations of a Child and Family Clinic-Plus Program.  Children’s mental health system values upheld by the Child and Adolescent Service System Program and OMH can be found in Appendix I.

Performance Contracting
Providers performing within the lowest 25th percentile of third-party allocation or non-Medicaid units of service, compared to the performance of their peers, will be subject to the review and potential recovery of State Aid.  This will occur on an annual basis.

4. Reporting Requirements

Child and Family Clinic-Plus providers will be required to report program level data to their Local Mental Health Department Director and to the Office of Mental Health (OMH) on a quarterly basis.  Providers will provide aggregate data on all their screening activities, comprehensive assessments, Clinic-Plus utilization, and in-home services through a Quarterly Service Report.  Program service reporting requirements are outlined in Appendix G. 

In addition to the Quarterly Service Report, providers will also report their quarterly utilization of Medicaid and non-Medicaid services to OMH on a Quarterly Utilization Report;a modified version of the OMH Fiscal Worksheet.  These reports will enable the OMH to track clinic performance and the provider’s congruence with their Annual Plan.

A Minimum Data Set for client level data has also been established, which will be completed for all children receiving a Comprehensive Assessment and can be entered via the Child and Adult Integrated Reporting System (CAIRS) or through data transfer directly to OMH. Detailed examples of the data elements that will be required reporting are also available in Appendix G.  All planning, fiscal, and data reporting requirements are outlined in the grid below:

All Clinic-Plus providers must currently be a part of or apply for the Clinic Quality Improvement Initiative under the OMH.  Additional rate enhancements associated with last year’s CQI Initiative are available for participating agencies.  Every Child and Family Based Clinic- Plus must agree to improve quality and express this commitment through the development and implementation of a continuous quality improvement program in accordance with guidelines established by the OMH.  Each clinic must collaboratively develop a process to systematically monitor, analyze, and improve its performance in assisting children and families in meeting their treatment outcomes.  This will include the development of a Quality Improvement Plan consistent with the mission and values of Clinic- Plus. 

For those clinics already enrolled in the OMH CQI Clinic initiative, year two and three deliverables shall clearly reflect Clinic- Plus involvement, i.e., the selection of indicators in accordance with Clinic- Plus values, the collection of satisfaction data relevant to Clinic- Plus principles, and the annual evaluation reflective of Clinic- Plus objectives.

Appendix A:  Case Examples

Case #1:  John - Clinic-Plus Screening- Assessment-Referral to School Program:

The Radiant Mental Health Clinic has an agreement with the Eagle School District to perform mental health screenings in each of their buildings.  Prior to implementing a screening, the Coordinator obtains class lists and home addresses from the school.  Each family is sent information about the screening at least two weeks in advance of the screen date.  Children will only be screened if the parent has signed and returned the consent form.  John is a 12 year old boy who attends Main Street School.  His mother provided consent for him to be screened by the Radiant Mental Health Clinic. On Monday, a screener from Radiant Mental Health screened John’s homeroom.  The screening occurred at approximately 8:00 am and there were 25 children included.  The screening responses were reviewed and scored by the screener by 10:30 am.  Three children were identified as in need of comprehensive assessment.  The screener then contacted John’s Mother and arranged a time for them to come to the Clinic for an assessment.

On Wednesday, John and his Mother arrived at the clinic for the first time.  They started off meeting together with a social worker named Nick.  Nick got to know John and his mother and explained the comprehensive assessment process that the clinic offered. There was a very noticeable tension between John and his Mom; however, there were no signs of psychosis or serious emotional disturbance.  In talking with John, Nick discovered that his parents had recently divorced and that he was very angry about this event.  He shared that he didn't feel like coming to school and didn't want to talk about it.  John’s Mother signed a consent form to allow the Clinic-Plus program to contact the school and his pediatrician for relevant records. A second appointment was made within two weeks.

At the second appointment, John and his mother met again with Nick (social worker).  John seemed more willing to talk at this visit and the tension between he and his Mother appeared to have lessened.  The school records showed that Nick was a very consistent B student until the last semester.  He was now at risk of failing 2 subjects.  His medical records did not reveal any significant findings. Nick worked with John to complete the Beck Depression Scale and the Parent/Child Conflict Scale.  His scores on both of these tools did not indicate the need for treatment. Based upon his social history and assessment, Nick did not find any prior psychiatric history, psychosis or functional impairment of significant duration.  Nick spoke with John about options that he had for talking with people who could help him to process his parent’s divorce.  John agreed that he would like to try a group at school.  In working with both John and his Mother, it was agreed that John would be referred to the Skills Group at the Main Street School.  His Mother also requested that a copy of the assessment and recommendations be sent to their pediatrician. Nick contacted the school and arranged for John to join the Skills the group within one week.

Case #2:  Sally - Clinic-Plus Screening – Assessment - Admission

Sally is a 17 year old junior attending the Elmwood High School.  The Radiant Mental Health Clinic screens students at this school every year with the active consent of their parents.  Parents are notified by mail at least two weeks prior to the screen date.  They receive information on the screening, mental health in children and a consent form.  Children must have the written consent of their parent/legal guardian in order to be screened.

 During a screening this week, Sally was identified as needing a comprehensive assessment.
The screener was particularly concerned about Sally, because she had noticed a level of agitation in her as she completed the screening.  It was difficult to reach Sally’s Mother to set up the appointment for the visit.  Finally, at 7:00 pm that night, the screener spoke with her Mother and was able to arrange an appointment the next day.

The screener had notified the Clinic staff that she was concerned about Sally’s level of agitation and had recommended that the Psychiatrist participate in the first visit.  Upon arriving, Sally met with the Psychiatrist and her mother met with a Social Worker.

A great deal of information was obtained at this first meeting, including Sally’s history of a suicide attempt when she was 13 that led to a psychiatric inpatient stay that same year.  Sally had not received treatment since that time. Her relationship with her mother was poor.  Her mom reported that “her room is a mess and I can’t get her to do anything!  I’m tired of her yelling at me”. Sally shared that she had recently stopped socializing with several friends that she had been close to for some time.   Several psychometric tools were used by the Clinic team.  These included the Child Behavioral Checklist, the Beck Depression Scale and the Child and Adolescent Needs and Strengths Survey (CANS).  The findings revealed that Sally was experiencing a significant depression and would require treatment.  Based upon the Psychiatrist and Social Worker’s initial assessment, it was determined that Sally was not an imminent risk, but did require a comprehensive assessment within the next week.  Sally and her mother agreed to participate in this process.

A comprehensive assessment was completed over the course of an additional 2 visits to the Radiant Mental Health Clinic.  During the assessment it was discovered that Sally had been a victim of child sexual abuse at the age of 10.  She was diagnosed with Depression and was admitted to the Clinic for treatment.  During the assessment, the team saw many strengths in Sally as well.  In particular, she enjoyed working with animals and was viewed by many to have a very strong singing voice.

At the first treatment visit, Sally met with her therapist, Michelle who explained that her treatment would last about 10 sessions and that it would involve visits to the clinic and visits in her home to help her practice the skills and strategies that they would discuss.  Sally had “never heard of people coming into houses for this stuff.”  She agreed after a long discussion to give it a try. Michelle also met with Sally’s mom to talk with her about the treatment.  Unlike Sally, her mother was delighted that she would have someone coming into the home to help her.

Sally began a course of medication and Cognitive Behavioral Therapy (CBT) at the clinic.  She was seen weekly at first to titrate her medication and to begin the CBT sessions.  After 3 visits and beginning signs of improvement, Michelle came to the home in the afternoon to work with Sally and her mother.  The focus of these visits became identifying constructive ways for Sally and her mother to deal with conflict.  Michelle guided Sally in establishing a “comfort area” in her room and worked with her Mom to develop more consistent boundaries and expectations for her daughter. 

While driving to the home one day, the Michelle noticed that Sally lived within walking distance of a large Pet Store. She walked with Sally to the Pet Store while getting the update on how she and her Mother were doing with their new communication skills.  Sally commented that she had always wanted to work with animals and that working in a place like this would be great.  Michelle asked the mother about her thoughts about Sally working at the Pet Store.  Her mother was very excited about the possibility, but wasn’t sure that Sally could handle it.  Michelle asked Sally to talk about the possibility of a job with her therapist and her mother to work with Sally to prepare a time schedule that would include time for school, homework and 10 hours of work per week. When Michelle met with Sally and her mother again, the next week, they had prepared a schedule and had even completed a job application at the store.

After 10 clinic visits (CBT series) and 4 in-home visits, Sally’s depression was fully treated.  She was referred to a Sexual Abuse Survivors group in the community for support and to her Family Practice Physician for medication management.

Case #3: Sally – without Screening at her High School

Sally is a twenty-one year old young lady who was admitted to the West-End Day Treatment Program this past week.  When she was 10 years old, Sally was sexually abused by a neighbor.  At the age of 13, Sally attempted suicide by ingesting a bottle of Advil and was hospitalized at the local psychiatric center for 3 weeks.  Upon discharge Sally and her Mother were referred to the Radiant Clinic.  Sally’s Mother never followed up on the referral and she did not receive any mental health services for the next 7 years.

Sally progressed through middle school as an average student.  Of note, she would miss 30+ days of school per year and spent most of her free time alone in her room.  As she entered the Elmwood High School, her grades began to slip.  She barely passed the 9th grade and began to experience pressure from her parents to “try harder”.  Sally school attendance continued to be sporadic.  When her parents met with the school, they commented that Sally was a nice, quiet girl who was very smart, but need to apply herself better.

In her junior year, Sally began to use alcohol and over the counter cold medications to “self-medicate”.  For a brief period of time, Sally was able to improve her grades and was happy to find her parents pressuring her less.  In the spring of that year, Sally’s academic performance plummeted.  She was failing every subject and her mother discovered 12 open packages of Pseudophedrine and alcohol bottles in her room.  Sally’s parents, frustrated by the many years of struggling, told Sally that she would have to find another place to live.

At age 17, she dropped out of High School, lived with various friends and cycled through a variety of minimum wage jobs for the next two years.  At age 19, Sally had a second, very severe suicide attempt.  She was hospitalized medically for three weeks and subsequently transferred to the local Psychiatric Center for the next 8 months.  She has a diagnosis of Severe Depression and PTSD. 

Appendix B:  Guidance for Screening Children and Adolescents

Knowledge on screening tools performance is extensive; however, our knowledge of their use in practice is limited. The information contained here is a starting point that can be used to guide decision making about where and how you will conduct screening for children and adolescents.  OMH has developed a table outlining the advantages, special considerations and tested screening tools by the chosen screening location (Table 1).  Further detail regarding screening instruments is provided in Table 2. 

Additionally, you may consider referencing an interactive resource on mental health screening that can be found at the Massachusetts General Hospital, School Psychiatry Program and MADI Resource Center at http://www.massgeneral.org/madiresourcecenter/schoolpsychiatry/checklists_table.asp.  This site also provides information regarding diagnostic specific screening tools at
http://www.massgeneral.org/madiresourcecenter/schoolpsychiatry/screeningtools_table.asp

EXCERPT FROM WEBSITE:
Source: Massachusetts General Hospital School Psychiatry Program and MADI Resource Center Website www.massgeneral.org
To help you decide whether a screening tool or rating scale might be appropriate to use with respect to a particular child (or group of children), you can click on the DETAIL link next to the tool or scale. The DETAIL pages give more detailed information about the tool or scale, including a color-coded summary of who the instrument is designed for (i.e., parents, teachers, students, and/or clinicians). The DETAIL pages also provide direct links to view, download, or order the tools and scales.
Cautions – please keep in mind the following cautions:

• Use of the screening tools and rating scales does not produce a diagnosis. Rather, the tools identify those who are in need of a more comprehensive evaluation.
• A particular “score” does not mean that a child has a particular disorder – these screening tools and rating scales are only one component of an evaluation.
• Diagnoses should be made only by a trained clinician after a thorough evaluation.
• Symptoms suggestive of suicidal or harmful behaviors warrant immediate attention by a trained clinician

Appendix B: Table 1 - OMH Child and Adolescent Early Recognition Matrix (Screening Tools described in Table 2)

Appendix C:  OMH Active Consent for Child and Family Clinic-Plus Early Recognition

Date: ____________

Dear Parent/Legal Guardian,

------------------School is participating in a new Screening Program to promote early identification of emotional health issues in children and adolescents.  The purpose of this letter is to ask your permission for your child to voluntarily participate in an emotional health screening.  This is a confidential service for you and your family.  You will be the only ones to receive information from the screening.  No information will be shared with the school or other agency without your written consent.

Why is Screening Important?
Emotional health issues can affect how well a child does in school, family relationships, and ability to make friends.  By identifying emotional issues early, this program can help you and your child get the support you may need.   Early intervention can:

• Keep problems from affecting emotional, intellectual or physical development;
• Keep problems from lasting as long or from getting worse; 
• Improve school performance and personal relationships.

What is the Screening Program?
The free, confidential screening is conducted by a mental health professional trained to carefully assess your child’s emotional health.  Here are some answers to commonly asked questions:

What is the purpose of the screening?
The screening will help to decide if your child may have an emotional health issue that would benefit from a further, more thorough assessment. 

When and how long will the screening take place?
The screening will be conducted during regular school hours.  It is expected to take no more than a half hour. 

Does my child have a choice to take the screening or not?
Your child’s participation is voluntary.  He or she may decide not to answer any question asked.  He or she may withdraw from the screening at any time.  

How will the screening be provided?
The screening will be provided by… [Describe how the student takes the test – computer? oral? pen and paper?]

What will happen to the results of the screening? 
The results of the screening are confidential and will only go to the screener (who works for a local mental health agency and you).  The results will NOT be shared with the school or anyone else without your prior, written consent.  You will be notified about the results of the screen and if your child is recommended for comprehensive assessment or services.  If there is no recommendation for further evaluation or services, you will be notified by mail. 

If you would like your child to participate in the screening program, please sign and return one copy of this form by ___________________ to:  ________________.

If you do not want your child to participate in the screening program, simply do not return this form and your child will not be screened.  Your decision about whether or not to have your child participate in this screening will not affect any other aspect of his or her school program.  

            Thank you very much.  If you have any questions about the screening or other available services, please feel free to call _____________ at any time.

Sincerely,

Name

CONSENT FORM

I, _______________________, give permission for my child _____________________
            (Print Your Name)                                                             (Print Child’s Name)

to participate in _______________’s Screening Program.

Appendix D:  OMH Evidence-Based Treatment and Diagnosis Matrix

Appendix E: Minimum Qualifications for Child and Family Clinic-Plus Screener

Education

1. A bachelor’s degree in one of the below listed fields* or
2. A NYS teacher’s certificate for which a bachelor’s degree is required; or
3. NYC licensure and registration as a Registered Nurse and a bachelor’s degree

AND

Experience

Four years of experience

1. In providing direct services to children with emotional disturbance and their families; or

A master’s degree in one of the below listed fields* may be substituted for two years of experience.

 *Qualifying education includes degrees featuring a major or concentration in social work, psychology, nursing, rehabilitation, education, occupational therapy, physical therapy, recreation or recreation therapy, counseling, community mental health, child and family studies, sociology, speech and hearing.

Appendix F: Child and Adolescent Needs and Strengths Survey (CANS) Trainer List

An updated CANS Trainer List was not available at the time of this publication.  A current CANS Trainer List will be made available shortly.

Appendix G:  Service Reporting Requirements

Quarterly Reporting Requirements:

Outreach

• # of consent letters distributed
• # of consent letters received
• Average length of time between consent letter distribution and screening

Screening:

• # children screened by age group and location of screening
• # children identified to need a comprehensive assessment by age group and location of screening
• Average length of time between screening and comprehensive assessments
• Average waitlist for comprehensive assessments

Comprehensive Assessments:

• # of children receiving comprehensive assessments by age group
• # of children receiving referral by type (to school based MH provider, primary care provider, Clinic-Plus program, Clinic-Plus home based services, others(?) by age group
• Average length of time between comprehensive assessments and intake/admission
• Average waitlist for intake/first appointment

Clinic-Plus Service Use

• # of children entering Clinic-Plus Treatment by age group
• # of no shows to first appointment
• # of children discharged from Clinic-Plus Treatment by age group
• # of Clinic-Plus visits
• # of in home visits

Minimum Data Set Client Level Measures
(To be completed for all children receiving a Comprehensive Assessment)
Last Name
First Name
Middle Initial
Date of Birth
Social Security Number
Medicaid Number
Residence Address Street
Residence Address City
Residences Address Zip Code
Residence Address County
School District
Gender
Residence type
Household composition
Primary language
Hispanic Ethnicity
Race
Educational placement
Committee on Special Education classification
Current Legal Status
Diagnosis Axis I
Diagnosis Axis II
Diagnosis Axis III
Diagnosis Axis IV
Diagnosis Axis V (GAF)
CANS-MH scale
Date of screening
Place screening occurred
Date(s) of assessment (up to 3 dates can be supplied since the assessment may take more than one session to complete)
Result of assessment – Referral offered (Yes/No), if yes, type of referral
If referred to Clinic-Plus:
            date of admission
            Are home based service part of treatment plan?

Follow-up Measures
To be submitted every 6 months and at discharge from the Clinic-Plus program
Record type (discharge or 6 month follow-up)
Date follow-up data submitted or discharge date
Last Name
First Name
Middle Initial
Date of Birth
Social Security Number
Medicaid Number
Residence type
Household composition
Educational placement
Committee on Special Education classification
CANS-MH scale
Date data submitted or discharge date
ONLY AT DISCHARGE:  Was child referred to another service?  If yes, what?

Appendix H: Clinic-Plus Quality Improvement Initiative

NYS OMH Clinic-Plus Quality Improvement Initiative

OMH is committed to the ongoing improvement of quality services in the public mental health system, with the goal of enhancing recovery outcomes for recipients.   The fundamental concepts underlying OMH’s strategic plan are continuous quality improvement and performance monitoring that incorporate the following principles: 

• Customer Focus.  High quality organizations focus on their internal and external customers, and meeting or exceeding customers’ needs.
• Employee Empowerment.  Effective programs involve people at all levels of the organization in improving quality.
• Leadership Involvement.  Strong leadership, direction and support of QI activities by the governing body and CEO are key to performance improvement.  This involvement of organizational leadership assures that QI initiatives are consistent with the provider’s mission and/or strategic plan.
• Data Informed Practice.  Successful QI processes create feedback loops, using data to inform practice and measure results.  Fact-based decisions are likely to be correct decisions
• Statistical Tools.  For continuous improvement of care, tools, and methods are needed that foster knowledge and understanding.  CQI organizations use a defined set of analytic tools such as run charts, cause and effect diagrams, flowcharts, pareto charts, histograms, and control charts to turn data into information.
• Prevention over Inspection.  QI entities seek to design good processes to achieve excellent outcomes rather than fix processes after the fact.   
• Continuous improvement.  Processes must be continually reviewed and improved.  Small incremental changes do make an impact, and the providers can almost always find an opportunity to improve performance.

A Quality Improvement Plan serves as the foundation for the clinic’s commitment to the ongoing improvement of quality services.  It provides the necessary infrastructure to support their quality improvement program and documents their organization’s commitment to quality improvement. The OMH is committed to the ongoing improvement in outcomes of care within its system of services.  As such, Clinic- Plus providers agree to continuously strive to ensure that:

1. services are provided consistent with the best evidence available
2. services are appropriate to individual needs and available when needed
3. risks to individuals served are minimized and errors in the delivery of services prevented
4. services are provided in a timely and efficient manner with appropriate coordination and continuity of care
5. rights are respected

Appendix I:  Mental Health System Values

Child and Adolescent Service System Program Core Principles

CASSP is based on a well-defined set of principles for mental health services for children and adolescents with or at risk of developing severe emotional disorders and their families. These principles are summarized in six core statements.

Child-centered : Services are planned to meet the individual needs of the child, rather than to fit the child into an existing service. Services consider the child's family and community contexts, are developmentally appropriate and child-specific, and build on the strengths of the child and family to meet the mental health, social and physical needs of the child.

Family-focused : The family is the primary support system for the child and it is important to help empower the family to advocate for themselves. The family participates as a full partner in all stages of the decision-making and treatment planning process including implementation, monitoring and evaluation. A family may include biological, adoptive and foster parents, siblings, grandparents, other relatives, and other adults who are committed to the child. The development of mental health policy at state and local levels includes family representation.

Community-based : Whenever possible, services are delivered in the child's home community, drawing on formal and informal resources to promote the child's successful participation in the community. Community resources include not only mental health professionals and provider agencies, but also social, religious, cultural organizations and other natural community support networks.

Multi-system : Services are planned in collaboration with all the child-serving systems involved in the child's life. Representatives from all these systems and the family collaborate to define the goals for the child, develop a service plan, develop the necessary resources to implement the plan, provide appropriate support to the child and family, and evaluate progress.

Culturally competent : Culture determines our worldview and provides a general design for living and patterns for interpreting reality that are reflected in our behavior. Therefore, services that are culturally competent are provided by individuals who have the skills to recognize and respect the behavior, ideas, attitudes, values, beliefs, customs, language, rituals, ceremonies and practices characteristic of a particular group of people.

Least restrictive/least intrusive : Services take place in settings that are the most appropriate and natural for the child and family and are the least restrictive and intrusive available to meet the needs of the child and family.

Interagency Service Planning and a System of Care Framework :  A comprehensive and effective system of care recognizes that children and adolescents with severe emotional disturbances often require services from more than one system. For example, a child with a mental health issue is usually in school, and may also be receiving services from the child welfare, juvenile justice, or health care system due to other difficult behaviors. Planning takes into account the strengths of the child and family and these multiple needs and involves different agencies. When a child or adolescent is identified as having mental health needs and requires the services of other systems as well, a team is convened to discuss the options for treatment, care and support. The team consists of all the key players in the child or adolescent's life, including family members and professionals from all of the child-serving systems involved.

What Might a Transformed Public Mental Health System Look Like?

A Transformed Mental Health System would actively combat societal stigma, value quality, embrace the use of information for continuous improvement, measure success using client-specific recovery outcomes, develop organizational structures to support the delivery of evidenced-based practices by a competent staff, offer science-based practices in combination, based on individual need to produce the best results. This vision would also stress adequate housing, employment and social integration. In other words:

• A transformed system will aggressively work to eliminate stigma associated with mental illness.
• In a transformed system, people will seek mental health care when they need it.
• A transformed system ensures that services and supports actively facilitate recovery and build resilience to face life’s challenges.
• In a transformed system, a diagnosis of serious mental illness or serious emotional disturbance will trigger a well-planned, coordinated array of services and treatments.
• All components of a transformed system will have a cultural understanding of the individuals being served.
• A transformed system is defined by respect, compassion and a collaborative partnership with the people it serves.
• In a transformed system, everyone gets the best available services and outcomes, regardless of race, gender, culture or geographic location.
• A transformed system will provide effective treatments and services that are easy to navigate and that use flexible funding streams.
• A transformed system will require an integrated technology and communications infrastructure.
• A transformed system will address the various characteristics and unique needs inherent to a community or locality.
• A transformed system provides treatment and services that work and result in recovery.
• In a transformed system, housing will not be a struggle for consumers to obtain; integrated systems would collaborate to develop housing and share information as to what is available.
• In a transformed system, skilled employment will be a reality.
• In a transformed system, wellness will be a focus in a larger sense.
• In a transformed system, data needs and resources line up to be helpful in planning for services and for informing people across systems.
• A transformed system would enable typical environments like schools and pediatrician offices to have the capacity for early identification of serious emotional disturbance in children, thus minimizing the consequences of delayed treatment.
• A transformed system will provide interventions that promote and enhance personal strengths and help children achieve developmental milestones.
• A transformed system will work aggressively to build working alliances through the child serving community, including youth and families, mental health, school systems, healthcare, juvenile justice, economic justice, and foster care systems.
• In a transformed system, treatment planning will incorporate real community support systems for each family based upon existing family resources and needs.

1. Department of Health and Human Services (1996) Prevalence of serious emotional disturbance in children and adolescents.  Mental Health, United States, 1996. Rockville, MD: Substance Abuse and Mental Health Services Administration, Center for Mental Health Services.
2. Reimer, M., & Smink, J., (2005) Information about the school dropout issue.  Selected facts and statistics. Clemson, SC: National Dropout Prevention Center/Network.
3. Office of Statistics and Programming.  Web-Based Injury Statistics Query and Reporting System (WISQARSTM). Atlanta: Centers for Disease Control and Prevention, National Center for Injury and Prevention Control