PERFORMANCE MEASUREMENT STUDY
HCD Health Care Data PO Box 234010 • Encinitas, California 92023 Telephone 760-943-8087 • Fax 760-943-8072 • Email lerogers@hcdinc.com Web www.hcdinc.com
Joint Commission on Accreditation of Healthcare Organizations
Executive Summary
Performance Measurement Study
Introduction
HHV took a far reaching step by initiating a performance measurement study to determine the effectiveness of their programs. Measurement is by no means unfamiliar in health care. Veterans Affairs, military and private sector health organizations routinely measure responses to therapy, staffing levels, and cost. However, to our knowledge, no organization similar to Help Hospitalized Veterans has conducted a performance measurement study. This voluntary initiative will allow HHV to quantify performance with measures of patient health outcomes, patient satisfaction and the effectiveness of the Therapist/Craft Care Specialist all in relation to the HHV Craft Kit Program.
In today's environment, accountability is an especially pressing concern. Organizations are being compared to one another and they must be prepared to demonstrate empirically, with data, that their program is appropriate, available, timely, effective, safe, efficient, continuous, respectful and caring. This measurement study will promote understanding and support for hospitalized veterans and will show the therapeutic value of the Craft Kit Program, not only to patients, staff and host hospitals, but also to the many generous HHV donors.
Health Care Data Inc. conducted this study over a three-year period, from August 1996 to September 1999. The first year of the study was a trial period to test the survey instruments and to make sure data were collected using the same methodology. Although we feel we have good data for the first year, many changes were made as a result of receiving input from the Craft Care Specialists. The final report shows the results of the first year but the first year data are not used in the last two years of comparative information.
Health Care Data Inc.
Health Care Data Inc. has been contracted by Help Hospitalized Veterans to conduct this performance measurement study. Health Care Data Inc. is a nationwide performance measurement and benchmarking company that develops outcome measures and compares the performance of healthcare organizations both internally and externally over a period of time.
Accreditation
Health Care Data Inc. has met the initial criteria for inclusion in the future accreditation process and is included on the Joint Commission's list of acceptable performance measurement systems. Health Care Data Inc. is committed to meeting future criteria established by the Joint Commission on Accreditation of Healthcare Organizations. This is important to the participating medical centers since accreditation is part of their quality improvement plan.
Discussion - Results
The patients receiving HHV Craft Kits and Craft Care Specialist assistance have an overwhelming satisfaction for the HHV Craft Therapy Program with a positive response rate of 98.6%. HHV was one of the top three volunteer organizations that have activities in the medical centers by 98.7% of the veterans.
At the onset of this study the Craft Care Specialists assigned levels of difficulty to categories of HHV Craft Kits; Level 1 being the easiest and Level 4 being the most difficult. As can be seen by the average number of hours for patients to complete kits with specific diagnosed diseases are increased by the levels of difficulty.
It is also apparent that ease or difficulties of completing Craft Kits vary among patients participating in craft activities that are risk adjusted by disease, i.e. anxiety disorder, dementia, etc.
This information is extremely helpful when Craft Care Specialists assign or assist patients in selecting craft kits for therapy. The medical center patient treatment team can use this information as an integral part of the patient treatment plan.
One of the most important measures in this study is the functional improvement of the patient resulting from Craft Kit therapy. Craft Care Specialists as members of the medical center's multi-disciplinary patient treatment teams participate in defining policies and developing the total care plan for their patients. For this purpose the Craft Care Specialists assessed their patients to determine if there were functional improvements. They found the following:
- Functional Improvement: 48% of the patients showed improvement
- Self Assessment: 47% of the patients showed improvement
- Attention Span: 45% of the patients showed improvement
- Motivation: 51% of the patients showed improvement
When compared to the patient's overall perception of the impact of the Craft Kit Therapy Program in relation to their wellness we found that 89.6% of the patients felt HHV Craft Kits therapy helps maintain or improve their physical capabilities. While the above clinical assessments demonstrate improvement, clinical outcomes are lower than the patient's perception of their improvement. Literature states that patients who have a good mental attitude and outlook toward treatment have better chances for improvement. Future comparative data will be most important in the evaluation of functional improvement outcome measures. It is remarkable to state that HHV's Craft Kit Program assisted patients in functional improvement both physically and mentally. Patient's comments, which can be found in the comment sections of the Patient Satisfaction Surveys, reinforce that data. It can be seen by these comments that many of the veterans give intrinsic responses, which is measured under motivation.
27.6% of the veterans selected leather as the #1 Craft Kit. Plastic models was ranked #2 at 19.3%. Suncatchers #3 at 15.0% and Paint-By-Number kits #4 at 12.4%.
In terms of Craft Kit difficulty, 46.7% stated the Kits they were working on were somewhat difficult or very difficult. With this quantifiable information and the average time to complete a Kit by level of difficulty and disease specific, Craft Care Specialists will be able to better guide and select a more appropriate kit for the patient.
Patients reported that Craft Care Specialists assisted 63.6% of them. Of those that were assisted, 88.0% said the assistance they received was helpful and 67.2% felt they had benefited from the assistance. From this we can conclude that the more Craft Care Specialists that are available to give therapy and assistance to veterans, the more functional improvement and patient satisfaction will be demonstrated.
Even though there is at this point no organized effort on behalf of HHV to recruit and establish volunteer chapters at the medical centers, .05% of all hours during the survey period were donated to HHV which accounted for 18,566 logged hours.
The area of least acceptance is the distribution and availability of HHV Craft Kits at the medical centers. It appears from the veterans' comments that there are a variety of policies and procedures in the various medical centers.
It can be concluded that the veterans not only appreciate Help Hospitalized Veterans, but also the donors who make the HHV program possible. 96.8% thanked the donors by sending a postcard of thanks to the person who donated the craft kit. Just as they feel it is important to remember the individuals who make it all possible... the donors.
Purpose of the Performance Measurement Study
To conduce an unbiased performance measurement study that will:
- Measure and enhance the therapeutic value of the HHV Craft Kit Program
- Establish skill levels for craft kits
- Measure and enhance the effectiveness of the therapist/Craft Care Specialist
- Have HHV Craft Kit Program included in the patient's treatment plan based on quantifiable data
- Measure patient satisfaction
- Establish a reference database
- Benchmark comparative data
- Detect variations in performance
- Improve performance (i.e. specific Craft Kits for different diagnosis or types of patients; special skills for Therapist/Craft Care Specialist)
- Assist medical centers in meeting accreditation standards
- Develop outcome measures that meet the objective requisites for reliability, relevance, validity, cost justification, and performance discrimination
- Provide the results to HHV donors for the purpose of demonstrating the therapeutic value veterans receive from their donations
Methods
HHV, through Health Care Data Inc. (HCD), developed a performance measurement system used for assessing and comparing the success of HHV Craft Kit and Craft Care Specialist Therapy programs provided to our nation's veterans. The intent is to provide data in a number of service areas so that HHV and the participating medical centers can identify processes that need improvement. The data will also integrate with the medical center's continuous quality improvement plan to meet accreditation standards. Data collected will provide a benchmark from which future performance can be compared. Similar data from many organizations can be used to compare one medical center's performance to that of others.
The performance measurement study is structured with consistent, well-defined goals and outcome measures. All information is confidential and exclusively for the use of Help Hospitalized Veterans, the Department of Veterans Affairs, and the California State Veterans Homes. The performance improvement framework developed by HCD incorporated the following key functions:
- Meetings with HHV management to define goals of study
- Determine the number of hospitals to be included in the study
- Meetings with VA Headquarters officials - Washington D.C. regional directors and medical center directors for their endorsement
- Meetings with VA Medical Center (VAMC) management officials and therapists/Craft Care Specialists to discuss possible outcome measures
Discussions included:
- Top four Diagnostic Related Groups (DRG) using Craft Kits in the following settings: acute care, long term care, alcohol and drug units, psychiatry, and medical/surgical rehabilitation
- Therapeutic value of Craft Kits
- Effectiveness of therapist/Craft Care Specialist
- Therapist/Craft Care Specialist turnover
- Craft Kit inventory control
- Volunteer hours per patient using Craft Kits
- Effectiveness of the Craft Kit in relation to the type of patient, i.e. visual impairment, anxiety disorder, stroke, etc.
- HHV's sponsorship of a two-day training program for VA headquarters officials and all potential participants in the study
Outcome Measures
As a result of the discussions, approval was received from all VA staff levels to conduct the study and the following ten outcome measures were developed:
- Patients receiving Craft Kits from Help Hospitalized Veterans Creative Arts Therapy Program
- Help Hospitalized Veterans therapist/Craft Care Specialist consultant turnover rates
- Volunteer hours for Help Hospitalized Veterans Creative Arts Therapy Program
- Patients diagnosed with an anxiety disorder completing a Help Hospitalized Veterans Craft Kit
- Patients diagnosed with dementia completing a Help Hospitalized Veterans Craft Kit
- Patients diagnosed with an alcohol and drug disorder completing a Help Hospitalized Veterans Craft Kit
- Patients diagnosed with stroke completing a Help Hospitalized Veterans Craft Kit
- Visually impaired patients completing a Help Hospitalized Veterans Craft Kit
- The average number of hours for all patients to complete a Help Hospitalized Veterans Craft Kit
- Patients using Help Hospitalized Veterans Craft Kits, attending scheduled Craft activities, and demonstrating functional improvement
Each outcome measure was formatted for both the therapist User's Manual and the Craft Care Specialist User's Manual for their reference:
- Outcome measure
- Rational for measurement
- Definitions
- Possible sources for data
- Calculations
- References
Data Collection
Each medical facility was given a User's Manual and computer disk to collect data. Outcome measurement data were collected during each quarter by the therapists or Craft Care Specialists and submitted to Health Care Data Inc.
There are innumerable outcomes for which meaningful data could be measured. This initial data collection system targets only ten outcome measures. They were chosen for their general applicability and the perceived need for accurate data in these areas.
To get a copy of the complete study, please contact HHV.
|