Queensland Stay On Your Feet® - Toolkit Phase 4 Impact Evaluation Tools
Toolkit Phase 4: Impact evaluation tools
This section helps you answer the question:
‘What did people do as a result of our work?’
For short-term projects of less than two years, impact evaluation may be the final evaluation method used to determine the effectiveness of the project.
Impact evaluation information can be collected at three broad levels:
- an individual level eg. knowledge, attitudes, balance, strength, fear of falling
- an organisational level eg. products, services, knowledge, attitudes, policies, procedures and environment
- a community level eg. skills within the community, attitudes, knowledge, services and environment.
This information is usually collected prior to the commencement of a project/program (‘pre’) and on the completion of the project (‘post’) but can also be collected periodically.
Most of this information can be collected by self-administered surveys and/or face-to-face or telephone interviews.
Aspects of the project/program that can be measured include changes in:
- community capacity
- products, services and programs
- attitudes
- knowledge
- skills
- activities of daily living
- behaviour
- physical activity
- balance
- nutrition
- environment
- social isolation.
Community capacity
Tools to measure the community's ability to address falls prevention and their promotion of healthy active ageing include:
- Community Capacity Index tool
- Community Capacity Index guidelines
These can be accessed from NSW website:
Products, services and programs
To measure changes in the types of products, services and programs that are being implemented, use a community stocktake. This involves surveys and interviews with key stakeholders.
Attitudes
To measure participant, stakeholder and community attitudes, use surveys, interviews and attitude scales [18, 22].
You can also go directly to the University of Illinois website Satisfaction with Life Scale
Falls Efficacy Scale - International (FES-I), which measures fear of falling
Attitudes to Fall Related Interventions Scale (AFRIS)
Research is continuing into the fear of falling and attitudes to fall related interventions, through the Prevention of Falls Network Europe (ProFaNE). It is preferable that you use the latest version of these tools from the ProFaNE website. It is east to stay up-to-date with this research by registering to access the latest falls prevention information, resources and tools for free. To register, visit the ProFaNE website:
KnowledgeTo measure changes in participant knowledge, use surveys and focus groups [18, 22].
Skills
To measure changes in skills, use interviews, self-completed questionnaires and a diary or direct observation [18, 22].
Activities of daily livingTools to measure change in participant ability to undertake their activities of daily living include:
If you wish to use these tools it will be necessary to seek the author’s permission.
Behaviour
To measure changes in participant behaviour, use self-completed questionnaires or a daily diary or observation [18, 22]. However, if there is a strong social push for people to be undertaking certain behaviours, then they may report what is acceptable to do and not what they really do [17]. If this could be the case with the behaviour you are reviewing, then daily diaries and observation can assist to accurately record actual behaviour.
Physical activity
To measure physical activity, use:
- Stanford exercise questionnaire (this tool is freely available for use)
Balance
Listed below are the balance assessment tools recommended in the "Queensland Stay On Your Feet® Community Good Practice Guidelines: Preventing falls, harm from falls and promoting healthy active ageing in older Queenslanders". You will need health professional expertise or training to use these tools.
To measure balance, use:
If you wish to use these tools it will be necessary to seek the author’s permission.
The Physical Performance Battery [219] provides simple validated balance measures that can be conducted in the community without a lot of equipment and training. They test:
- timed standing balance
- repeated chair stands
- timed walking
These can be accessed from the Department of Kinesiology and Gerontology Center, University of Georgia website.
Nutrition
To measure nutrition, use:
- The Home and Community Care (HACC) Malnutrition Screening Questions from the Ongoing Needs Identification (ONI) which includes a question on hydration. This tool is based on the Malnutrition Screening (MNT), which is a simple two question tool that identifies individuals who are malnourished or at risk of being malnourished [117]. The full tool is available from the Home and Community Care (HACC) website.
Environment
To measure changes in the environment, use interviews and environmental audit tools:
- Home Falls & Accidents Screening Tool (HOMEFAST), a home environmental audit tool for use in population based studies
- Public environment audit tools developed by Townsville Thuringowa Safe Communities Project
- Shopping centre safety audit checklist
- Shopping centre safety audit checklist (After dark)
Social isolation
Social isolation is an important issue for older people. This means a low level of social participation and feelings of loneliness. There has been a cross-government project to reduce social isolation of older people. For more information, contact:
Department of Communities
Office of Seniors
Telephone: 1300 132 654 or +61 7 32242625
Department of Communities Social Isolation Project
Some of the tools being used to measure social isolation include:
- The Friendship Scale, which is a six item user-friendly scale that measures social isolation. The Friendship Scale is available for use with permission from A/Professor Graeme Hawthorne, PhD Principal Research Fellow, Department of Psychiatry, The University of Melbourne. It can be accessed by completing a registration form on the following website:
Others tool that can be used also include:
- Duke Social Support Index, which measures multiple aspects of social support and has been used in cross sectional and longitudinal studies. There is a 35 item, 23 item and 11 item instrument [107].
- John Gierveld Scale, which is a tool that measures loneliness and the individual’s perception of social isolation. This tool can be used in telephone surveys and with a large sample size [108].
- VanTilberg Scale, which is an eleven item scale that asks questions about characteristics of the respondent's social relationship networks [109].