Outcome Monitoring Toolkit

Introduction

Background

The Outcome Monitoring Toolkit was produced by the DSHS HIV/STD Program. The toolkit was developed to provide prevention partners with information and resources useful in reflective practice.

Description

The Outcome Monitoring Toolkit for HIV prevention programs contains:

• A description of the program, including suggested assessible objectives using the questionnaire;
• A questionnaire tailored for use with that program; and
• Data entry forms.

Questionnaire Development

Questionnaire development involved several steps:

• Program staff reviewed or created logic models for each prevention program.
• Program staff matched to the logic models to existing questionnaires to determine appropriateness.
• Program staff invited prevention partner staff to discuss the surveys and asked to suggest areas of improvement. These discussions addressed questionnaire length, language appropriateness for the target population, and the match between questionnaire items and the prevention program goals.
• In some cases, program staff used pre-existing surveys used in pilot studies and, in other cases, developed new surveys.
• Prevention partners piloted questionnaires in the field.
• Following the pilot studies, program staff revised the questionnaires based on program partners and DSHS feedback.

Appropriate Use of Surveys

Programs in the field have used these questionnaires in the field. Program staff did not subject the questionnaires to rigorous psychometric analysis. As such, program staff find the questionnaire results appropriate for identifying potential areas for program improvement. They can also identify program strengths. However, the results they provide should not be seen as the basis for inferences about the associations between determinants and client behaviors.

Toolkit

Mpowerment

• One Pager – Mpowerment
• Post Only Questionnaire – Mpowerment 
• Data Entry Form – Mpowerment

PIP-WE

• One Pager – PIP-WE
• Pre Questionnaire – PIP-WE
• Post Questionnaire – PIP-WE
• Data Entry Form – PIP-WE

SISTA

• One Pager – SISTA
• Pre Questionnaire 1 – SISTA
• Pre Questionnaire 2 – SISTA
• Post Questionnaire – SISTA
• Data Entry Form –  SISTA

VOICES/VOCES

• One Pager – VOICES/VOCES
• Pre/Post Questionnaire – VOICES/VOCES
• Data Entry Form – VOICES/VOCES

WiLLOW

• One Pager – WiLLOW
• Pre Questionnaire – WiLLOW
• Post Questionnaire – WiLLOW
• Data Entry Form – WiLLOW

Documentation

Questionnaire Item Documentation

Questionnaire Items Sources

1. Developed in discussions with Mpowerment program staff at two prevention programs in Texas.
2. UTSA staff collaborative discussion with UT evaluation expert
3. Austin Outreach SISTA Program Questionnaire
4. Adapted from Project Fate Baseline Survey, Partners in Prevention, The University of Texas Health Sciences Center at San Antonio
5. Advocates for Youth. Training Youth to be Peer Educators. TAP Evaluation Survey: Pretest. Cited in Monitoring Outcomes of HIV Prevention Programs – question bank.
6. Toolkit authors
7. Valley AIDS Council staff in collaboration with UT staff
8. WILLOW HIV Transmission Reduction Among Women Living with HIV. (1999) Original Evaluation Instruments:
   • Self-Esteem Scale, p. 6.
   • Knowledge Scale, p. 42
   • Condom Attitudes Scale, p. 58
9. SISTA Implementation Manual, p. 291