Ideally, surveys should be administered by well-trained evaluation staff or program staff not involved in service delivery.
Data collection staff should:
- Ensure that only a unique identifier-and not the participant's name-is on the survey
- Provide clear and easy to understand verbal instructions for survey completion
- Allow participants to ask questions to ensure that they understand the survey
- Provide safeguards, such as envelopes for completed surveys, to ensure participants' security and confidentiality
Interviews, Focus Groups
Because interviews and focus groups are not anonymous, they should never be conducted by program staff or staff from the sponsoring organization; it violates confidentiality.
They need to be conducted by well-trained evaluation staff. Your evaluator may need your help in scheduling a discussion at a time and place convenient for the interviewee or focus group participants.
It is helpful for evaluation staff to observe the healthy marriage/relationship education program being evaluated to gain insights into the activities, content, and group dynamics of the program.
Your evaluator will need to work with you on several issues in preparing for a class observation, including:
- Letting you know whether instructors will be notified of a visit in advance, or whether the visit will be unannounced
- Scheduling a class observation so that the evaluator can observe a "typical" session, as opposed to something unusual that won't offer a sense for how the program usually operates
- Gathering information on attendance to find out how much of the program and which topics were received by participants. Like information on participant characteristics, class attendance is typically collected by program staff or staff from the sponsoring agency. These data should be purged of all identifying information.
Work with your evaluator to make sure you understand how the class attendance data is collected and what it means. Keep in mind that when creating a form to collect the data someone has to input all the information so the form should be easy to read and understand to prevent data entry problems. You may want to develop a form that is standardized for all of your classes so data is consistent.
Some questions to keep in mind when creating a form:
- Are classes offered in one day or in multiple sessions?
- If you have multiple sites (e.g., schools) involved in the evaluation, understand how the methods of collecting, and the meaning of "present," may differ across sites, as well as how you will reconcile any inconsistencies when analyzing data. For instance, you would want to create a sign in sheet on one piece of paper instead of on multiple sheets of paper. Dates of classes would be listed at the top in columns and then participants’ names on the left in rows. Participants would sign as they attend the class.
- How do you want to verify that participants were in attendance (e.g. roll call, initialing a pre-printed roster, signing their full name for each class)?
- How do you want to count participants’ as completing a class?
- Are those who join the class late counted as having attended? Are participants marked present if they attended even one minute of the class, or only if they attended the entire class?
You may want to develop an attendance tracking form if class attendance is not taken or if you have multiple sites from which you would like consistent attendance data.
Program Records/Administrative Data
An evaluation can benefit from a review of program records-including the funding application, curriculum materials, reports to funders, and any previous evaluation reports. You can serve as a liaison between your evaluator and staff from the sponsoring organization in obtaining these data.
Make sure that you do not receive confidential information if you have no need for these data (e.g. participant names, social security numbers, program budget information, including staff salaries).
- U.S. Department of Health and Human Services, Administration for Children and Families, Office of Planning, Research & Evaluation. The Program Manager's Guide to Evaluation Glossary.
- Human Subjects Research, Protecting Human Subjects Training Health Resources and Services Administration
- Boyd, H. (2002) Making sure your electronic data are accurate: Quick tips #22. Madison, WI: University of Wisconsin-Extension, Cooperative Extension.
- StatPac, Inc. (n.d.). Designing surveys and questionnaires. Bloomington, MN: Author. (Web Page)