Conducting Surveys

A survey may not be the best way to gather the information you are looking for. Be sure to review these considerations first, before proceeding:

• Are you looking for specific, in-depth information from students? Perhaps a focus group is a better fit.
• Conduct a literature review on the topic with respect to your target population. What is already known about the topic you are interested in? Could you generalize that information to meet your needs instead? The public health librarian at Rush Rhees could assist you with this process.
• Are there national data sets available that would provide the data you need?
• Are there surveys already conducted at the University that you could use? (i.e., National College Health Assessment, UHS Consumer Attitudes Survey, Patient Satisfaction Survey, etc.)
• Reuse of de-identified or secondary historical (~1-3 years) survey response data may still be relevant.
• Consider survey fatigue. Surveys should be thoughtful on design, length, and schedule (calendar year). Investigate the University’s survey schedule and determine if your survey will interfere.
• Consider the benefit to the survey taker, and if the survey responses will be actionable. What is the likelihood that collecting this data from students will lead to change?

Before you begin creating survey questions, it is important to be clear about the survey goals. Be specific! Start by writing down 2-3 specific goals for your survey.

• What do you want to learn as a result of the survey?
• Are you hoping to gain an understanding about the respondents
  • behavior, habits, beliefs, or knowledge?
  • opinions or attitudes?
  • interests in certain topics or types of programs?
  • barriers, issues, or problems?
  • motivators?
  • past experiences?

All the survey questions you create should link back to one of the goals you identified. If a question doesn’t align with your goals, consider eliminating it.

First consider when you need to have the final, analyzed data completed by. Keep this as your target date and work backwards from here. Be sure to include these important steps in your timeline:

• Collecting and analyzing existing data / reviewing the current literature on your topic (1-2 weeks)
• Create the first draft of the survey tool (1-2 weeks)
• Submit draft for review (2 weeks)
• Possible submission to the UR’s Institutional Review Board (2-4 weeks)
• Piloting your survey with a small sample of students (1 week)
• Make necessary revisions (1 week)
• Survey promotion, launch, collection of responses (up to 3 weeks)
• Data set deidentified and formatted by UHS survey sponsor (1 week)
• Analysis, review, and presentation of results (2-3 weeks)
• Total time: approximately 3 – 5 months

• What population(s) do you want to gather information about with your survey?
  • Do you want to survey specific populations, or a broad population with different sub-groups?
  • Broad population – all students, all undergraduate students, all graduate students
  • Specific populations / sub-groups – international, race, gender, sexual orientation, ethnicity, Greek affiliation, varsity athletes, first generation, class year, etc.
• How many respondents do you need to have complete your survey for you to have useful data? It is not necessary to survey an entire population to have valid, generalizable results. A random sample will likely be sufficient. Refer to the Survey Monkey Sample Size Calculator to help you determine your sample size, and work with your UHS Survey Sponsor.
• If the survey includes students under the age of 18, RSRB regulations may apply. (If you are unsure of population age, be sure to ask this as a survey question if the study is considered research.)
• Consider different sampling methods to support adequate representation of responses in your data set. Refer to these resources from Qualtrics and Statology.

It is also important to consider the overall design of your survey. This step includes considerations for formatting, survey flow, types of response options, and question design.

• Think about a survey like a conversation with your participant.
• Be thoughtful about the overall flow and arrangement of questions.
• Group questions with similar content together.
• Use open ended questions sparingly – these are more difficult to analyze.
• Provide a progress bar if possible.
• Response categories should include an array of choices. Search “Likert scale examples for surveys” for examples. Include “N/A”, “other”, and “don’t know” where applicable.
• Be sure that your survey questions are not leading and include clearly understood language.

To reduce survey fatigue:

• Keep the survey as brief as possible while still accomplishing your survey goals. Surveys which take 15 minutes or less to complete will generate more responses.
• Add page breaks.
• Do not ask questions you already know the answer to.
• For “choose all that apply” questions, limit the number of options.
• If the information you’re hoping to gather becomes too great, you should consider breaking down your list into a “need to know” and “nice to know” list before proceeding.

Conduct user tests before you launch:

• Test your survey yourself and ask others to take it as well.
• You can gather feedback about the overall flow, clarity, length, typos, and other suggestions through email or through conversation.

For the privacy of your survey participants:

• No external identifiers should be collected in surveys. Examples include, but are not limited to, social security numbers, date of birth, identifiable images of participants and home address.
• Never collect high-risk data in a survey.

Surveys should begin with an introduction which is a concise description including all the pertinent information about the survey. The introduction helps prospective respondents decide if they want to complete the survey or not. A survey introduction can be provided to you upon request.

A thorough survey introduction should include the following parts:

• Survey purpose describing why the survey is being conducted, what you hope to learn, and what will be done with the information learned.
• Define the population being surveyed.
• A sentence ensuring prospective participants that participation is voluntary.
• A sentence describing if the survey is confidential or anonymous (see definitions of each to ensure you are choosing the correct one).
• Deadline for completion.
• Approximately how long it will take the user to take the survey.
• Whether or not the data will be shared and who it will be shared with.
• Describe the reasonably foreseeable risks, discomforts, hazards, or inconveniences related to the subjects’ participation in the survey i.e. experiencing uncomfortable thoughts or feelings during the course of the survey. If risks are identified, describe the probability, magnitude, duration, and methods of mitigating the risk.
• Contact information for the person administering the survey.

The Research Subjects Review Board (RSRB) review research that is conducted or supported by the University of Rochester to determine that the rights and welfare of the human subjects are adequately protected.

• Does this survey involve humans?
• Does this survey include people under the age of 18?
• Is this survey defined as research?
• Is this survey a systematic investigation contributing to generalizable knowledge?
• Will this data be published?
• Will this data be presented outside of the University (ie: at a conference)?

When in doubt, it is always best to consult your HPO Survey Sponsor and the UHS IRB Coordinator to be sure if the RSRB should be included or not.  The RSRB approval process could take a few days up to many weeks or months. It depends on the quality of the submission.

Here are the steps you should be aware of:

• Complete a Study Protocol document, provided to you by your HPO Survey Sponsor.
• Include the survey tool, survey invitation, incentive strategy, promotional strategy and materials, etc. for review through an online portal.
• The HPO Survey Sponsor will upload all the above to the Click IRB platform for review by the RSRB.
• UHS IRB Coordinator: Suzanne Coglitore.

• Understand the difference between conducting an anonymous survey vs a confidential survey. Clearly share which type will be conducted with participants.
  • An anonymous survey does not track or collect identifiable information about a participant such as an IP address or email address.
  • A confidential survey may collect identifiable information, but the survey administrator ensures the response data will not contain personally identifiable information when being shared, although certain survey administrators may access the data and identify respondents for additional program evaluation purposes.
• Ensure access to raw data is properly secured and is not shared with those without permission.
• Communicate to the participant how the data may be shared after analysis. Be specific, such as whether data will be presented in aggregate or summary form and if personally identifiable information will be shared, and under what circumstances.
• Never share individual participant information without the consent of the participant unless there is a legal, safety, or compliance requirement.
• Any survey data that may be used to contribute to generalizable knowledge (aka research) must be reviewed by RSRB prior to data collection. This might include results posted to a web site for annual
• All surveys should be included in the university survey inventory to help record and track surveys conducted across the University.
• Data should be saved in University of Rochester approved survey systems/platforms or University of Rochester provided storage services such as Box.
• Where possible, data should be saved in a comma delineated / excel format.
• Survey data must be de-identified prior to sharing with student for data review and/or analyses.
• Survey data should be reviewed bi-annually to determine the relevance to help determine disposition.
• Survey response data should be retained for no more than 5 years unless longer-term storage is legally required.

• The most basic and easy to understand survey reports provide a frequency report, presented in tabular or graphic form. (i.e., “how many students answered “no” to question #1”).
• Comparisons of different populations of respondents is also helpful. (i.e., for question #1, how did first-year students respond compared to seniors)
• To protect individuals’ privacy, results should not be reported for any categories or questions with less than five respondents.
• In addition to providing the above, it is also important to include an overall summary of your findings and an outline of next steps to demonstrate how the results will be used to create action and change.
• Please note, survey results can NEVER be shared, presented, or published beyond the University of Rochester community without RSRB approval.