OSR operates a centralized telephone interviewing facility located in the basement of Berkey Hall. The interviewing center has 48 dedicated workstations for interviewers, supervisors and managers. The workstations, connected to OSR’s network servers, run a state-of-the-art Computer Assisted Telephone Interviewing system (CATI). The network servers are protected by firewalls and passwords from unauthorized access. Software and study files are backed up daily and data being gathered during each interview are backed up after every five key strokes. The CATI system allows output of data and interviewer notes in ways that can readily produce SPSS, SAS, STATA, or Excel data files, as well as codebooks with item frequencies..
- Staffing and Training:
OSR employs between 60 and 160 part-time interviewers, data entry personnel and supervisors, depending on the volume of work scheduled. Most are MSU students but many are residents from the surrounding Lansing/East Lansing communities. All interviewers complete an initial 15 hour training program on proper standardized interviewing techniques before beginning to call on actual projects. All interviewers are regularly monitored for quality control purposes and to encourage continuing improvement in effectiveness. All interviewers must complete and annually renew standard training modules on the rights and ethical treatment of human subjects in research, protecting confidentiality and providing informed consent and are MSU IRB certified. Supervisors and managers are appointed from among the most proficient, experienced interviewers. We also offer internship opportunities for some students. (For more information regarding employment opportunities in OSR, click here)
- Hours of operation:
Generally, the interviewing facility operates from 8:30 am (10:30 on Saturdays, noon on Sundays) until 9:30 pm (7:30 on Fridays and Saturdays) seven days a week throughout the year except during major holidays. Calling hours are this extensive to ensure that each potential respondent can be reached at a time when they might be home no matter what their schedule is.
- Calling Protocols:
OSR assumes clients desire a rigorous attempt to maximize the response rate for their study. Consequently, we assume that clients will want at least up to 12 call attempts at different times of day and days of the week before giving up on a sampled phone number. We assume clients will want refusal conversion attempts. We expect to send advance letters to selected households or respondents because the research literature and our own research shows that, in the long run, this improves response, reduces refusals, decreases the number of call attempts needed to complete interviews, and actually saves money.
Respondents are able to schedule call back appointment to begin or to complete an interview. Our CATI system scans across all sampled cases for scheduled appointments during the next time block and delivers theses appointments to interviewers for calling. These automated schedulers typically refresh the lists of cases to be called on twenty-minute cycles.
We utilize Dillman’s Tailored Design Method to conduct mail surveys and design instruments for maximum response. The guidelines and recommendations contained in Don Dillman’s first book in 1978 and updated in the three more recent editions are virtually all heavily research-based and have been demonstrated to improve response and to reduce respondent error.
- Mail Administration:
Some clients welcome OSR’s assistance in designing the layout and format of their questionnaire and some clients have already developed the questionnaire they believe will work effectively for their project. Whichever is the case, OSR works with clients to assist in the production, printing, assembly, mailing of mailed questionnaires to the intended sample of respondents. OSR routinely builds tracking databases to log returned questionnaires so future follow-up mailings can be directed most efficiently to the remaining eligible non-respondents. OSR also administers the follow-up mailings on the schedule agreed upon with the client whether based on the Dillman protocol or based on the best judgment for the particular sample population.
- Coding and Data Entry:
Whether pre-coded or not, OSR uses its computer-assisted survey software systems to program data entry applications that are tailored to the questionnaire design and the response options available for each question. The computer assisted data entry program (CADE) is designed to follow questionnaire skip instructions and to accept only the limit set of valid codes for the responses allowed to each question specified by the questionnaire or its designer, eliminating out-of-range data entry keystrokes as a source of possible error. Typically, clients ask OSR to conduct data entry on returned, completed questionnaires. If response options are not pre-coded on the questionnaire, the clients often ask OSR to develop appropriate response codes first and then code the responses prior to data entry as well. The ‘gold standard’ for error checking to produce clean, error free data is double data entry. OSR programs data entry instruments that enable a second person to data enter each questionnaire while automatically comparing each keystroke entered with that of the first person and requiring a reconciliation of any discrepancies. OSR permits each researcher to specify the percentage of cases that are to be double-entered, but we assume 100% double-entry unless otherwise indicated.
- Data Products:
The cleaned, verified data set can be output into ASCII, SPSS, SAS, STATA, EXCEL or other formats. OSR typically provides the client with an electronic copy of the data set in the format the client prefers along with a codebook with the frequency distribution of responses for each item in the questionnaire. OSR can also provide a file containing all of the text responses to open-ended questions or ‘other (specify)’ responses.
- Design and Response Rates:
Questionnaire design, screen templates and sample contact procedures follow current best web survey methodology practices. We pay close attention to the design of the web survey screens as interfaces with the respondent to ease the respondent’s burden, make the task clear, provide a sense of progress, and make the questionnaire pleasant. Response rates for web surveys, as in all other surveys, depend heavily on the type and number of contacts with the intended respondents. We routinely assume we will make several varied attempts to contact and elicit cooperation by the respondents. Some contacts may be by U.S. Mail and others by e-mail. For many web surveys, offering or providing incentives either in advance or as rewards for response improves cooperation by those invited to respond. OSR has experience with the utility of a variety of different types of incentives for various types of populations.
OSR uses three different software systems for conducting web surveys. They each can accommodate almost any type of question a researcher is likely to use, but the three packages have features or strengths that may make one more useful than another on a particular project. Two of the packages allow OSR to host the web survey from our own secured servers so that no data or sample information has to leave the security of our own facility. One of the packages is hosted from the corporate server of the package, but it too is well protected by layers of firewalls and encryption. One of the packages parallels our CATI system so it is especially useful for mixed mode surveys that involve, for example, providing respondents with the option of either a mail questionnaire or a web survey, or a phone interview or a web survey. One package makes tracking contact attempts with sample members easier than the others. OSR works with each client to choose the package best suited for the individual needs of each client’s project. All three packages enable respondents to complete the questionnaire over multiple sessions if necessary or from different computers, allowing the respondent to resume work to complete the questionnaire from the point he or she left off. All three packages collect interim data prior to the respondent finally submitting his or her completed questionnaire. All three packages enable output of the data in a variety of formats, including SPSS, SAS, ASCII, Excel, or CSV.
- Data Collection (phone, web, mail, in-person, mixed-mode):
OSR’s primary function is data collection. Since 1989, we have collected hundreds of thousands of telephone interviews and about 100 thousand of each mail questionnaires and web survey responses. Some projects have involved both interviews and mail questionnaires or mailed questionnaires or online questionnaires. We have also done a smaller number of projects that involve face-to-face interviewing, although we have also worked with some faculty researchers to enable them and their assistants to conduct computer assisted personal interviews using our software systems.
- Data Entry/Keypunching:
OSR has conducted data entry on tens of thousands of questionnaires either as a part of mail surveys we have conducted or of questionnaires or forms researchers have collected on their own. Using our Computer Assisted Survey Execution System or other software, we program a data entry application specifically for the questionnaire, permitting only keystroke entry of codes that are acceptable for each question. The ‘gold standard’ for error checking to produce clean, error free data is double data entry. OSR programs data entry instruments that enable a second person to data enter each questionnaire while automatically comparing each keystroke entered with that of the first person and requiring a reconciliation of any discrepancies. OSR permits each researcher to specify the percentage of cases that are to be double-entered, but we assume 100% double-entry unless otherwise indicated.
Given their training and vast experience across a wide range of surveys, OSR staff can review and provide advice or suggestions regarding the most practical mode of data collection for a particular project, the availability of suitable sampling frames, the sampling design options, improving questionnaire questions or question sequencing, pretesting, strategies for contacting respondents, the use of incentives, techniques for maximizing response, timeline development, and data analysis. An initial consultation of up to one hour is provided at no charge. Lengthier consultations typically involve a fee.
- Cost Estimation for Grants/Proposals:
OSR staff members have prepared cost estimates for hundreds of different survey and focus group projects, including estimates for work to be included in grant proposals before they are submitted to agencies and foundations. Each project has its own unique features based on the combination of method chosen, sample type, sample size, respondent contact protocols, etc. Informed by our past experiences, we take the unique features of each project into account to estimate the costs for staff time, labor, supplies and materials, fees, and other expenses that will enable the project to be completed successfully and within budget.
- IRB Assistance:
OSR staff will either provide researchers with advice on how to prepare descriptions of the research protocol to submit to the Institutional Review Board responsible for overseeing the protection of human subjects or can prepare and submit the proposal descriptions on behalf of the researcher. OSR can also help address any questions or concerns raised by IRB reviewers.
- Cognitive Interview Testing:
Cognitive interviewing is a set of methods for discovering how individuals hear and understand the questions they are being asked to answer and how they think about and formulate answers to those questions. This is especially useful when new questions are being developed for use in a subsequent survey so questions can be improved to ensure respondents actually answer the question the researcher intended them to hear and answer. OSR has conducted either concurrent think-aloud or retrospective think-aloud cognitive interviews for numerous, varied surveys.
- Questionnaire & Instrument Design:
Using research-based ‘best practice’ guidelines, OSR can assist in the design and formatting of mail questionnaires, web survey questionnaires, telephone interviews, face-to-face interviews, focus group scripts, advance letters, cover letters, follow-up letters and reminders for effective contact with prospective respondents.
- Designing Sampling:
OSR staff members have extensive experience designing samples from discrete lists of targeted populations; designing random digit dial samples of city, county, statewide and national populations; designing samples of rare populations; and designing simple or disproportionate stratified samples of households and businesses. OSR has worked extensively with Survey Sampling, Inc., and Marketing Systems Group (GENESYS) on list-assisted survey samples. OSR can help individual researchers develop appropriate sampling designs and sample sizes to meet the specific research goals of the researcher’s project.
- Focus Groups:
OSR has designed, conducted, and reported on focus groups since 1997. Typically, this has involved recruiting and scheduling participants, developing the moderator’s script and probes. While these have most often involved audio-recordings, they have sometimes involved video-recordings as well. For some projects, OSR has been involved in the recruitment and scheduling phase to enable a faculty researcher to conduct the focus group discussion.
Since 1989, OSR has trained several thousand individuals to be effective telephone interviewers. We have trained numerous others on survey and data management skills. We have also trained many others either informally or formally on data analysis using SPSS and weighting data. OSR has at least occasionally provided training as a service separate from other services such as data collection.
- Data Coding:
OSR can assist researchers in the development of coding systems for closed- and open-ended questions and the building of codebooks for data sets. Especially when OSR is conducting the data entry of questionnaire, the text responses to ‘other (specify)’ items or open-ended questions can be entered and then all text responses for the question output to aid in the development of categories of kinds of responses. These categories can be assigned numerical codes, which then allow individual text responses to be assigned the appropriate code, making statistical tabulation of results possible. OSR can assist in both the development of the coding categories and in the actual coding of individual responses.
- Weighting Data:
Calculating and applying caseweights is typically necessary for surveys that involve stratified random samples. Doing so is also typically desirable to correct for differential response rates across segments of the sample population to improve the representativeness -- even for simple random samples. OSR staff members have developed the caseweights for hundreds of different data sets. Dr. Hembroff has conducted workshops for CSTAT [Link to CSTAT] on the weighting of complex survey data.
- Analysis and Report Writing:
At the conclusion of data collection or data entry, OSR routinely provides researchers with an electronic file containing the study data in raw ASCII format or in SPSS, STATA, SAS or Excel format, depending on the researcher’s preference. This is usually accompanied by or contains the data dictionary. Typically, OSR also provides a codebook for the data set which includes the question wording and response options for each question in the questionnaire and the frequency distributions of responses to the questions in the data set. Also, typically, OSR provides a copy of the programmed version of the interview or data entry instrument containing relevant skip instructions or logic checks. Frequently, the preparation of the data file for analysis involves the calculation and application of caseweights. Oftentimes, researchers also request methodological summaries of the sampling, data collection, and data preparation procedures. Clients also sometimes request that OSR analyze the data and prepare tables or charts/graphs that report survey results. The analysis may be a simple as frequencies and crosstabs or much more complex multivariate analysis.
OSR staff members are frequently asked by clients to give presentations summarizing survey results. Staff members are also often asked to conduct workshops, give guest lectures, or give presentations on survey methods, sampling challenges and solutions, survey error, or the findings of OSR methodological experiments or analyses. OSR has given numerous presentations to student groups, professional audiences, the general public, in live venues, via net meetings and webinars, and via television broadcast.
With more than 110 combined years of experience conducting surveys and covering a wide variety of survey methods, technologies, and tasks, OSR staff can assist researchers or organizations in numerous ways. Below, we list many of the services we more commonly provide and a brief description of each. In each case, we offer a collaborative relationship to assist each researcher or group with their unique goals and needs.We provide a somewhat more detailed description of our approach, experience and facilities regarding the key methods of data collection, coding and data entry.
Types of Surveys Available
OSR conducts telephone surveys nationally, statewide, and within individual counties or communities. It conducts telephone surveys with individuals sampled from lists or random digit dial samples of landline and cell phone numbers. OSR has frequently conducted telephone interviews in Spanish, Arabic and other languages as well as English. All interviewing is closely supervised and sampled calls are monitored by supervisory personnel or recorded for quality control.
OSR’s general approach to telephone surveys is standardized, structured interviewing. To that end . . .
As is the case with all other modes of survey data collection, response rates in mail surveys are very much a function of the numbers and timing of contact attempts to reach the intended respondents. But, moreso than in other modes, the visual design of the mailed questionnaire is critically important to gaining cooperation and response from sample members.
Since 2004, OSR has conducted more than 200 web surveys in English and a dozen other languages. While many web surveys and polls conducted by others are simply postings to websites for anyone who happens to access the site to answer, the web surveys OSR conducts are virtually all based on probability samples of individuals in some targeted population (e.g., students, teachers, etc.) or all members of some targeted group or organization (e.g., members of a professional association, faculty within a college, etc.). We have procedures to ensure confidential responses solely from sampled respondents without duplication.
OSR provides the option to make combinations of mail, web, and phone data collection modes available to respondents in order to maximize their opportunities to respond.