Conclusion The worth of data from a survey study hinges on whether or not people give true and accurate answers.
When participants give wrong or misleading answers, they do so for two principal reasons: In the Cameron group's survey study, there is reason to believe that the accuracy of the data was affected by both factors. Validity problems resulting from respondents' inability to provide accurate information. The Cameron group's self-administered questionnaire consisted of items and required approximately 75 minutes to complete.
It included a large number of questions that dealt with highly sensitive aspects of sexuality, many of them presented in an extremely complicated format. This procedure raises concerns about respondent fatigue and item difficulty. By the time they reach the later stages of a very long task such as filling out a questionnaire for more than an hour , respondents tire. They often become careless in their responses or skip questions entirely in their hurry to finish.
One way that researchers assess whether respondent fatigue created problems in a long questionnaire is by including consistency checks: Questions from an early section of the questionnaire are repeated in a later part either in identical form or alternatively phrased so that the reliability of responses can be checked.
The Cameron group did not report any systematic checks for the internal consistency of questionnaire responses, although in one paper they noted discrepancies between responses to some of the survey items about early sexual experiences.
Another problem results from using highly complex questions. The Cameron group's questionnaire contained questions that not only addressed sensitive topics but also required respondents to read a large number of alternatives and follow intricate instructions. In one section, for example, respondents were expected to read a list of 36 categories of persons e. Another item asked respondents why they thought they had developed their sexual orientation, and gave a checklist of 44 reasons, including I was seduced by a homosexual adult, I had childhood homosexual experiences with an adult, and I failed at heterosexuality.
Many respondents probably found such tasks confusing because of their length and complexity or alienating because of their content. In addition, it is likely that many respondents did not read these long lists of response alternatives carefully and completely. A related problem is that the questionnaire used language that was probably difficult for many respondents to understand.
Whether such problems led to underreporting or overreporting of various experiences cannot be known from the Cameron group's data. Validity problems resulting from intentional misrepresentations by respondents. Even when survey respondents can understand the question, they sometimes purposely lie or hide the truth. Self-report measures are necessarily based on the assumption that respondents do their best to provide truthful answers.
In some cases, however, people do not wish to divulge sensitive information about themselves. This is especially likely for questions about finances or behavior that is stigmatized, illegal, or potentially embarrassing.
In other cases, they intentionally give false answers out of a mischievous or malicious motivation. In the Cameron group's survey, most questionnaire items focused on highly personal and sensitive sexual issues. Recognizing the inherent difficulty in getting honest answers to such questions, experienced survey researchers use various techniques to overcome respondents' reluctance to reveal sensitive information or respond accurately.
One of the most important of these is convincing respondents that their privacy will be preserved. However, internal contradictions in the Cameron group's survey reports make it unclear whether respondents could reasonably believe that their answers truly were anonymous. Throughout their reports, the Cameron group described the questionnaire as anonymous and reported that it was returned in a sealed envelope.
But in a paper they reported that "postquestionnaire inquiry with selected respondents indicated that many homosexuals did not count persons contacted in an orgy or restroom type setting as 'partners'" Cameron et al.
For that last statement to be true, the researchers had to know which respondents to select for the post-questionnaire inquiry in order to reach "many homosexuals" who had participated in orgies or sex in restrooms there were too few such individuals to have been detected simply through a small number of randomly targeted follow-up interviews.
How the supposedly anonymous questionnaire answers e. Apparently, however, respondents' anonymity was not absolute, a factor likely to discourage some respondents from divulging sensitive information about themselves. Whereas many members of the sample simply refused to participate, others probably completed the questionnaire but provided bogus answers. There were many reasons for potential respondents not to take the Cameron survey seriously. It was presented by a stranger who simply appeared at the door, with no affiliation to a university or prestigious research institute that would inspire confidence.
As noted above, the questionnaire was excessively long and contained many questions about highly personal topics. In one city, the local newspaper quoted a police officer who advised a neighbor not to participate, describing the survey as "kind of raunchy" Omaha World Herald, May 23, , p.
Given the many reasons not to take the survey seriously, at least a few people probably decided to have a bit of fun with the researchers. Suppose that someone purposely gave untrue responses with the mischievous intention of portraying himself as an individual who routinely engages in what might be considered outrageous sexual behavior.
He probably would have overstated his general level of sexual activity, reported frequent participation in multiple unconventional sexual acts, and provided an unusual sexual history e. If as few as 3 people in each city faked their responses in this manner, then a substantial portion of the total number of reports of such activities would be invalid. The Cameron group's survey is highly vulnerable to faking by a small number of mischief makers for three important reasons.
If only a few of these respondents were lying or faking their answers, it dramatically alters the findings. The impact of mischief makers is maximized in samples with low response rates, like that of the Cameron group. Such samples tend to exclude respondents who provide dispassionate, honest answers that would offset the influence of individuals who purposely provide false data. Because they lacked systematic checks on the validity of responses to their questionnaire, and because interviewers did not directly observe respondents while they completed the questionnaire, the Cameron group could not determine how many of their respondents gave false answers.
None of these concerns were addressed in the Cameron group's published papers. The interviewers may have been biased and may not have followed uniform procedures.