What Do I Know?

By John Benson

Until today, few of my colleagues in the polling profession would have known my secret: On my own time, I edit (with my wife Anke) a small-press magazine about people out of place or excluded from society, for whatever reason. We use fiction and poetry of any genre to look at how a character deals with her or his own "otherness."

It was in my capacity as editor/publisher of Not One of Us that I found myself this weekend at Gaylaxicon, "the annual national gay, lesbian, bisexual, transgender & friends science fiction, fantasy, horror, and gaming convention," which met this year in Cambridge, Massachusetts. A short story published in Not One of Us was up for a Gaylactic Spectrum Award, and the winners were to be announced at the convention. So I went there with the story's author and attended sessions on two of the three days.

But the pollster in me never sleeps. I tried to picture Gaylaxicon as a convention intercept, a sort of convenience sample for interviewing a large number of people at little expense. Then I thought, what if I wanted to generalize those responses to represent the national adult population? (Believe me, I've seen worse methodologies.)

The gender distribution pretty much matched U.S. Census proportions. I had expected the attendees to be far younger than the national adult population, but the skew was not that great. A small weight there. Education? Big-time difference: Attendees were mostly college graduates and higher. As for race--almost entirely white; hardly a person of color in sight.

But wasn't I forgetting something? Oh yeah, sexual orientation. Far more LGBT than straight folks show up at this convention. Do we think that once we apply the traditional weights for age, race, gender, and education, we will have an even marginally representative sample? We certainly would not, if we were asking about gay marriage, discrimination, and hate crimes. But would the difference in the distribution of sexual orientation affect responses on other topics?

I hear someone ask, "Why don't you just weight by sexual orientation?" Well, the Census does not ask that question, and I think we are all aware that people are reluctant to reveal on surveys a characteristic or behavior considered undesirable by a large part of society. So using the responses to a sexual orientation question from a telephone survey of the general population as a benchmark would not be helpful. As researchers at Harris Interactive have shown, internet surveys get a higher proportion of respondents who, even after weighting, describe themselves as LGBT; but is that figure necessarily more reliable?

This was all just a thought experiment: I never had any intention of interviewing anyone at the convention. But it serves as an extreme-case example of how we can sometimes forget that standard weighting techniques may not account for everything.

By the way, the convention turned out to be an interesting experience, listening to writers with an LGBT sexual orientation discuss their profession-which is writing, not being gay-in light of their own experiences. Seeing the world through someone else's eyes: this is not so different from the reason Anke and I edit Not One of Us, or why I do public opinion research as my profession.

John Benson is managing director of the Harvard Opinion Research Program, Harvard School of Public Health, and associate editor of Public Opinion Pros.

Readers who wish to respond to this or other articles appearing in Public Opinion Pros, or to contribute commentary of their own in 800 words or less, should consult our author submission guidelines and editorial policies under "Letters to the editor and op-ed articles." Op-ed submissions must be received at least two weeks before the first of the month for the issue in which they are to appear.