(Indianapolis) We have an opportunity to ask questions on a national survey that will gauge the extent of civic engagement online. We hope to repeat the same questions in subsequent years to follow trends.
It’s hard to get this right. If you ask people whether they do specific activities, such as blogging or posting on message boards, two problems arise. First, these forms of engagement change very rapidly. Yesterday, it was blogging; today it is podcasting and MySpace; tomorrow it will be something else. Second, these activities are only partly “civic” or “political” (by any definition of those terms). If you ask people whether they have created a blog, you can’t tell whether they have done something relevant to politics or community issues. The blig might concern knitting or porn.
Therefore, we might be tempted to ask more abstract questions, such as: “Have you used digital media for civic purposes?” But obviously, most respondents will have no idea what this question means. So we need somewhat abstract questions that can outlast changes in technology, yet ones that people can understand.
I have pasted some draft questions below in case anyone has any advice. These draft items include abstract leads and then concrete follow-ups:
1. Within the last seven days, have you used the Internet to express opinions about politics, a social issue, or a community problem? (“The Internet” includes email and text messages as well as websites.)
If no, skip to question 3.
2. I am going to read you a list of specific Internet technologies. Please tell me whether you used each one within the last seven days to express your opinions about political or social or community issues. a. Email. b. Your own blog. c. Comments on someone else’s blog. d. A social networking site like MySpace or Facebook. e. By making a photo, video, or audio and sharing it online. f. By commenting on someone else’s photo, video, or audio.
3. Within the last seven days, have you used the Internet to gather information about politics or a social issue or a community problem?
If no, skip #4
4. Now I am going to read you a list of specific Internet technologies. For each one, please tell me whether you used it within the last seven days to gather information about political, social, or community issues. a. Search engines such as Google. b. Professional news websites, such as CNN.com or washingtonpost.com. c. Blogs. d. Social networking sites like MySpace or Facebook. e. Sites that contain shared pictures or videos, such as Flickr or YouTube. f. Wikipedia or another wiki site.
Peter,
I agree completely about the difficulty of crafting these questions. For example, question three says “gather information” which may set the bar too high for a respondent. If I were asked this question, I would wonder whether this meant “doing research” or some such. But I suspect that you want to catch those participants who regularly visit political blogs or who read about political issues on regular blogs. People need not post themselves in order for this information to pass over into daily conversation.
http://www.boingboing.net/
and
http://www.blogpulse.com/
are types of sites that just follow what is unusual, interesting, or popular, and often involve something political. How would we capture whether participants select political options when they appear? Not easy stuff.
Surprising to me, Jon Krosnick strongly advises asking “In a typical week” rather than “Last week” in surveys. I can give you some references if you’d like to examine the evidence yourself.
More centrally, your task is indeed a difficult one. I just saw Diana Mutz talk at an ICA preconference, and she pointed out that the majority of political discussions on the Internet happen in non-political fora, like hobby groups.
My advice: pretest a lot. Use the think-aloud procedure, because it is amazing how respondents interpret questions. Make sure that you gather all the possible answer choices for q2 and q4, even if it is a long list. If these choices are specific and make sense to participants, you should get good data.
A final note: be concerned about defining concepts by giving examples (e.g., “sites site like MySpace or Facebook”). Do you have clear idea of what a social networking site is? I don’t and so I suspect that respondents won’t either. One of my colleagues has analyzed data comparing myspace to facebook users, and he fuond that users of these sites differ dramatically, even after controlling for education and other demographic differences, on civic behaviors and attitudes, calling into the question whether they should be lumped together into one category anyway.