index construction

For a client, CIRCLE is preparing an index of American civic engagement that will aggregate the trends in more than 40 indicators over the past thirty years. Our various trends begin and end at different times, depending on the whims of survey designers and changes in actual behavior. Therefore, it wouldn’t work to build an index by averaging the indicators for each year. If an activity that happens to be common (such as wearing a political button or sticker) is suddenly included in the available surveys, then the whole index would jump up arbitrarily. Likewise, we couldn’t add measures of new and relatively rare forms of engagement, such as writing blogs, without arbitrarily lowering the index.

Therefore, I’m proposing that we measure the percentage-point change in each indicator compared to the baseline year in which it was introduced. For instance, watching the TV news enters the National Election Survey in 1984 at 72% and rises to 85% in 1986. I don’t include it at all in 1984, but count it as 13 points in 1986. The index is thus a weighted average difference in all the indicators compared to when they began. Adding blogging doesn’t lower the index, but as blogging becomes more popular, the growth tends to raise the index.

I don’t know if this is an original method–probably not. I also don’t know for sure if it’s a good idea. I think it has some advantages, especially for measuring a category (such as “civic engagement”) that is likely to evolve over time.

This entry was posted in Uncategorized. Bookmark the permalink.

3 Responses to index construction

  1. Joseph Sinatra says:

    I’m just curious…

    1. would each indicator be equally weighted or will some be given more value?

    2. aren’t levels important to some degree? isn’t an increase from 0 to 10 qualitatively different from an increase from 90 to 100?

    3. how would you deal with the changes in civic behaviour and thus the indicators over time? for example, how would one incorporate blogging or community journalism into an index like this? does one include the variable for the entire time period, entering zeros for when the technology was not in use?

  2. An interesting problem. For the stock market, they have a procedure for adding new stocks and taking out new stocks that preserves the continuity before and after the addition. (Essentially, the new stock comes in at the weight necessary to preserve that continuity.) In the context of a civic index, you might say that blogging should now replace writing letters to the editor. So in 2003, you figure out what the letter writing part of the index and determine the factor necessary to make blogging equal for that year. In 2004, you multiple the actual blogging figure times the scaling factor from 2003 and add it to the index, which presumably will be growing.

    That isn’t my favorite solution.

    Here’s what I think is the best solution, albeit somewhat difficult. Find out what’s universal about civic engagement. Two examples: number of hours of engagement per week or number of people engaged in any civic activity in a week. Now, you’ll want to take whatever data you have and estimate those underlying metrics. For example, consider the introduction of blogging. If blogging displaces other civic activities, the index would remain unchanged. However, if it adds to the amount of time people spend on civic acitivities, it ought to be reflected properly as an increase.

    Just my $0.02 – probably only worth what you paid for it. 🙂

  3. Peter Levine says:

    Michael’s suggestion is good, but I’m not sure I can pull it off. I’ll be thinking about it.

    In answer to Joe’s questions ….

    1. We have used weights that reflect a set of value-judgments. We will disclose these but also show what the index would look like if you were to weigh the indicators differently–and especially what would happen if you decided to drop various measures altogether. That gives the reader a sense of the value-judgments that are implicit in the index.

    2. The idea that there’s a non-linear relationship between various indicators and their civic importance is interesting. I’m not sure we can do much more than treat each increment as equally important–but that’s arbitrary.

    3. One of the reasons I like the current method is that we can add indicators. They only show up once they begin to change, because we incorporate the change into the index. When a variable is not available, I don’t count it as zero; I average a smaller number of variables.

Comments are closed.