The Challenge of Online Surveys for the Arts

Good morning
"And the beat goes on……….."

We're already knee deep in the 2016 Presidential election, and that means public opinion surveys are going to be a regular staple of the news.

While these surveys are usually fairly simplistic in that they only seek to register voter candidate preferences at a moment in time, they are nonetheless very important because they break down the responses by demographic category and thus give a glimpse of which groups favor which candidates and in what percentages. In any, and every, field, opinions classified by demographics are a crucial piece of information.  And the public opinion polling experts have gotten fairly sophisticated in their reach for accurate information.

Most of these political polls are telephone interviews, and the pool of those tapped is a demographically representative sampling of a specific geographic territory (more often than not the whole country, but also of individual states like Iowa and New Hampshire).

Online polls are very similar, but there is an extra challenge to these polls in that sending out unsolicited emails inviting people to take the survey is prohibited.  That ban makes it harder to start out with a demographically representative sampling.  Usually the pollster must first use snail mail, the telephone or  some other approach to ask if they may send an unsolicited email - and thus create a starting representative sampling pool.

The same challenge exists for us in the nonprofit arts.  There are any number of questions and issues that it would be informative, and instructive for us to sample our field on so that we could amass more data and information on how we do things and why, what and how things impact us, on our perceptions, and what our opinions and choices might be on a host of things.  Most of the people in our field would very much like to have that data and information, and most would have no objection to being invited to take an online survey (whether or not time and circumstances would allow them to always say yes).  But the prohibition on sending out uninvited emails makes it burdensome and expensive and thus difficult for us to conduct that kind of research.

Each organization can, of course, conduct surveys and invite those people / organizations on their own lists and that is permitted.  Thus a foundation can query their grantees, an arts organization can query their members and so on.  But those surveys, while unquestionably valuable, still don't necessarily represent the whole of our field and thus their application may be limited.

What to do?

The Pew Research Center (one of the major players in all kinds of research, including online polling) has, as a solution to the challenge, created what they call the American Trends Panel.  I think I have this right:  They created a pool of some 2500 or so respondents that is demographically representative of the country on a number of criteria (I am supposing including geography, age, gender, income, ethnicity, education, etc. etc.).  They recruited those on the list with an initial small payment, and those that agreed to participate over a period of time agreed to take X number of online surveys, and be paid some small stipend for doing so.  The number in each survey pool necessary for the pool to be representatively credible is perhaps half the size of the entire pool, and the response rate number necessary for credible results is some percentage of that.

Thus they have a ready pool that meets standards they can use for their inquiries.  They don't have to go through the whole process each time they want to sample the public's opinions.

I think we in the nonprofit arts could benefit from a similar undertaking.  We need a pool of perhaps 2000 arts organizations representing every sub strata of the field - geography, budget size, discipline, area of operation (e.g., arts education, advocacy, performing arts etc.), leadership / audience ethnicity, etc. etc. and individual administrators according to job title, years in the field, gender, education etc. etc.)  If each organization / administrator was willing to take three  (less than 30 minute surveys) per year, over a two year period, and we paid each person / organization $10 for each survey they took, then those organizations that wanted to conduct online surveys could use this pool if they paid the cost - i.e., assuming 1000 names in each pool x $10 = $10,000 (a reasonable fee given the alternative of creating every pool from scratch), and it would be relatively easy to use.  That would open wide the possibility of more accurate polling for our field and arguably yield some important data and information of value to the field.  This would create the opportunity for 12 reliably representative surveys to be taken during a two year period by perhaps 12 different researchers.

It would cost very little to initially set it up. And once created - at least for two years - essentially pay for itself by each research user.  Very little administration would be required.   It is a way we could help each other and the whole of us.  The token money paid to participants isn't that important.  The reason for participating is to help the field.

I write of this because I am in the process of doing a sector wide online survey inquiry on the subject of Communications, and I want the potential responder pool to be representative of our whole field.  I can't use my own master list of some 5000 arts organization names because I want to honor and respect the prohibition against sending unsolicited email invitations..  And the 12,500+ names on my subscriber list aren't organized to constitute a credible representative sampling.    More on my survey next week, and a plea for your help.

But for the future, I would hope some entity might help us emulate the Pew Research approach.

Have a great week.

Don't Quit
Barry