Monday, June 8, 2009

June 8, 2009



Hello everyone.

"And the beat goes on............................."

Yes, this blog is long. But I think the subject is important. Speed read or scan if you wish.

Data Mining / Pattern Analysis / Relational Software

Earlier this year I raised the subject of marketing in terms of Casting A Wider Net (which is just another way of saying that we need new potential donors and supporters, new volunteers, new people in our audiences. The question of course, is how to we get those people?

There are, it seems to me, some basic precepts if you want to Cast A Wider Net.

First, you need to identify whom to target. It is inefficient and largely ineffective not to narrow down the “universe” of people to whom you are sending your message. That universe has to be those people who are most likely to respond positively to your message. Trying to sell Hip Hop jewelry to AARP members is probably a waste of time and money.

Second, once you identify your target you have to determine how to most effectively deliver that message.

Third, only then can you craft the specific message for the specific audience you are going after (and just as there may be multiple target audiences, there may also be multiple variations of your message).

The question then is what tools can we employ that will help us to accomplish the first two specific goals – both as a field and as individual organizations? This is the second blog on that subject, and specifically about Data Mining and the value that marketing device might be to the arts & culture marketing efforts. Data mining is (according to Wikipedia) simply the process of analyzing data from different perspectives and summarizing it into useful information. In practical terms it is the automating (via computers) of the process of searching for patterns in data.

I have two barstools that sit in front of my kitchen center island. I was thinking of getting one more, so I went online on Google to see if I could, first, match the stools I have, and then find out if I could get a really good price. I found what I was looking for pretty easily, but decided I didn’t really need another stool after all. But for the next week, whenever I opened the browser connection, there on my home page (which is Yahoo, not Google), there was an Advertisement for Bar Stools.

At the Supermarket (like most people I generally shop at the same market, a Safeway for me, close to my house) last month I bought some South Beach Breakfast Bars. Last week I bought some sugar free Maple Syrup. I am obviously trying to reduce my intake of bad sugars etc. Yesterday, I bought some items that had nothing to do with such a lofty goal, but when I paid for my purchases I got a coupon printed out for a dollar off Nabisco reduced calories cookies.

I’m being watched and monitored. We all are – to a depth we fail to appreciate. Big Brother has begun folks. Seemingly innocent now, but who knows in the future. But this blog is not about the loss of privacy and all that scary stuff; it’s about the arts sector being as sophisticated as it can be in its competition for ever scarcer resources and support. It’s about successfully Casting That Wider Net.

We now live in the age of instant computer generated data mining. At the base level, Data Mining is about purchasing and inquiry pattern analysis. In both of the above situations, the computer identified a ‘behavior’ on my part and its (rather primitive) algorithmic programmed response was to suggest to me something that my inquiry or purchase suggested I might favorably respond to. This is kindergarten stuff.

Data mining software is one of a number of analytical tools for analyzing data. It allows users to analyze data from many different dimensions or angles, categorize it, and summarize the relationships identified. Technically, data mining is the process of finding correlations or patterns among dozens of fields in large relational databases. Data warehousing is defined as a process of centralized data management and retrieval. None of this is new – it has been around for decades. What is new however, is the amount of data being collected and warehoused, and the development of ever more sophisticated software to process the data, analyze it and use it for predictive behavior applications.

Every time you shop at Safeway (or your market) and use your Safeway card, or your CVC Pharmacy card, or Costco card or any one of dozens more you might have; every time you use a credit card, or debit card, every time you make an inquiry or respond in some way on the internet, information about you and your habits and preferences is being gathered and stored. Virtually every consumer decision you make is recorded somewhere and a profile of you is expanded and filed away. From the retail seller’s point of view, they are primarily interested in targeting their budgets to maximize future sales and thus they are interested in what you buy and what you might buy if they can present that option to you in the right way, at the right time. And how they do that differs depending on your age, gender, socio-economic class, education, geographic location and, most importantly, on your buying patterns, and an analysis of trends in your past and recent behavior. What seems clear is that it works.

The amount of data on each of us is staggering and growing exponentially, and the more data available the more accurate the predictive software can be because it has a more accurate profile of the individuals being analyzed (and, frighteningly, data is now being collected on children as young as pre-school and so the data banks will someday go from cradle to grave). Beyond basic demographic information (age, gender, geographic location, etc.), add to your file political and philanthropic contributions, hobbies / interests, clubs and affiliations, political party affiliations, religious preference, financial profile (based on job, property owned, credit reports, DMV info, etc.), education background, personal information from Facebook, LinkedIn, My Space and countless other sites, magazines subscribed to, and criminal history – and that’s just to name some of the information readily available on most of us. But gathering and storing all the data is only half of the equation. What has made this a growth industry and of so much interest in so many quarters is the software that analyzes all this seemingly isolated data and uses it to predict future behavior; software that finds obscure relationships between millions of bits of data to determine connections virtually impossible for human beings to make (even if they could wade through the data). In the retail application, the predictive behavior software also analyzes what kinds of advertisements and marketing approaches actually work once they have narrowed down their targets.

Of course the data being kept on all of us is far more sophisticated than the relatively pedestrian retail / marketing application of this phenomenon. Homeland Security doubtless collects all kinds of additional information on us all – gleamed from public records and – to the horror of many – from private sources including emails, website surfing choices, telephone records, travel, and much, much more. But that is a whole other subject. And there may be serious privacy concerns that raise fundamental policy issues about our sector participating in this kind of approach at all – but I leave that to be discussed and debated. My purpose here is to consider what the nonprofit arts field can do to avail itself of all the marketing tools it might use to improve its bottom line in an ever increasingly competitive world.

Where is all this headed? Far, far beyond the most basic uses of this kind of marketing tool - i.e., Netflix notes you like Pixar Animation movies, so it sends you an email on the release of Kung Fu Panda; or Safeway prints out a coupon for me for some diet product. That’s fairly unsophisticated application of what has developed since then. Data Mining now includes in each folio on each person (and yes, just like you have a credit file, there is a much larger file of information on you that retailers can tap into), relational data on what is sometimes called “tethered individuals” (i.e., family members, neighbors, business colleagues, close friends etc.) By analyzing the data of those people and their profiles and their “relation” to you, Data Miners can get an even better and deeper level of insight into your behavior. Moreover, artificial neural networks (resembling actual biological neural networks in structure) actually learn human behavior patterns through a primitive form of artificial intelligence, and allow increasingly more sophisticated (and accurate) prediction of human behavioral response to specific stimulus (e.g., one advertising approach over another).

I’ve spent months now wading through tutorials on a host of subjects in the data mining area, and I don’t want to confuse you here with a lot of technical jargon because it simply isn’t necessary for us to even understand all the tools that computer scientists have developed to make data mining useful and relevant to a host of applications, including marketing ones the arts & culture sector might productively and beneficially employ. Suffice it to say that there are a host of tools and devices to determine which subsets of data could be the most helpful in trying to analyze variables that might have bearing on predicting future behavior (called Decision Trees). And new tools are being developed constantly. We are at the tip of the proverbial iceberg here.

Retailers from Walmart to Sony Pictures are already using Data Mining on an almost unimaginable scale. From what kinds of stimulus – promotions, shelf placement, advertising, discounts, release dates etc. - have on purchases, to creating new products in response to unmet demands. But Data Mining applications go far beyond retail. Sports teams are accessing data on their opponents play and game plans, identifying trends, and developing strategies to counter performance by opposition teams that may result in defeat. And this kind of game analysis has been going on for years. Only now computers make it easier and faster to accomplish. Thus if the Jets rush the Quarterback on third and long 52% of the time, the opposing team, knowing this, can hold back a tight end for blocking purposes, and thus run a wider variety of plays more successfully. It gets a lot more complicated than that. Anti-terrorism efforts are based largely on data mining and predictive behavior software. It is the analysis of trends and behavior patterns and the interface between that analysis and predictive behavior that is of the most interest.

For our purposes, we want to know which individuals are likely prospects to attend our performing arts events, and which variables influence their decision to attend or not. We want to know which specific consumers are most likely to purchase art. We want to know which people are likely to respond to our pleas for donations (and which donors might give more), and we want to know which elected officials might carry our banner and fight for our needs. And Data Mining can help us get more sophisticated and targeted.

Here’s an example: Let’s say we identify someone who has been to a jazz concert in the past six months, recently bought a jazz CD, and last year bought a biography of a jazz musician from Amazon. Not terribly complicated today for data miners. We could use that information to try to sell him / her a ticket to an upcoming jazz concert. But a more sophisticated approach would be to identify that individual’s “tethered” people, sort out and identify the two that share that interest in jazz, and then develop a mini campaign that packages an entire evening out -- sending email invitations (and the invitation could include a sample concert clip of a song, some background bio info, photos, reviews etc. - and could even include a pre-concert “meet & greet” bonus perk) to the cell phones of those three people to attend the jazz concert (at a 20% discount), followed by dinner at a nearby local restaurant (featuring the kind of food that group likes – with another discount) It could include maps to the locations (including a nearby parking garage – with yet another discount coupon). After the concert it could twitter the those people to push a CD of the concert, tickets to another concert next month, and a coupon good for a subsequent visit to that restaurant or another in the same neighborhood.

That would be neat. Our problem is we can’t identify that (unknown to us) person who went to the jazz concert in the past six months in the first place. We lack the data and the software to analyze the data. But it already exists out there, and is available to us for a price. Our next problem is we don’t have the money, nor do we have the on-staff expertise to know exactly how to deal with this kind of tool. No problem with that second challenge – those who are in the business of selling this data and the software to make it useful are only too happy to help us apply it to our needs, to help us to figure out how to best use it for our purposes and train us in its application. We do have to make some effort to get involved (and alas, we haven’t). The money to afford it? In small applications it isn’t necessarily prohibitively expensive, but we’re financially challenged right now. We need to figure out ways to work together, to act more like a sector or industry and share information, data and most importantly budgets – because the only way we can intelligently and efficiently avail ourselves of data knowledge and application is for us to work together. We’re too small to do it alone as individual organizations.

What should grab our attention as the arts sector is how can we avail ourselves of some of these increasingly sophisticated marketing techniques to sell our products more successfully. If we want to “cast a wider net” and compete for the same dollars and support that scores of other sectors are competing for, then don’t we have to figure out a way to employ the most impactful tools available, and ways to acquire those tools that we don’t yet have? Whether or not we should pursue such strategies is yet another larger policy question that I am not considering here. That we are neither organized enough to be able to employ these marketing techniques, nor able to afford access to them is likely a given. But there are private companies compiling all this data, and firms like Oracle and others writing increasingly sophisticated adaptable software to analyze this data to provide knowledge and information on which marketing decisions can be predicated and with very successful results. This is another area that national service organizations, foundations, state and regional agencies and the NEA ought to be playing a role (at least in investigating the potential in the tools and sharing their findings with all of us.

Arguably any process or procedure that allows any enterprise to more successfully market its goods and services will help its’ bottom line. We could never amass the necessary data on our own. Few sectors or industries could. We’re talking terabytes of data here, which is like trying to imagine the planets in all the galaxies in the universe. The collection of data is already a new industry itself. Data already exists, and software is available, that could help performing arts organizations to more successfully target potential audiences than their current efforts at casting that wider net. We don’t use that which is available in part because our level of sophistication is remarkably low in the marketing area, and, of course, our budgets (even of our most well heeled cultural giants) hardly provide adequate funding to tap into the latest tools. And, of course, we tend to want to market our products in ways that address the experiential level of our consumers, even if such a subjective approach is difficult, if not actually impossible, to quantify.

In fact, our marketing expertise is rooted (and many would argue, stuck) in a much earlier time. We collect precious little data of our own, and that which individual organizations do collect, is relatively simplistic, and even that limited data we are loathe to share with each other. Most performing arts organizations know very little about their audiences. Oh, they might have some basic demographic information but it is spotty, not collected on any on-going basis or updated regularly, and it is virtually never analyzed using anything anywhere near the level of sophistication as algorithmic predictive software. (Hell, we don't even systematically collect the email addresses of all those people who intersect with us at one point or another - and how insane is that?) The same can be said on the data we collect on our donors and other sources of fundraising. For an industry so heavily dependent on specific sources of cash flow and income, we are remarkably primitive in the way we collect, store and analyze the data that may be right out there waiting (begging in today’s world) to be analyzed. Alas we can’t, for the most part, even tell you the basic generational composition of our core audiences – at least not on an individual organizational basis – let alone which of our small donors are likely candidates to give more -- and it is at that level that we need to empower ourselves towards greater success.

Data Mining’s applications to our sector are limitless. Thus, for example (and I am not suggesting this would be a priority application – but it IS something that would have long ago been done in the private sector) it would be relatively easy today to identify what grants the top 100 arts funding foundations have made over the past five years, the backgrounds of those foundation’s current senior arts program officers, and the foundation’s Boards of Directors (and what both of those groups in each foundation favored in terms of allocation of funds), and with very great accuracy, analyze that data using off-the-shelf software that could be adapted to the purpose easily enough, and predict the odds of any given arts organization getting a grant from any specific foundation for a specific purpose. How much grant writing time could be saved? Ah, but that raises all kinds of other questions and problems Barry. Yes it does. But my point is simply that we ought to discuss these things. Aren’t you tired on being a 21st Century person living in a Jurassic world?

More to the point, here’s a better example. We could easily enough (in partnership with AE) analyze what percentage of American Express cardholders (and which ones specifically) in a defined subset of an identifiable category (based on age, income and educational levels – and ones within a set zip code) would be likely purchasers of season tickets to a growing theater company – and, what would be the best way to target those people with the theater company’s message (i.e., direct mail, email, phone call, AE Card bill insert etc.) Or we could run a state by state analysis of how local government (from city councils to boards of supervisors to state legislatures to members of Congressional delegations) not only voted on support to the arts, but where they appropriated money from (a variety of funding pools -- e.g., Education, Transportation, Parks & Recreation and General Fund budgets etc.). Add variables such as political party policy positions and platforms, and a host of other relevant data that the software can identify – and all of that data might suggest which elected officials we should target with our support and messages, and that info might be valuable to us in crafting a real strategy to reinstate government support for the arts. We don’t do that. At best we know how the national Congress and some State Houses vote. That’s pretty much it. And, but for the AFTA Action Fund, for the most part, at best, we send them letters and invite them to our gigs – usually when there is a crisis and we are trying to save something. That’s not a very sophisticated 21st Century political lobby / advocacy strategy. You can bet that other special interests groups are more sophisticated than that. And we are in direct competition with them. The point is there is data out there that can help us, really help us in developing specific strategies, and we are not using it.

We talk a lot about strategic planning, but much of our planning isn’t strategic at all. It is mere guesswork - long on speculation and short on verifiable data. In some ways it’s almost as though we continue to try to sell snake oil out of the back of a wagon. In successful advertising and marketing, mere repetition of a message is still necessary, but what the most effective message is, when and where to apply it and a host of other questions make that whole strategy much more complicated.

So if we are to cast the wider net and succeed in growing our audiences, expanding our donor bases, increasing our number of volunteers, we must consider using the most sophisticated tools that exist, and to be able to do that we are going to have to first figure out a way to cooperate and collaborate and share data, budgets and leadership in gaining new expertise and understanding. Our marketing savvy is just too primitive as compared to those who compete against us. We can’t go it alone anymore. In terms of marketing, the whole arts sector needs to be much more collaborative.

Casting the Wider Net by using Data Mining techniques is just one aspect I think in a much, much larger picture of what Arts Marketing is, or should be about, now and in the short term future. It will take time for us to get up to speed, but this is another one of those areas that we need to start now if we are ever to move down that new highway. We can’t continue to market the arts like they were marketed in the last century. Some of the fundamentals may remain the same, but too many of the variables have changed, and unless, and until, we are using the latest tools we will not be competitive.

I am trying to develop some basic outlines as to how arts organizations might employ Data Mining so as to share information with you on this tool. More on Data Mining and Casting the Wider Net – including how to craft the right message -- to follow in a later blog.

I would love to talk to people more about this subject and so if any of you marketing people are going to Americans for the Arts in Seattle in a couple of weeks, please come up if you see me and let’s talk.

Have a great week.

Don’t Quit