Monday, January 20, 2014

Data and Informed Decision Making

Good morning
"And the beat goes on..................."

The Cultural Data Project has just published a report:   New Data Directions for the Cultural Landscape: Toward a Better-Informed, Stronger Sector that looks at cultural data collection issues and the implications for our field.  It's a very good report; not a technical report for data mavens, and I urge you to read it.

As the Executive Summary indicates, the  "report identifies six factors —three at the system-wide level and three operating at the level of individual cultural organizations—that influence the ways that cultural data are collected and used and which may be limiting the sector from effectively incorporating data into decision-making processes. At the system-wide level, it explores:

 Concerns about the accessibility, quality, and comparability of cultural data, which stem in part from the ad hoc way that the cultural data infrastructure has been developed, but which curtail the usefulness of existing cultural datasets.

 Norms in the cultural sector that have traditionally made data collection and use a relatively low priority, including the widespread (though perhaps waning) belief by many cultural practitioners that data are of limited—or even pernicious—value when it comes to making programmatic and/or artistic decisions.

 The ways in which the lack of coordination and standardization among existing data collection efforts inhibit progress in the field.

At the level of individual cultural organizations, the report considers:

 The very real capacity constraints within cultural nonprofits, in terms of both resources and “know-how,” that make it difficult to develop good data collection and interpretation capabilities.

 Dynamics in the internal culture of the organization, particularly with respect to its vision of its relationship to its community and audiences and its orientation to change, that can undermine the effective use of data.

 The lack of a strong organizational vision for how data can be used to inform internal planning and decision-making, as well as the lack of examples of such vision from around the field."

The report offers several suggestions to address these factors.  The one that caught my eye in addressing the "lack of a strong vision, and of examples of such vision from around the field, for how data can be an integral part of internal planning and decision-making":

"Shift the conversation from data’s value as an accountability tool to data’s value as a decision-making tool. Though funder-driven data requirements have helped to develop the cultural data landscape, they have also limited the scope of the conversation; it is time to broaden the conversation beyond accountability and reframe the power of data in terms of responsive decision-making."

The question that always looms is "how"?  How do you refocus all the data, research, information and input that is out there from being merely a tool to prove, after the fact, that a given program, project or approach has met its objective to information that informs decision making in the first place?

For example.  the California State Department of Finance projects that:

"California is growing older and more diverse. 
The Latino population is projected to surpass that of whites in California in March to become the single largest "race or ethnic group," according to a report on shifting demographics in Gov. Jerry Brown's 2014-15 budget proposal. Also, the number of residents 65 and older will jump by 20.7 percent over the next five years, the report said. 
State demographers expected Latinos to surpass the non-Hispanic white population seven months earlier, but Latino birth rates were lower than anticipated. Now, officials say, by March Latinos will make up 39 percent of California's population, edging out non-Hispanic whites at 38.8 percent. Nearly 25 years ago, non-Hispanic whites made up 57 percent of the state, while Latinos made up 26 percent."
"Growth rates vary drastically between age groups, with retiring Baby Boomers projected to reshape the labor force in the next 15 years as more than 1,000 Californians will turn 65 years old each day. At the same time, lower birth rates have resulted in fewer young people, with the 18-to-24-year-old group experiencing a 4.5 percent decline and 5- to 17-year-olds increasing just 0.2 percent." 

Simple enough.  The Latino population in the state is the majority group and growing, and the older cohort (very likely with a white majority) is growing relative to younger cohorts.  That is a profound sea change that will impact everything.  So how do the arts organizations, foundations, public agencies and other groups in the arts use that data to inform their decision making - short and long term on all levels (programming, funding, leadership, communications, marketing and more)?  What do these two simple stats suggest for other kinds of research and data to help in the making of informed decisions?  And how will those data sets be used to help all of us make better decisions?

Without arguing for or against what kinds of decisions ought to be made in light of these projections, the issue is rather what other kinds of information will help in whatever decisions are made to do something, anything or even nothing based on that data.  And that question applies to the whole range of other data available to us now, or in the future about all kinds of things.  What blueprint exists or ought we to create that we can follow to apply the exponentially expanding data and research that is going on?  What help can we provide arts organizations to use data and research?

Central to these questions is how an individual arts organization sees research and data as informing its decision making.  The cultural data report suggested that:

"The extent to which cultural nonprofits use data to inform decision-making depends in complex ways on the internal culture of the organization, particularly its vision of its relationship to its community and audiences.   Many cultural nonprofits, however, seem less focused on anticipating change and creatively strategizing ways to manage it. The leaders or boards of " those organizations tend to concentrate on short-term survival rather than long-term planning, often out of sheer necessity.
Programming and artistic departments generally remain hesitant to use data in planning and decision-making in the first place—even as they wrestle with changing audience behaviors and expectations that research and data could help them understand."

The problem, the report notes, is that data and research continue to be seen as a way to make the case for the organization rather than as a tool to inform future decision making.  What to do?  The report suggests:

"To bolster cultural organizations’ use of data in all areas of planning, it will be important for the field as a whole to do a better job at conducting and disseminating research that reveals ’what works,’ both programmatically and with respect to organizational practices. “The biggest gap in the arts knowledge economy is in the area of practice,” noted one contributor. “What new or different artistic programs are leading to successful artistic outcomes? What new or different business practices are leading to successful operational outcomes? We have such a decentralized system, and the national service organizations are not resourced sufficiently to do a good job of identifying, track"ng, and diffusing promising new practices in governance, fundraising, capitalization, programming, etc.” 

The report concludes with suggestions for other questions to be explored:

"[W]e still have a long way to go before the arts and cultural sector has data that are sufficiently complete and robust to meet user needs. We have lots of partial, idiosyncratic, time- or geography-bound data on specific aspects of arts and cultural activities in the US. ... We don't have a national arts data archive in the same way that we have archives of education statistics, health statistics, labor statistics, etc. It's not that there aren't also many interesting ‘research questions’ that could and should be asked, but many of our basic information needs about the scope, activities, employees and audiences of arts and cultural organizations are still full of gaping holes. 
Participants also called for further inquiry into some fundamental questions about the sector, including: 
New Data Directions for the Cultural Landscape: 
 the relationship between vibrant cultural participation rates and the health of individual cultural organizations (“This would require studying arts consumption behaviors and attitudes in tandem with studying the health of the organizations that those consumers and patrons support. Neither exists in a vacuum. What we're currently missing, by and large, is accessible data on arts consumption/patronage”); 
 the impact of funding investments, as well as new sources of earned revenue, on organizational culture and programmatic success (“The sector requires an honest longitudinal analysis of the impact of different types of institutional investments in organizations of all types. Funders often do not understand the impact of their investments, where they can make the most difference, where they are substituting for market choice, and where they are actually causing dysfunction”); 
 the impact of cultural policy on the cultural sector (“We don't have consistent information about the effects of public policy on resources available to the arts or on public attitudes toward the arts and their different sources of revenue acquisition.”)
All good ideas.  But central to everything is some commitment to helping the people to whom the data might be valuable in decision making, to understand how to go about that use. Specifically what works.  This can't just be an academic question.

The bottom line is that this is an issue that ought to be addressed on a systemic basis, at the national level; it is simply too much for most individual arts organizations to address alone - they need help.   As we ramp up our research and data collection efforts, a lot more data will be coming.  A lot more. But all the data in the world won't be of much real use, if we don't empower and equip our people in how to use it to their advantage.  Data and research ought not to just drive macro policy formation; it ought to help with specific decision making at the micro level.  I would hope national service organizations could address how research and data can help their constituent membership make better and more informed decisions.  I think funders, public and private, need to support those efforts - otherwise they will be difficult to implement.   And there must be links to the research community itself - with a pipeline between those who gather the data and do the research and those who need to use it effectively.

Have a great week.

Don't Quit