This session took place on May 3, 2018 at the Creative Placemaking Leadership Summit in Madison, New Jersey.
“What criteria can we use to evaluate creative placemaking projects?”
“‘Creative placemaking’ is a noun, adverb, process, field, product… How do you evaluate all that?”
"What if the measurements we're being asked for don’t correspond to the measurements we took?"
“What’s the non-nonprofit approach to evaluating creative placemaking projects?"
“These projects can be hard to measure aside from testimonials. What other metrics do we have?”
“Can we analyze experiences as well as outcomes?”
If you’ve asked yourself any of these questions, you share some common ground with the audience at this panel discussion all about measuring, evaluating, and reporting the impact of creative placemaking initiatives, moderated by ArtPlace’s Jamie Hand. The session brought a special focus to the role artists can play in effectively and creatively evaluating placemaking projects.
HOW ARTISTS CAN COLLECT DATA TO EVALUATE CREATIVE PLACEMAKING PROJECTS
Susannah Laramee Kidd of Metris Arts Consulting kicked off the session with her presentation “Collaborating with Artists to Evaluate Artist-Led Engagement.” She explained how the LA County Arts Commission, where she worked at the time, commissioned a public art and community engagement initiative dubbed the Creative Graffiti Abatement Project, which sought to promote the value of civic spaces and deter vandalism at four facilities in south L.A. County through these arts-based projects:
- Greenmeme’s sculptural environment “Rise and Shine” in the A.C. Bilbrew Library courtyard;
- Fausto Fernandez’s mixed media assemblage "Dominguez Field and the Famous Titans of Aviation” at the East Rancho Dominguez Park & Community Center;
- Cocina Abierta Collective’s series of cooking videos that profile members of the Victoria Park community for the Victoria Community Regional Park; and
- Swift Lee Office’s sculptural canopy and mural “Butterfly Wings and Scales” at the Woodcrest Library.
Along the way, Sara Daleiden and Mark Escribano developed the documentary video Civic Art: Four Stories from South Los Angeles to chronicle the development of each artwork.
From the beginning, Laramee Kidd worked with artists to plan how they would evaluate the projects. The Commission wanted their evaluations to match their theory of change: namely, that art and community engagement would lead to increased involvement and user attachment to the sites, which would in turn lead to a decrease in graffiti. To this end, the Commission hosted artist-led focus groups; held embedded evaluations in which a Commission staff member went on-site to ask visitors questions and record their feedback; and undertook “data collection as engagement” strategies—for example, offering survey respondents a locally sourced recipe card in exchange for their time and thoughts.
The Commission also employed Animating Democracy’s Aesthetic Perspectives framework.
“It can be hard to separate an artist’s artistic goals from their social change goals,” Laramee Kidd said. “This arts-for-change language gives us a way to evaluate that’s different from the white European traditional way.”
The framework’s categories and qualities include openness (the artist has designed for multiple generations and other avenues of inclusivity); sensory experience (the “beautification” factor: bold forms, bright colors, and attractive scale that shifts perceptions of place); and the creation of space (for communal meeting and social interaction). The report Art as Infrastructure: An Evaluation of Civic Art and Public Engagement in Four Communities in South Los Angeles County details the Commission’s findings.
HOW WE CAN DOCUMENT CREATIVE PLACEMAKING PROJECTS CREATIVELY
Next, Alexis Stephens of PolicyLink presented “Creative Documentation of the ArtPlace Community Development Investments Program [CDI].” CDI launched in 2015 to investigate and support six diverse place-based community development organizations as they endeavored to sustainably incorporate arts and culture into their core work. CDI provided the organizations with $3 million in flexible funding for a period of three years.
PolicyLink served as ArtPlace’s research and documentation partner for CDI, and helped us to develop a research framework and share key lessons and results. The framework consisted of three thematic areas:
- Organizational Evolution
- Collaborative Practice
- Community Development Outcomes
Operating alongside these areas was a group of cross-cutting questions posed as part of the unique “learning agenda” customized by all the organizations. The agendas took the particular context, issues, and interests of each organization into account.
Then, there was the creative documentation component. Artist, curator, writer, and California College of the Arts professor Chris Johnson documented the work of the six sites. “Chris has been tasked with showing—collecting and presenting—the meaning of these organizations’ efforts, as well as the nuts-and-bolts of their work,” Stephens said. “His work is a complement to traditional social science research.” She pointed the audience to Question Bridge, a transmedia project focused on black male identity in America, as an example of Johnson’s work.
Stephens said Johnson had so far completed one round of interviews with CDI stakeholders, and will do another in the fall. “Our hope is that his work will be a reference point for the field as well as helpful to the organizations themselves,” she said. She played clips of video interviews with leaders from CDI grantees. Carol Gore, President and CEO of the Cook Inlet Housing Authority said the arts have broken new ground for her organization. “Our mayor and governor have declared housing a priority, and suddenly we’re seeing new opportunities for funding,” she said. “Those doors would never have been open to us if it hadn’t been for ArtPlace.”
Stephens related Johnson’s finding that, “This process tunes participants into their own agency: they’re not following a script.” She said she believed that such creative approaches to creative evaluation would help the field grow.
HOW ARTISTS CAN DESIGN PROJECT EVALUATION METRICS
Rachel Engh of Metris Arts Consulting presented on “Adding It Up,” her organization’s 2016 evaluation of two creative community development projects initiated by Pillsbury House + Theatre: Art Blocks and Arts on Chicago.
Minneapolis-based Pillsbury House had been running these projects for several years—and had collected significant data about them—but weren’t sure what impact they were having, so they engaged Metris Arts to help them evaluate. Engh referred the audience to the Adding It Up Technical Appendix for a list of all the data sources Metris used in their reporting, but in summary said she and her colleagues asked how the projects were influencing three main factors: access, attachment, and agency.
Metris recruited artist Peter Haakon Thompson to help them craft a survey to distribute to residents in areas where Pillsbury House programming had happened and hadn’t happened. Since 2003, Thompson’s “The A Project” has encouraged residents to post a red letter “A” in their windows to show support for artists—in the way that some residents post window signs indicating their home is a “safe house.” Metris’ hypothesis was that on blocks where Pillsbury House activities had taken place, people would feel heightened levels of the three “A” factors.
Their research concluded that residents in these areas were indeed measurably more likely than “control” respondents to rate their neighborhood as “good” or “excellent” at providing opportunities for creative expression (evidencing access); more likely to feel more connected to their neighbors because of arts offerings (evidencing attachment); and more likely to imagine positive futures for themselves and their neighborhoods (evidencing agency). Metris also noted a spike in posted “A’s” following the Pillsbury House programming, indicating increased support for and interest in artists and the arts.
HOW OUR EVALUTATIONS CAN CONTRIBUTE TO GREATER EQUITY
Maria Rosario Jackson of the Kresge Foundation and Arizona State University (ASU) reported on the school’s Creative Measurement Lab (CML) pilot, which launched in fall 2017. The CML convened participants from an “intentional mashup” of disciplines—including architects, economists, urban planners, public health workers, sociologists, artists, students, and community organizers—from both within and beyond the ASU area who share an interest in comprehensive place-based strategies in vulnerable communities.
Jackson characterized their conversation as “responding to a field need that keeps getting magnified.”
“As we begin to understand why community development matters, we see there are big gaps in it,” she said. “Those gaps have to do with evaluation, but also with how we think about how change happens and how we measure what we think matters.”
Jackson said there is increasing evidence about the relationship creative design can have to stewardship of place, agency, social cohesion, and both internal and imposed narratives of place. “These are community qualities that we know are important and necessary to achieve longer term change, but we haven’t thought about them as part of our theory of change, or as preconditions for other kinds of change,” she said. “What we haven’t done is shift the gaze onto the systems that impact these qualities. If we think about measuring what matters, it’s not just measuring what’s on the ground, but also what systems are influencing our measurements. How do we fundamentally change our thinking about the change that needs to happen, and how do we know we’re looking at what we need to look at?”
Jackson spoke about Kresge’s involvement with Equitable Evaluation (EE), an effort by a group of evaluators and funders that posit “there is a moral imperative for the philanthropic sector to design and implement evaluations that actively contribute to equity. In not doing so, philanthropic evaluation practices risk reinforcing or even exacerbating the inequities philanthropic change initiatives seek to address.” Jackson explained that there have been efforts to promote resident-led evaluation, but that EE is “a call for the evaluation field [itself] to look at what role they’re playing—or not playing—in expanding opportunities and equitable outcomes. It really examines why we do the things we do; the industry standards we have.”
Is a literature review or a survey always the right tool to start or measure a project?, Jackson asked as an example of the kinds of inquiries EE is making. Are “experts” really the people we should be centering in our projects, versus the people who’ve lived the experience we’re trying to deal with? “Some of the people at CML wanted to examine these current methods,” she said. “To others, this interrogation of underlying beliefs was excruciating. How do we build the environment for people to do this hard work of reframing?”
Jackson also noted, “You can do the reframing and the expansion of methods, but if it doesn’t land in a real space, it stays academic. So the third part is implementation. How do you get people who are in program design and related fields to have an appetite for different kinds of data? How do we change industry standards around understanding neighborhood change?”
Moderator Jamie Hand had said at the beginning of the session:
“There is no silver bullet for creative placemaking evaluation; it all depends on what you’re trying to do.” To wrap up, she offered, “The idea here was to inspire, to get permission to think differently about the different expertises that can come to bear on evaluation, and to ask, ‘How can we push different industries to ask what really matters to measure?’”