Evaluation of any kind is a challenge for nonprofit organizations generally and for nonprofit arts organizations in particular. Resource constraints and focus on mission, sometimes at the expense of critical management issues, make evaluation a frequent afterthought if considered at all.
Evaluating community engagement is particularly difficult because it is in its infancy as a practice for arts organizations. As such, it is no surprise that techniques of evaluation specific to it are not nearly as advanced or as systematized as those available for fundraising, marketing, or other arts management functions.
In addition, there are at least two vastly dissimilar categories of evaluation that are important for community engagement practitioners. One is, of course, evaluation of the outcomes of the work. The other, however, may be as important because it determines how successful any community engagement project can be. That is the effectiveness of the engagement process itself.
Evaluating Engagement Process Effectiveness
ArtsEngaged holds that effective community engagement is based on four fundamental principles: strength of the relationship, mutually understood benefit to the parties involved, partnership in planning and implementation, and quality of the relationship maintenance plan. The last is extremely important because, due to the arts industry’s focus on events it is easy to move away from a newly built relationship once the originating program is over.
There is little common understanding of the nature of community engagement. Confusion about the meaning and practice of as well as the relationships between audience development, audience engagement, and community engagement is typical. The principles stated above are the work of ArtsEngaged and do not yet represent consensus in the field. What follows is a preliminary presentation of categories for evaluation based on the four areas presented here.
Pre-existing relationship with identified communities (partners) built on respect
How long-standing is the relationship with the participating communit(y)(ies)?
How deep is/are the relationship(s) with the participating communit(y)(ies)?
- #/% involved
- Enthusiasm for relationship on part of participating communit(y)(ies)
- Enthusiasm for relationship on part of the organization
Mutual benefit (beyond or in addition to $ for community organization)
[Note: If there is no community-recognized benefit of the project, this is not a community engagement project]
Is the only benefit to the community financial? (Yes/No)
If no:
- How well articulated/understood are community-selected goals for the project?
- To what extent does the project meet community goals?
- How well articulated/understood are the organizational goals for the project?
- To what extent does the project further organizational mission and meet project goals?
Collaborative design, implementation
At what point in the planning/development did the participating communit(y)(ies) begin to contribute to project design?
Was/were the communit(y)(ies) involved in:
- Selecting programming?
- Marketing efforts?
- Production/participation details?
- Developing post-event relationship maintenance plans?
Relationship Maintenance Plan
How clearly detailed are the organization’s post-project plans for maintaining the relationships with the participating communit(y)(ies)?
For each of these questions, response options and ratings as currently articulated can be found here
Evaluating Engagement Results
Evaluation of community outcomes, while still not widely understood within the arts industry, has the advantage of being directly related to the field of community development in which a good deal of work has been done. Based on work by Americans for the Arts’ Animating Democracy initiative and the University of Pennsylvania’s Social Impact of the Arts Project, here are some rudimentary sample categories for various kinds of engagement projects.
Evidence of relationships, for example:
- Participation by community members in discussions, surveys, events
- Press, social media mentions
- Instances of community seeking organizational assistance
Evidence of mutual benefit
- Success in community-desired outcomes, for example:
- Decreased violence
- Increased school retention
- Reduced racism
- Evidence of success in organizationally-desired outcomes, for example
- Variety of new funding sources
- Increased ticket sales
- Vibrancy in programmatic offerings–vibrancy in genres, styles–reflecting influence of the community
As with any successful evaluation project, the intended outcomes must be articulated in the planning process rather than created after the fact. They are the means by which the project should be judged. This aspect of the work demands extra time and thought on the front end and is a key reason any evaluation, not just of community engagement, is ignored or avoided in practice. The tendency to shortchange evaluation is perhaps understandable in the context of the industry’s extremely limited time and human resources, but it is also shortsighted and counterproductive. For ourselves, for our community partners, and for our funders, we need to be able to have information to improve our work and demonstrate its value.