What Are You Evaluating? Part 1

When developing a plan for evaluating engagement processes, consider that there are at least three different aspects of a process which you can evaluate: the purpose of the process, the mechanics of the process, and the substantive outcomes of the process.  While evaluations of the purpose and mechanics may be more of interest to internal audiences, the evaluation of the outcomes of an engagement process are usually of greater interest to a broader community audience.  In this post we will focus on evaluations related to the purpose.  In the next two posts we will focus on evaluating mechanics and substantive outcomes.

Before undertaking any engagement process, you should be able to articulate why and what you hope to accomplish. This does not mean that substantive outcomes should be predetermined. Designing an engagement process to validate a predetermined outcome is generally ineffective and over time breeds cynicism rather than trust. However, when designing a process it is worth asking questions like: Why public engagement?  What are we hoping to accomplish in terms of creating relationships, setting policy, increasing understanding, improving implementation, etc.? What types of issues will we be discussing and why? What would progress look like? What types of change are we hoping to see? or, What types of efficiencies would we like to create? These types of questions, asked at the outset of a process, can help you decide who to involve, establish realistic expectations for the process, and develop time-lines and checkpoints that will help you keep a process on track. Asking these questions will also help you set a baseline that can be used at the end of the process to determine whether and to what extent your process met its intended purposes.   For example, if a core goal of the process was to obtain input from across the community, at the end of the process you would ask questions like: How many community members participated?  Did we obtain a fair representation of the diversity of thought in the community? Why or why not? Did rates of participation increase over time? To what extent did public understanding of or interest in the issues increase over time?  There is, of course, an interaction between purpose and mechanics.  Considering both from the outset will help you develop evaluation forms that better allow you to capture the information needed for effective analysis of what works and what doesn’t throughout the process.