Step 5: Assess the Quality and Results of the Engagement

Your team may not have the resources or funding to complete a formal evaluation, but it is important to periodically assess your approach to engaging stakeholders and your group’s impact to ensure that it is effective over the lifetime of your initiative. When assessing the activities and functions of the stakeholder group, it is particularly important that you solicit structured feedback directly from the stakeholders so they have an opportunity to voice their opinions and concerns. You should ask for this feedback on a regular basis so that it can be used to improve the stakeholder group and its processes and increase the likelihood that the group can achieve its goals. It is also essential to share the results of the assessment with the involved stakeholders.

Three tasks are critical to assessing your stakeholder engagement process and its impact:

1. Identify the purpose of the assessment.
2. Select or create a sustainable and user-friendly assessment strategy.
3. Field the assessment periodically and create a plan to encourage responses.

Task 5.1: Establish methods and schedules for convening and communicating with stakeholders

You can evaluate many elements of stakeholder engagement. Start by clearly defining the purpose of your assessment so that you can select an appropriate strategy. Questions you might ask include:

• What do you want or need to learn from this assessment?
• Will the results be used to improve the stakeholder engagement process, for accountability purposes, or for both?
• Do you need to assess the entire stakeholder engagement process, or just certain elements? Which elements would benefit most from assessment?
• Do you want to learn about the impact of particular recommendations or products?
• With whom will you share the results of the assessment?

Key Considerations:

• It is important to begin thinking about assessment and data collection early on and throughout the process of establishing and convening the group, not just at the end.
• Possible components of the stakeholder engagement process to assess include:
  • Group membership, both composition and size.
  • Stakeholder structure, including the governing body and any workgroups.
  • Clarity and satisfaction regarding stakeholders’ roles and involvement.
  • Communication content and mechanisms, both internal and external.
  • Satisfaction with scheduling and amount of work.
  • Experiences of stakeholders.
  • Efficiency in use of resources.
• You may want to assess the impact of your group’s efforts by obtaining feedback not only from those who are directly involved but also from nonmembers who use the group’s activities or products. Possible results to assess include:
  • Cost of producing products or achieving other results.
  • Quality of products or activities.
  • Whether products have been successfully disseminated and used by the target audiences.
  • Programmatic or policy changes resulting from the group’s efforts.
  • Spillover effects to other child health care or QI initiatives.
  • Whether you have support from individuals or organizations that were initially resistant to your initiative.
  • Reduction in the redundancy of QI efforts.

Task 5.2: Select or create a sustainable and user-friendly assessment strategy

The tools and strategies used for assessment can impact both the rate of responses and the quality of feedback you receive. In designing your assessment strategy, questions you may want to ask include:

• What individuals or groups could support the assessment of your group’s impact?
• What information is required to accurately assess impact, and how can you obtain that information?
• How do you balance the need for a thorough assessment with the capacity and interests of respondents?
• Can you use or adapt existing tools, or do you need to create a new tool?

Key Considerations:

• Using a survey is a relatively easy way to gain feedback from your stakeholders and from those who may have been impacted by the stakeholder group. Advantages of using this method include:
  • It is much less time-consuming than one-on-one interviews.
  • It is easier to protect anonymity, allowing respondents to feel secure in providing an honest assessment.
  • There are free or low-cost online platforms for surveys that can make it easier for respondents to complete the survey and for staff to analyze it.
• Although in-person interviews may be more difficult to implement and not anonymous, they may also provide more complete or insightful information and may be able to address more specific concerns than a survey.
• A combination of surveys and interviews can provide you with both broad and in-depth feedback.
• To conserve time and resources, you may be able to use or adapt an existing survey for assessment of stakeholder engagement. Two examples of existing tools include the Coalition Self-Assessment Survey¹² and the Coalition Effectiveness Inventory.¹³
• Use a survey instrument that records both quantitative and qualitative data to identify potential solutions to the issues raised.
  • Quantitative data could be collected with a list of questions to rate on a scale of 1–5. For example, a question could be, “Were the objectives of this group clear to you?” followed by a scale from very unclear (1) to very clear (5).
  • Qualitative data could be collected with brief free-response questions, such as “What worked well? What could be improved?”²
• A relatively quick method of gaining useful feedback from stakeholders is to ask them to complete evaluations immediately after meetings, when the experience is fresh in their minds.
• Assessment of stakeholder engagement also can be done in more informal ways. For example, you can:
  • Conduct exit interviews with stakeholders who leave the group.
  • Compare the number of participants at in-person meetings to those of Webinars to determine the most popular convening mechanism.
  • Gauge continued participation and enthusiasm of stakeholders, a simple measure of stakeholder satisfaction with the process.

Task 5.3: Field the assessment periodically and create a plan to encourage responses

Ideally, you will assess your stakeholder engagement process on an ongoing basis. However, simply fielding the assessment is not enough. You need a plan to make sure stakeholders and others who may have been impacted by the group’s efforts actually complete the assessment. Questions you may want to ask yourself include:

• How often do you need, or can you realistically use, feedback?
• How often should you ask stakeholders and other respondents to complete the assessment?
• How can you incentivize and motivate stakeholders to respond to a survey or other feedback mechanism?

Key Considerations:

• Fielding an assessment at periods throughout the engagement allows midcourse corrections and improvements to the group and its activities. For example, if the group is consistently behind in reaching key milestones, you need to understand why and how to get back on track.
• Based on feedback you receive, you and the group’s leaders may need to reevaluate and refine expectations and engagement supports, such as offering continuing education credits or financial incentives.
• As stakeholders and other respondents have other responsibilities, you might consider providing supports, reminders, and incentives for completion of requests for feedback, including:
  • Giving stakeholders clear directions for providing feedback, including language regarding the confidentiality of their responses.
  • Communicating clearly to stakeholders how the information gained from the assessment will be used to improve the effectiveness of the group and the satisfaction of the stakeholders.
  • Communicating clearly to those who may have been impacted by the group’s efforts how their feedback will be incorporated into the future work of the stakeholder group.
  • Setting a deadline for completing the assessment and sending reminders as that date approaches.
  • Providing additional incentives for responding to requests for timely feedback, such as gift cards.
• An assessment can become burdensome if administered too often, especially if it is long. Alternatively, you can administer the survey at regular intervals, such as quarterly, or timed to key milestones for the group.
• You can then use the results to improve the group’s ability to complete tasks and achieve its goals by discussing the results at a meeting, incorporating the information into a strategic planning process, or setting new objectives.

Return to Contents
Proceed to Next Section