Evaluation

Overhead view of a group dance exercise class in a community centre

We want to understand not just what change has happened. We want to know how and why it happened. We also want to explore how this might differ in different contexts.

In March 2019, the Greater Manchester (GM) Place Partners commissioned Substance and Sheffield Hallam University to evaluate their work, which was called the local pilot. They have used a mix of realist, developmental, and participatory methods. We want to know how five 'enablers of change' can help everyone live active, long lives.

The methodology has three stages. They develop, test, and refine theories for each of the five enablers of change. They follow a cyclical process, repeated over seven times in the past four and a half years.

To test the theories, the evaluation team used a range of methods and data sources. Amongst other things, these included:

  • Embedded researchers to capture qualitative observations of meetings or events.
  • In-depth qualitative interviews with partners and community members on new topics.
  • Extensive "Counts and Measures" spreadsheets:
    • Combine qualitative and quantitative data and work examples about the five enablers of change.
    • Updated every six months.
  • Reflective practice with three steps:
    • What: Describe the story or problem to reflect on.
    • So what: Identify challenges and learnings. What worked and what didn't?
    • Now what: Define next steps to use that learning.
  • Connected to the five enablers and examples from the Counts and Measures.
  • A yearly survey asks local stakeholders to rate the maturity of five 'enablers of change.'
  • Secondary data analysis of the National Active Lives data.

After each cycle, insights from these and other data sources are used to refine the theories. The aim is to learn how different work methods affect outcomes in different places. 

This involves a complex process. It requires synthesising and coding various numerical and rich qualitative data sources. A key part of refining theories was the "sense making sessions." In them, place-based partners discussed their data and outputs with the research team. The discussions included how to scale up, deepen, or expand insights from place-based work. After writing the insight and impact reports, a new cycle would begin. It would develop and test new and adapted theories. 

Since 2019, there has been an annual update on partner perceptions and active lives data. A learning update every six months on the C&Ms, reflective practice, and deep dives has been provided. You can find the process evaluations below.

Read below to learn how systems work, why they support active lives, how to enable change, and what we’re discovering about this approach.

Further details please contact

Enablers of change

For five years, we have been finding ways to get more people moving using a whole system approach.  

We have examined evidence from other whole system approaches to develop our understanding. We have also explored other ways to capture the value in our work. We discussed this with our partners across Greater Manchester. 

We have identified five key conditions needed for sustainable, fair change. These conditions help us work together to promote active lives for everyone. Additionally, they guide our future plans. 

Watch the video (right) to learn the five enablers of change. You can explore each using the links at the bottom of this page.

The focus on the enablers was for two main reasons. It’s still hard to tell if GM Moving partners’ actions are helping, because it’s complicated to link actions directly to results. However, the different types of data we’ve gathered give us confidence. They suggest that these efforts are likely changing the system and making an impact.

The latest Active Lives data is presented alongside. It will assess the impact of GM Moving on population-level physical activity rates. The aim is not to claim a cause from GM Moving's work on physical activity levels in the region. Rather, the figures provide an overview of data that speak to Sport England’s strategic aims. They show recent trends. We know the work is complicated. So, it might take time for local projects to boost physical activity.

Some evidence shows a mismatch. Stakeholders' views on the system's progress don't fully align with local work. It may also help to improve physical activity levels.

To learn more about the system, read below. It explains how it works, why it may support active lives, and what we're discovering from working this way. It covers each part that helps make change possible.