Measures bank: evaluation questions for STEM outreach providers

Key information

The measures bank as created in 2021 from a range of resources including The Engineering Brand Monitor and EngineeringUK’s own student and teacher evaluation surveys.

It has been updated using resources like the Science Education Tracker 2023 and findings from EngineeringUK’s cognitive testing with young people in 2024. 

The measures bank has example questions to ask young people, teachers, and parents. They can be included in, or provide inspiration for, STEM engagement providers’ research and evaluation. The questions come from:

• The Engineering Brand Monitor an annual survey of young people, their parents, and STEM secondary teachers on their knowledge, perceptions and understanding of STEM and engineering
• teacher feedback surveys which collect feedback from teachers whose students have taken part in EngineeringUK programmes
• student evaluation surveys which assess the effectiveness of EngineeringUK’s events and activities to inspire young people to pursue STEM education and careers
• findings from EngineeringUK's cognitive testing with young people 2024. We asked young people to explain their thought process while answering survey questions. This helped us assess how clear and accessible the questions were

The resource is designed to help users consider the outcomes they want to evaluate, and find relevant questions for their own programme evaluation.

It will continue to be updated periodically based on internal reviews, user feedback, as well as measures shared by others across the STEM community.

Key learnings from our cognitive testing, 2024

We interviewed 12 young people aged 10 to 14 for the cognitive testing and asked them to explain their thought process while answering survey questions. This helped us assess how young people interpret our questions, and whether the language used is clear and accessible.

We then piloted the questions at The Big Bang Fair 2024. Students completed surveys after spending at least an hour exploring the event. 

We've updated the young people questions using these key learnings:

1. Tailor questions to content: make sure your questions align with the specific content of your STEM outreach. This means deciding whether to include questions about engineering, technology, or science based on the main focus of your activities. Including all options will lengthen the survey and increase the burden on young people responding, so choose relevant topics carefully. For example, if a programme focuses solely on engineering, avoid technology or science questions, as they may not reflect the programme’s impact accurately
2. Separate STEM fields in questions: address engineering, technology, and science separately in surveys. Responses differ between these fields, so it's important to be specific in your questions to gather accurate data
3. Provide definitions of key terms: defining key terms such as ‘engineering jobs’, ‘technology jobs’, and ‘science jobs’ helps young people understand key terms. The content of STEM outreach needs to be explicit about its definitions of STEM jobs. This helps to accurately measure changes in interest resulting from the activities, rather than relying solely on young peoples' pre-existing perceptions
4. Use ‘jobs’ instead of ‘careers’: a 'job' is more meaningful, especially for younger respondents who might not be thinking about a longer-term ‘career’