April Community Call Recap – The impact of short-form professional development training in STEM

At this month’s community call, we were talking about the impact of short-form professional development trainings – focusing not only on how individuals use what they learned during a training in their day to day work, but also considering how such trainings may result in changes at the level of the STEM ecosystem by affecting common practices and connecting learners across projects and organizations.

The call included an overview of the Bicycle Principles, a framework for designing and evaluation inclusive and engaging trainings, as well as presentations about two different methods for gathering and analyzing impact. 

In this blog post, you’ll find recordings of the three presentations from the call, as well as a brief summary of what each talk focused on. Do join us for our call next month, Wednesday 29 May at 12pm EDT / 4pm UTC, when we’ll be taking a closer look at the application and utility of community playbooks (a.k.a. Collaboration guides, lab handbooks, and more). Add to calendar. 

Three bicycles stand on a set of concrete steps, with long grass on either side. The bicycle in front is pale blue with white wheels, the one behind is white with black wheels, and the one in back is black with yellow wheels.
What do bicycles have to do with short-form training? Read on to find out! Photo by Solé Bicycles on Unsplash

The Bicycle Principles for designing and evaluating short-form training

Jason Williams (Cold Spring Harbor) kicked off this month’s call, reminding us of past research suggesting short-form professional trainings in STEM had little to no impact. Since ongoing training is necessary for researchers to upskill as methods and norms change, this finding highlighted a need to design better trainings, and/or reconsider how to evaluate them. This led Jason and various co-organizers to host a workshop to discuss these challenges, the result of which was the creation of The Bicycle Principles, an iterative framework to support the creation of inclusive and engaging trainings. 

Using reflexive thematic analysis to assess the impact of OLS’ Open Seeds program

OLS’s Open Seeds training and mentoring program, which is about to launch its ninth cohort, is designed to support future leaders in open science. Over the course of 16 weeks, participants in the program receive mentoring and peer feedback as they work on their own personal open science project. Paz Bernaldo is leading an evaluation of the program, using reflexive thematic analysis – a qualitative approach to finding themes and patterns in interview data. Paz focused her presentation on this methodology, since the report summarizing OLS’s findings is still in preparation. You should be able to find it published within the next month or two! 

Measuring impact at three levels of scale – CSCCE’s evaluation of Scientific Community Engagement Fundamentals (CEF)

Scientific Community Engagement Fundamentals (CEF) is CSCCE’s foundational, and longest-running, multi-week online course. Between 2020 and the end of 2023, the period of the assessment, it ran 15 times, training more than 300 learners from 150+ organizations. In their presentation, Lou Woodley and Camille Santistevan explained our motivation for following up with learners months to years after completing the training, and how the study was designed to explore impacts at three different levels of scale (the individual learner, their organization/community, and the STEM ecosystem as a whole). The results from this work are available in a free-to-download report, so Lou and Camille focused their presentation on a high-level overview of some of the positive impacts the study revealed. If you’d like to go deeper, you can download a copy of the full report from our Zenodo repository. And if you’re interested in taking the course, the next general cohort is currently open for registration (deadline, 23 August 2024). 

Thank you!

A big thank you to all of our speakers this month, and to everyone who joined us on the call. We really appreciated the thoughtful discussion of research methods, professional training design and delivery, and how evaluations like OLS and CSCCE’s can be used to refine future training offerings. We’re also grateful to the Chan Zuckerberg Initiative, who funded CSCCE’s medium-term assessment of CEF’s impact. 

Coming up next month

Next month, we’ll be focusing on community playbooks, and how they can be used to support community member engagement through reducing barriers to participation by documenting shared expectations and norms. We hope you’ll join us on Wednesday, 29 May 2024 at 12pm EDT / 4pm ETC to hear from three STEM community managers who recently created their playbooks after taking part in our training course, Creating Community Playbooks (also open for registration!). 

Related resources

A selection of resource shared during the call: 

Bicycle Principles

Thematic Analysis

Scientific Community Engagement Fundamentals (CEF)

Evaluation