At CSCCE, one of the ways we support scientific community managers is through professional development training courses and workshops. Our 8-week online course Scientific Community Engagement Fundamentals (CEF) is our longest-running training option, and has reached more than 250 community managers over the past three years across a total of 15 cohorts (with another currently in session).
Two of these CEF cohorts were private offerings for the Chan Zuckerberg Initiative (CZI), and late last year they asked if we could assess the mid- to long-term impact of CEF training on the individuals that took the course. CZI, like us, were particularly curious about the impacts of the course on the learners – as well as the wider impacts on their organizations and communities, and beyond in the STEM ecosystem more generally. And so, in December 2023 we surveyed graduates from 12 general registration CEF cohorts, 2 CZI cohorts, and a cohort that ran in an Australia-friendly time zone.
In this blog post, we share a little more about what we’re hoping to learn as we start analyzing the data, as well as how we’ll be sharing the outcomes of this study in a couple of months’ time. If you have any questions, or you’re interested in engaging us as consultants on your own evaluation program, please reach out to info@www.cscce.org.
About CEF – a cohort-based learning experience
CEF offers STEM community managers a grounding in frameworks that guide engagement strategy and programming, as well as a shared vocabulary to describe their skills and the work they do. In addition to the materials covered, CEF is designed as a social learning experience in which participants make new connections with their peers. Members of each cohort have multiple opportunities to share their experiences as community managers and so guide each other’s learning alongside the CSCCE curriculum.
The cohort-based learning experience is not unique to CSCCE, however it is a model that we have leaned into and refined. As part of this evaluation project, we asked questions to assess the impact of cohort-based learning, and whether the ways that we work to connect learners with similar communities or professional backgrounds has resulted in lasting connections. This could look like peer-to-peer support between individuals, cross-pollination of expertise between organizations (e.g., with one community manager participating in an event led by another) or wider ecosystem-level collaboration such as organizations collaborating on grants or other new community projects.
“We know in the short-term that learners see so much value in working collaboratively with others in their cohort – but now we’re asking how this translates into long-term, ongoing peer support.”
Camille Santistevan, CSCCE Director of Learning.
Evaluating the various impacts of CEF
After each cohort of CEF ends, we distribute a course experience survey to collect feedback from our learners and understand their initial reactions to the course. These surveys have historically been used to shape the course from its initial pilot in the fall of 2020 into the version that we teach today. The feedback from our learners has over and again emphasized that CEF is filling a vital need for a population of professionals who generally lack formal training in their roles.
While this short-term, individual impact is important, our goals for CEF also include shifting the mindset within organizations and the wider STEM ecosystem towards seeing community and community managers as vital, not optional. This new study is an opportunity to assess our progress towards this goal, and ask how the climate might be changing for our graduates.
“With the data from this survey, we’re hoping to further make the case for the professionalization of the role of the community manager in STEM by emphasizing the value of formal training. This includes giving community managers the impact data they need to advocate for such training as part of their professional development.”
Lou Woodley, CSCCE Founder and Director
Sharing our findings
We’ve received survey responses from almost 100 of our learners, and we’ve just started our analysis of the data. We are planning to publish a report of our findings towards the end of March, and hope you’ll join our community call on 17 April 2024 at 11am EDT / 3pm UTC to discuss these findings in more detail.
We’re also excited that we are not the only people currently evaluating cohort-based training in STEM. At our April community call we’ll be hearing from Paz Bernaldo, who’s been working on an evaluation of OLS’s training program for open science advocates, and Jason Williams (Cold Spring Harbor Laboratory) who, along with his collaborators, recently published the Bicycle Principles for short form trainings in STEMM (read about how CSCCE was involved). We hope this will be a rich conversation about how cohort-based trainings have an impact far beyond their trainees. We’ll be sharing more details about the call soon.
We’re grateful to CZI for funding us to undertake this evaluation and share our findings broadly with the community. If you have any questions, or if you’re considering a similar evaluation of your professional development training, please let us know by emailing info@www.cscce.org.