evaluation Archives

How does CSCCE online training impact community managers and their organizations? Read our report to find out!

Thanks to funding from the Chan Zuckerberg Initiative, we recently completed a medium-term evaluation of our foundational training course, Scientific Community Engagement Fundamentals (CEF). We’ve just published a report that summarizes the results of this work, which shows impact across three levels of scale – the individual, their community/organization, and the wider STEM ecosystem.

In this blog post, we’ll recap some of the rationale for the report and a high level overview of our findings. Subsequent posts will share more about our user-centered design approach to creating professional training courses, what we learned about the impact of CEF at each level, and how this work will impact our ongoing training offerings.

DOWNLOAD THE REPORT

April Community Call Recap – The impact of short-form professional development training in STEM

At this month’s community call, we were talking about the impact of short-form professional development trainings – focusing not only on how individuals use what they learned during a training in their day to day work, but also considering how such trainings may result in changes at the level of the STEM ecosystem by affecting common practices and connecting learners across projects and organizations.

The call included an overview of the Bicycle Principles, a framework for designing and evaluation inclusive and engaging trainings, as well as presentations about two different methods for gathering and analyzing impact.

In this blog post, you’ll find recordings of the three presentations from the call, as well as a brief summary of what each talk focused on. Do join us for our call next month, Wednesday 29 May at 12pm EDT / 4pm UTC, when we’ll be taking a closer look at the application and utility of community playbooks (a.k.a. Collaboration guides, lab handbooks, and more). Add to calendar.

Three bicycles stand on a set of concrete steps, with long grass on either side. The bicycle in front is pale blue with white wheels, the one behind is white with black wheels, and the one in back is black with yellow wheels. — What do bicycles have to do with short-form training? Read on to find out! Photo by Solé Bicycles on Unsplash

April’s Community Call: Evaluating the impact of short-form training in the STEM ecosystem

On our April Community Call, we’ll be focusing on how to evaluate the impact of professional development trainings on individual participants, their organizations, and the STEM ecosystem as a whole.

Evaluation is something that we’ve been doing more and more at CSCCE in our client work – capturing the value created in various community programs and proposing improvements for future interactions. We’re especially interested in programs that support group-based learning in some way such as those that provide training and/or mentorship for community champions.

Over the last few months, thanks to funding from the Chan Zuckerberg Initiative, we’ve been turning the spotlight back on one of our own learning activities, by researching the impact of our foundational training course in community management, Scientific Community Engagement Fundamentals (CEF). At the same time, Open Life Science (OLS) have been conducting an evaluation of their Open Seeds cohort-based training and mentorship program. These evaluations have taken place on the backdrop of an ongoing conversation about how to measure the impact of short-form trainings in the life sciences in general, thanks to the work of Jason Williams and Rochelle Trachtenberg.

Evaluation project underway to assess the impact of CSCCE’s foundational training, Scientific Community Engagement Fundamentals

At CSCCE, one of the ways we support scientific community managers is through professional development training courses and workshops. Our 8-week online course Scientific Community Engagement Fundamentals (CEF) is our longest-running training option, and has reached more than 250 community managers over the past three years across a total of 15 cohorts (with another currently in session).

Two of these CEF cohorts were private offerings for the Chan Zuckerberg Initiative (CZI), and late last year they asked if we could assess the mid- to long-term impact of CEF training on the individuals that took the course. CZI, like us, were particularly curious about the impacts of the course on the learners – as well as the wider impacts on their organizations and communities, and beyond in the STEM ecosystem more generally. And so, in December 2023 we surveyed graduates from 12 general registration CEF cohorts, 2 CZI cohorts, and a cohort that ran in an Australia-friendly time zone.

In this blog post, we share a little more about what we’re hoping to learn as we start analyzing the data, as well as how we’ll be sharing the outcomes of this study in a couple of months’ time. If you have any questions, or you’re interested in engaging us as consultants on your own evaluation program, please reach out to info@www.cscce.org.

July Community Call Recap: Organizing inclusive, accessible, and successful virtual events

This month, our content and programming focused on organizing and implementing virtual events. With the global COVID-19 pandemic, virtual meetings, conferences, and other events have become part of everyday life for many people, and the task of planning, executing, and evaluating them in STEM often falls to community managers. So, for our July community call we invited three members of our community of practice to share their knowledge and start a conversation about best practices.

Watch the three presentations from July’s call in their entirety.

Presentations

“It’s Dangerous To Go Alone, Take This – Non-Player Characters & Prepping For Your Virtual Event” – Tom Quigley, ConservationXLabs (slides)
“It’s All About Access: Planning Meetings for Wider Audiences” – Rebecca Carpenter, Deaf and Hard of Hearing Virtual Academic Community (slides)
“Evaluating Virtual Events” – Emily Lescak, Code for Science and Society (slides)

July’s Community Call: Planning and evaluating accessible virtual events

On Wednesday, 22 July 2020 at 2pm we’re hosting our next monthly community call. This month’s call will focus on virtual events, a topic that is likely on the minds of many scientific community managers at the moment.

We’ll cover three key aspects of organizing virtual events: planning and preparation, access and accessibility, and evaluation, both before and after your event. With three experts from our community of practice presenting, and ample time for discussion and Q&A, this month’s call promises to provide actionable information for you and your colleagues, so we hope to see you there!