If you’ve been following the growth of CSCCE’s online training program over the past few years, you may have noticed that every time we launch something new, we call it a “pilot.” This is because we know that the “v1” of anything needs feedback and iteration to make it better. But what happens after we pilot something? How do we take a training through to a high-fidelity, steady state version that we can deliver consistently time and again?
In this blog post we pull back the curtain on that process of moving to steady state delivery to make visible some of the less obvious infrastructure work we’ve been engaging in over the past few years. And we explain why we’ve made that investment and how it supports us in continuing to deliver trainings that are engaging, effective – and reproducible!
A three-phase release cycle for all CSCCE trainings
While CSCCE is not a software development organization, we do have a founder and director with experience in that space, so one mental model that we’ve adopted over the past couple of years is that of a three-phase release cycle for our courses and Mini-workshops.
Phase 1: Pilot
The cycle for any training starts with a pilot offering (our v1) the first time that we deliver the training. Our pilot offerings are carefully scoped – we create strategy blueprints for all our trainings (more on that in a future post) – but we hold open the possibility that learners may have specific questions about the materials, or offer feedback about their own contexts, that we’d want to incorporate in the final version of the course.
Phase 2: Post-pilot
Based on feedback and our own reflections on the pilot, we then move to a second delivery, the “post-pilot” phase, when various materials are updated, refined, or archived with a goal of getting to the final version of the course once this second delivery is complete. Often, at post-pilot the course experience for learners is the same as at steady state – slides are released in advance and guidebooks and other supporting items are finished and available. Meanwhile, behind the scenes we’re busy perfecting our copy and our materials!
Phase 3: Steady state
Finally, once we’ve completed the post-pilot delivery and done any final tidying up (which typically takes less time than any updates made after the initial pilot), we are able to deliver the training for the third time at “steady state”. This means that all lesson plans, slides and activity instructions are in a final state. So too is the supporting infrastructure – there are template project management cards for the training that can be cloned and used, as well as templates for the pages we’ll use in our learning management system, Canvas.This means we can take a steady state training “off the digital shelf” and deliver it efficiently for our general registration learners or clients.
So, how do we deploy the learning management technology for our trainings? Let’s take a look now at why having this release cycle process is so crucial to syncing all of the activities involved with the delivery of a training…
Canvas LMS: CSCCE’s online learning platform
In the early days of CSCCE, we primarily coordinated our trainings using Google Drive. We created structured folders for learners to share notes and access lesson slides, sent updates and reminders via email, and devised internal-facing spreadsheets to track learner progress. This worked well while we were mostly delivering one multi-week course (Scientific Community Engagement Fundamentals, CEF) and a handful of Mini-workshops.
But, as our course catalogue started to grow, and we began planning out the CSCCE Community Manager Certification Program, it started to feel quite cumbersome for us to administer and sub-optimal in supporting learners as they sought to revisit materials – especially as we made plans for learners to take multiple trainings with us.
Enter Canvas LMS, a Learning Management System commonly used in K-12 and higher education to organize course materials and deliver trainings both online and in-person. Canvas is not the only LMS out there, but after doing some research (and hiring Adrienne Gauthier, a Canvas expert) we decided that it was the platform for us. We set about customizing our own instance of Canvas, designing page layouts to align with CSCCE’s branding, probing the limits of what was possible, and creating a new online space for our learners. (Read more on how we use Canvas in the accompanying post in this series).
Building a training in Canvas
Defining team roles
Having created an overarching design template for CSCCE’s Canvas instance (More on the details of that in this accompanying post), we then had to develop a process for creating individual trainings. How would we describe each week of the course? Who would gather the logistical information (e.g., Zoom and Google Drive links)? What about homework assignments and regular reminders?
We realized over time that we could define various roles associated with course development and delivery – with some team members playing more than one role for any specific course. These roles include:
- Curriculum Designer – who designs the course, the course description, lesson activities and original copy for Canvas
- Tech TA – the technical administrator who adds copy to Canvas, preps notes docs and onboard learners
- Learner Liaison – who communicates directly with learners from when they register to when the course starts, ensuring they have all the scaffolding and information to enjoy their course
- Learning Technologist – who build the course shell the first time a course runs, making any necessary customizations so that the Tech TA can then add copy to the course when it runs
- Canvas Tester – who reviews all prepped pages for a course before it’s published and shared with learners so we can be sure all copy and links are correct
Documentation and process to support us
To manage all of these moving parts, we established a system for creating courses using the project management tool Trello in combination with a collection of Google docs. Essentially, we build the Canvas copy for a training in docs, where we can edit and proofread, and then a “course builder” takes that content and populates a Canvas course shell (a replicable template for a specific course). Before anything goes out to learners, a “course tester” makes sure everything works as expected and conforms to our style guide. Several supporting docs provide coordination of the individual docs containing the copy for each week of the course and keep a log of any changes to our overall templates – and the Canvas playbook we developed provides more detailed support for team members.
In order to deliver a course we need the following set up:
| PREPARATION | A template Trello board for each course – with cards for each week of the course and separate cards for setting up Canvas A set of copy docs for each week of the course where we have template copy for Canvas plus customizable sections for each instance of the course Process docs – Including copy for detailed welcome emails and accompanying logistics sheets and quick start guides |
| DELIVERY | A lesson plan to guide instructors towards learning objectives A Zoom link unique to the training Slides to guide the live sessions Shared virtual notes docs to facilitate learner interactions and guide activities (and templates from which to create these) |
| ADDITIONAL EXTRAS (vary by training) | Worksheets for learner reflections and/or homework Background reading assignments to prepare learners for discussions Additional online tools like Mural and Padlet |
| OFFBOARDING | Exit surveys to understand the learner experience Follow up emails with summaries and reminders about homework |
So how does this work differently at the different phases of the release cycle?
Pilot phase: Creating a new training
When we pilot a new course, chances are we are refining all of the materials related to a lesson or Co-Lab right up until the point of delivery. That’s not to say we’re “winging it” – we create a detailed course blueprint at the very earliest stage of curriculum development (this will be the topic of a future blog post!). It’s more about the fact that each training session requires infrastructure, and when a course is in its pilot phase, we have to build every piece of that infrastructure – from activity instructions that go in the virtual notes docs to Canvas pages to slides and more.
This “just-in-time” nature of pilot training delivery means that there’s a lot going on behind the scenes. To develop a final slide deck, for example, the course instructors make a v1, which is then proofread and accessibility checked by our comms director. The pilot deck is then uploaded to Drive and made available in Canvas by the course Tech TA. After the live session, the course leads make any adjustments to the deck (which may also impact the lesson plan for the session), and a final version is checked and uploaded for learners. And that’s just one slide deck!
Wrapped up in all of this coordinated team work is ongoing quality control. To track the status of all of our training materials, we’ve recently developed an auditing dashboard. Materials are tagged to track their status, and as courses move from pilot to steady state, we use the dashboard to make sure we haven’t missed anything.
In other words, in the pilot phase we are working together as a team to build, refine, and template a new course. It’s highly collaborative and iterative in approach – and for this reason we try to only work on one training in this state at a time!
Steady state: Delivering an established training
At steady state, on the other hand, there’s a lot less behind-the-scenes back and forth, and we have clear roles assigned to team members performing discrete tasks. We have templates for notes docs and onboarding communications. We have a standard slide deck and lesson plan that we know work. And, critically, everything can easily be made available to learners ahead of time via Canvas, with automations to ensure the course runs smoothly.
In this phase, it’s much easier for us to plan a training in terms of discrete phases, with defined roles and responsibilities across the team.
From pilot to steady state: Post-pilot activities
In general, once we’ve run a course once in “pilot phase,” the second offering is “post-pilot,” For learners, the experience of taking the training is much more like a steady state delivery. But for our staff, there is still a little bit of tidying up and refinement going on. For example – if a pilot cohort gave feedback that led us to significantly change the timing of a lesson, or the way we do an activity, that piece of the course will still feel like pilot content.
But, assuming all goes well the second time around, all materials are finalized at this point and a course officially moves to “steady state.”
Three ways that “steady state” is great!
So, that’s a lot of information about what goes into delivering a CSCCE training! But why are we so excited about getting our trainings to “steady state” that we wrote a blog post about it? It’s not just about making it easier for us behind the scenes. It’s also about a better experience for our learners.
i) Advance release of materials to learners – which supports accessibility
At steady state, learners have access to polished materials before live sessions take place, including descriptions of what will be covered, the learning objectives and access to virtual notes, worksheets and slides. This means they can look ahead to explore what’s coming up later in a multi-week course, and reliably receive information and reminders about what’s due as the course runs.
It also supports learner agency in preparing for trainings in a way that matches their needs – whether that’s viewing materials through a screen reader, or taking the time to plan out their upcoming schedule.
ii) High fidelity delivery – learners across cohorts get the same experience
Steady state courses are also much easier for us to “replicate.” In other words, while each cohort of a CSCCE training course is different in terms of learner contexts and their participation, we’re now able to ensure that the core content and scaffolding remains the same.
iii) A flexible foundation from which to build private cohorts and customizations
Lastly, steady state trainings are much easier for us to deliver for our clients. We regularly offer CSCCE Mini-workshops as private training, and now create private cohorts of our courses several times a year. While these trainings occasionally need some customization, it is now very straightforward for us to take a training off the shelf and follow a clear process to deliver it, as needed.
Approaching a big milestone for CSCCE trainings
With all this implemented and part of our regular processes, we’re excited to be fast-approaching a major milestone for our trainings: all our currently active courses (CEF, PBK, CODE and soon PMB) as well as almost all of the Mini-workshops associated with our certification program (a total of 11) are now at steady state!
This has been a multi-year investment in building infrastructure, training staff, and perfecting our offerings that we believe sets us up for years to come. Thanks to everyone over the years who’s given us feedback via exit surveys, focus groups, or who took the time to send us an email and tell us what wasn’t working for them – or what was!
Connect with us about delivering training online
Are you building out a training program for your organization? Or are you having a hard time setting up a new, complicated workflow? We’d love to host a future CSCCE community call on these topics, if there’s interest. Let us know if you’d like to participate in some way: info@cscce.org!