Her effortless writing style makes this book an easy to read and easy to apply field guide.
If you are at all involved in creating learning experiences, tread this book!
Showing posts with label retention. Show all posts
Showing posts with label retention. Show all posts
Saturday, January 15, 2011
Evidence Based Learning Design and a Practical Approach
Ruth Colvin Clark is known for her meta-study to on the ground approach to writing. Her most recent book is the most concise and useful that I've read so far. She busts learning myths with real studies and takes the work out of mapping research to practice with useful guidelines and checklists.
Monday, May 10, 2010
Content Development Tips: Empower SMEs as Partners & (Co)Designers
I would love to think that I have a unique view of program development. Maybe. I have a program that I use with SMEs as I coach them through creating eLearning, ILT, and instructional videos. I have a worksheet for each step.
I start with an open mind and push away all previous content. Most SMEs hate this part. However, this process gets us to the most efficient learning: lowest audience seat time (a.k.a. Cheeks-in-Seats) and lowest audience mental energy (Cognitive Load) for the desired outcomes.
Check out my basic overview and tell me how your process is similar or different.
1. Business Outcomes
What, in business terms, will be different as a result of this training? How will we measure success? What metrics need to improve? Can we measure them today? If not, what do we need to set up in order to measure them? In other words, if the CEO asks us what the company got in exchange for our time on this project, what are we going to show? Why did we pick our outcomes? What are the results?
2. Learning Objectives
What will our audience be able to do after they leave the program? Learning objectives are very formulaic. They have an action verb, an environment, tools, and measurable standards. I avoid the "action" words understand and know. By avoid, I mean ban. We need specific and measurable action words like identify, match, or judge.
3. Activities and Assessments
I try to create one learning activity for the audience to practice what they learned and one test question or test activity for each objective.
4. Content Outline
Content is a scope creep black hole, which is why I don't lead with it. What does the audience need to know to successfully complete the Activities & Assessments? That is the only content we need. I use my objectives as bumper-lanes: SMEs bump up against them less painfully then bumping up against me.
5. Storyboarding
This is a much cheaper and saner way to work.
I shuffle the objectives into a sane order, and I think about your audience. What are they bringing to the table? They can absorb 7 +/- 2 new things at a time before needing a serious break to practice what they learned. If we are teaching someone who hasn't used a computer before about PCs, turning on the machine is at least 1 thing of the allowed 7 +/- 2. Overload they audience, and they will be incapable of remembering anything. An audience can only track 1 or 2 tasks at a time. Period. Incoming email is probably 1 tasks of the two if you are in the eLearning biz. The longest they can sit put & learn without distraction, supervision, or boredom is less than 5 mins.
A storyboard needs to include a box for the learning objective it addresses, a box for what the audience sees, the player controls, a box for narration/script, and a box for assets (the graphics and other media we need to make the screen happen). I include a box for numbering the page. This template will vary depending on the development tool. I make one and copy it. Most programs of about 15 mins need about 20 pages, tops.
I use visual thinking create a storyboard, and I try to include a fun plot or organizing logic. I number the pages when we are done, in case we decide that the ordering needs to change.
This part is very empowering for SMEs who are technophobes or who are not "good with graphics." Advanced stick-figuring is something anyone can master is about 2 mins. I use highlighters to make arrows indicating animation. SMEs let go of bad ideas more easily if they didn't spend 20 hours working on a slide that turns out to be irrelevant over-kill. Plus, they have lots of input into the output, which helps set expectations up front... which saves working relationships and time equally.
If you are a good coach and they want a coach, SMEs will now think you are 1) sane, 2) fun, & 3) easy to work with. They will also think they did all the work.
6. End User Feedback
I like to take the storyboard on tour before investing on the real deal. I want to talk to stakeholders, partners, AND ACTUAL END USERS.
7. Redesign Storyboard
I only like to make high value changes at this point. Some SMEs will quibble about the finer points of taxonomy and nomenclature.
8. Develop
I open PPT or whatever tool we're using and have the best graphics experience of my life. The SME & I already know what we need form ClipArt! We can focus on making the screens look great without having the pressure of composing in what is really a development tool anyway. We all know approximately what the output needs to look like. Sometimes, I'll have the SME take the first pass and give them feedback/redesign tips.
This. Is. PPT. Nirvana.
Record audio. If we are using Jing, we need a hot signal on your mic, and that means the audience will hear a mouse sneezing in the next conference room. At least it's only 5 mins. Or 5 x how many ever times we re-record. Not so bad to do after hours.
For longer programs & eLearning, I prefer the Rode Podcaster. The mic is awesome, but the signal is weak. I use Audacity to boost the levels. Long ago, a hotter mic meant I had to reduce noise in Audacity, making me sound like C3PO's girlfriend. I'll take the boost method and the awesome sound isolation of the Rode Podcaster any day.
Oh, and I drink warm water to avoid smacky sounds. No tea, coffee, or cold water. I use headphones to take eLearning, and hearing smacky noises on headphones is... gross.
9. Publish/Integrate
I publish to my clients' favorite LMS compatible platform (I prefer AICC), or publish the video in whatever format their Intranet will take without being hateful.
For LMS integration, having a flow chart with swim lanes on hand is nice. SMEs really don't know (and likely don't want to know) everything that goes into LMS integration & testing. Yes, testing.
Writing assignment rules can be wither very easy (everyone!) or very painful (sales people whose first name ends with a vowel or engineers who use paper towls instead of the hand drier). I ask the LMS team for a rules tree or something that I can use with SMEs. I talk to the leadership of our audience to make sure they agree with our assignment criteria. They are the ones paying for the time we are holding their valuable workers hostage. They want input.
Establishing what info we can get back out of the LMS is key as well. Question-by-question data may be a problem, but we can get a completion rate. We can usually get a list of overdue or nearly-due people. Handy for nagging & compliance.
For Intranets, I find a job aid for publishing video content pretty useful. This allows SMEs to own minor updates with minimal support from moi. I also give them a heads up: your content is unsearchable, so account for that intelligently with great keywords and some SEO. Unless you have YouTube for enterprise, this will not be a short job aid.
10. Announce
Market! Our audience will not listen to us, mere learning gurus and SMEs, and complete their training unless we hold the keys to their next promotion. I contact the communications team of the leaders we just talked to about assignment rules. Chances are, they are the ones who will write the heads-up email for our audience. I flip the switch on assignment rules a couple hours after this email goes out.
Some other marketing channels are internal social networks, Intranet banners & "ads," and posters. I had a ton of fun pretending to be any of 4 super heroes on a company microblog once. It was a blast.... a corney, attention-getting, blast.
If our video is in place to help the SME stop "answering the same question 20 times a week, I wonder... who asks them over and over? The people who email them! If this is a short Intranet video stuck out in deep space without tags or anything to call it to the audiences attention, there is one sure-fire method to reach this target audience: email signatures. I help the SME create a clever catch phrase and a link. I want the audience to know the content is useful, quick, and painless. For example, "Click here to learn how to talk to Twitter trolls in brand voice without sounding like a marketing clone... in 5 mins or less."
11. Support
Customer service. It happens. Plan for it. Staff for it. Brace for it. Embrace it... even when it's a freakin' pop-up blocker on a browser IT and the LMS vendor never intended. A 2-5% failure rate is normal. If we are using agents, we give them a heads up about volume as well as information for troubleshooting and their escalation points. Also, we try to buy them pizza, candy, or puppies. They deserve it.
12. Evaluate
We ask for feedback from our audience. My standard survey has 5 questions: 3 Likerts on ease, length, and relevance; one open ended on how they will synthesize what they learned; and one open ended question on what we should change in the program. I prepare SMEs for polarized opinions, and I advise that they take everything with a grain of salt, good or bad. Ego and self-loathing are equally annoying.
Not even close to the end of evaluation! (But here is where it usually comes to a grinding halt).
Next, I want to look at the assessments. In the training environment, could our audience perform what we set out to teach them?
Next, we measure to see if the audience is doing what we taught them to do when it really counts: on the job.
Finally, we tie it all to time and cost, and we go back to our business outcomes. I make a 1-2 page dashboard style report out that shows before & after. Now, we can brag about results! We share what we learned, too. We talk about what we discovered during the process and what we 'll do differently in subsequent programs. This highlights how we grew and will grow.
Whew!
Now, my SME is a learning "expert." I am level 2 support. The line outside of my cube is shorter. life it good.
I start with an open mind and push away all previous content. Most SMEs hate this part. However, this process gets us to the most efficient learning: lowest audience seat time (a.k.a. Cheeks-in-Seats) and lowest audience mental energy (Cognitive Load) for the desired outcomes.
Check out my basic overview and tell me how your process is similar or different.
1. Business Outcomes
What, in business terms, will be different as a result of this training? How will we measure success? What metrics need to improve? Can we measure them today? If not, what do we need to set up in order to measure them? In other words, if the CEO asks us what the company got in exchange for our time on this project, what are we going to show? Why did we pick our outcomes? What are the results?
2. Learning Objectives
What will our audience be able to do after they leave the program? Learning objectives are very formulaic. They have an action verb, an environment, tools, and measurable standards. I avoid the "action" words understand and know. By avoid, I mean ban. We need specific and measurable action words like identify, match, or judge.
3. Activities and Assessments
I try to create one learning activity for the audience to practice what they learned and one test question or test activity for each objective.
4. Content Outline
Content is a scope creep black hole, which is why I don't lead with it. What does the audience need to know to successfully complete the Activities & Assessments? That is the only content we need. I use my objectives as bumper-lanes: SMEs bump up against them less painfully then bumping up against me.
5. Storyboarding
This is a much cheaper and saner way to work.
I shuffle the objectives into a sane order, and I think about your audience. What are they bringing to the table? They can absorb 7 +/- 2 new things at a time before needing a serious break to practice what they learned. If we are teaching someone who hasn't used a computer before about PCs, turning on the machine is at least 1 thing of the allowed 7 +/- 2. Overload they audience, and they will be incapable of remembering anything. An audience can only track 1 or 2 tasks at a time. Period. Incoming email is probably 1 tasks of the two if you are in the eLearning biz. The longest they can sit put & learn without distraction, supervision, or boredom is less than 5 mins.
A storyboard needs to include a box for the learning objective it addresses, a box for what the audience sees, the player controls, a box for narration/script, and a box for assets (the graphics and other media we need to make the screen happen). I include a box for numbering the page. This template will vary depending on the development tool. I make one and copy it. Most programs of about 15 mins need about 20 pages, tops.
I use visual thinking create a storyboard, and I try to include a fun plot or organizing logic. I number the pages when we are done, in case we decide that the ordering needs to change.
This part is very empowering for SMEs who are technophobes or who are not "good with graphics." Advanced stick-figuring is something anyone can master is about 2 mins. I use highlighters to make arrows indicating animation. SMEs let go of bad ideas more easily if they didn't spend 20 hours working on a slide that turns out to be irrelevant over-kill. Plus, they have lots of input into the output, which helps set expectations up front... which saves working relationships and time equally.
If you are a good coach and they want a coach, SMEs will now think you are 1) sane, 2) fun, & 3) easy to work with. They will also think they did all the work.
6. End User Feedback
I like to take the storyboard on tour before investing on the real deal. I want to talk to stakeholders, partners, AND ACTUAL END USERS.
7. Redesign Storyboard
I only like to make high value changes at this point. Some SMEs will quibble about the finer points of taxonomy and nomenclature.
8. Develop
I open PPT or whatever tool we're using and have the best graphics experience of my life. The SME & I already know what we need form ClipArt! We can focus on making the screens look great without having the pressure of composing in what is really a development tool anyway. We all know approximately what the output needs to look like. Sometimes, I'll have the SME take the first pass and give them feedback/redesign tips.
This. Is. PPT. Nirvana.
Record audio. If we are using Jing, we need a hot signal on your mic, and that means the audience will hear a mouse sneezing in the next conference room. At least it's only 5 mins. Or 5 x how many ever times we re-record. Not so bad to do after hours.
For longer programs & eLearning, I prefer the Rode Podcaster. The mic is awesome, but the signal is weak. I use Audacity to boost the levels. Long ago, a hotter mic meant I had to reduce noise in Audacity, making me sound like C3PO's girlfriend. I'll take the boost method and the awesome sound isolation of the Rode Podcaster any day.
Oh, and I drink warm water to avoid smacky sounds. No tea, coffee, or cold water. I use headphones to take eLearning, and hearing smacky noises on headphones is... gross.
9. Publish/Integrate
I publish to my clients' favorite LMS compatible platform (I prefer AICC), or publish the video in whatever format their Intranet will take without being hateful.
For LMS integration, having a flow chart with swim lanes on hand is nice. SMEs really don't know (and likely don't want to know) everything that goes into LMS integration & testing. Yes, testing.
Writing assignment rules can be wither very easy (everyone!) or very painful (sales people whose first name ends with a vowel or engineers who use paper towls instead of the hand drier). I ask the LMS team for a rules tree or something that I can use with SMEs. I talk to the leadership of our audience to make sure they agree with our assignment criteria. They are the ones paying for the time we are holding their valuable workers hostage. They want input.
Establishing what info we can get back out of the LMS is key as well. Question-by-question data may be a problem, but we can get a completion rate. We can usually get a list of overdue or nearly-due people. Handy for nagging & compliance.
For Intranets, I find a job aid for publishing video content pretty useful. This allows SMEs to own minor updates with minimal support from moi. I also give them a heads up: your content is unsearchable, so account for that intelligently with great keywords and some SEO. Unless you have YouTube for enterprise, this will not be a short job aid.
10. Announce
Market! Our audience will not listen to us, mere learning gurus and SMEs, and complete their training unless we hold the keys to their next promotion. I contact the communications team of the leaders we just talked to about assignment rules. Chances are, they are the ones who will write the heads-up email for our audience. I flip the switch on assignment rules a couple hours after this email goes out.
Some other marketing channels are internal social networks, Intranet banners & "ads," and posters. I had a ton of fun pretending to be any of 4 super heroes on a company microblog once. It was a blast.... a corney, attention-getting, blast.
If our video is in place to help the SME stop "answering the same question 20 times a week, I wonder... who asks them over and over? The people who email them! If this is a short Intranet video stuck out in deep space without tags or anything to call it to the audiences attention, there is one sure-fire method to reach this target audience: email signatures. I help the SME create a clever catch phrase and a link. I want the audience to know the content is useful, quick, and painless. For example, "Click here to learn how to talk to Twitter trolls in brand voice without sounding like a marketing clone... in 5 mins or less."
11. Support
Customer service. It happens. Plan for it. Staff for it. Brace for it. Embrace it... even when it's a freakin' pop-up blocker on a browser IT and the LMS vendor never intended. A 2-5% failure rate is normal. If we are using agents, we give them a heads up about volume as well as information for troubleshooting and their escalation points. Also, we try to buy them pizza, candy, or puppies. They deserve it.
12. Evaluate
We ask for feedback from our audience. My standard survey has 5 questions: 3 Likerts on ease, length, and relevance; one open ended on how they will synthesize what they learned; and one open ended question on what we should change in the program. I prepare SMEs for polarized opinions, and I advise that they take everything with a grain of salt, good or bad. Ego and self-loathing are equally annoying.
Not even close to the end of evaluation! (But here is where it usually comes to a grinding halt).
Next, I want to look at the assessments. In the training environment, could our audience perform what we set out to teach them?
Next, we measure to see if the audience is doing what we taught them to do when it really counts: on the job.
Finally, we tie it all to time and cost, and we go back to our business outcomes. I make a 1-2 page dashboard style report out that shows before & after. Now, we can brag about results! We share what we learned, too. We talk about what we discovered during the process and what we 'll do differently in subsequent programs. This highlights how we grew and will grow.
Whew!
Now, my SME is a learning "expert." I am level 2 support. The line outside of my cube is shorter. life it good.
Wednesday, April 28, 2010
Cog Research Implications for eLearning: Multi-tasking, CPU Use, and Learning [UPDATE]
eLearning turning into hamburger flipping? Check the box training design is crazy-making enough. Don't be an eLearning zombie-maker. Instead of eating brain power, use your cognitive research knowledge for good.
...zombie learners.
Brrraaaaaiiiiiiinnnnssssss!
- Stop multi-tasking. You are bad at it.
- Your brain's system resources are maxed.
- Your brain runs on exercise. Fuel it.
- Your brain works with pictures.
...zombie learners.
Brrraaaaaiiiiiiinnnnssssss!
Friday, March 5, 2010
Mind Mapping & Active Reading Suggestions for Online Learners
When I'm designing a course like intro art, I understand that very few of my students will go on to careers in art or art history. I try to transfer skills that they can use in life. For example, I'm big on "Technology Literacy" and "Technology Fluency" projects. I also try to give them some pointers the first week of class to help them succeed in online learning and learning in general.
One skill that is important in all learning environments is active reading. Here is my verbatim suggestions to students from Week 1 of Understanding Art, a 16 week online course. I include suggestions for some tools that they can deploy to support their adoption of active reading techniques.
"Beyond a pencil and paper, you can use tools like mind42 and Visio to organize your mind maps. However, hand writing notes usually leads to better learning & retention. Sharing mind maps is a great way to divide & conquer in a study group. For example, make each person responsible for mapping a single paper and leading a discussion board thread on the topic."
Active Reading Suggestions by Susan Burroughs is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
Based on a work at docs.google.com.
One skill that is important in all learning environments is active reading. Here is my verbatim suggestions to students from Week 1 of Understanding Art, a 16 week online course. I include suggestions for some tools that they can deploy to support their adoption of active reading techniques.
"Print these articles and read them actively.
"Active reading helps you understand complex arguments and provides models (and bad examples) that will make you a better writer. Use your understanding of authors in your responses to the Introduction posts. You may cite them casually for now, but you must use some sort of citation. We discuss formal citations next week.
"Active reading includes strategies like using a pencil to note two or three word topic summaries in the margins of reach paragraph, underlining important phrases, and using symbols to note your reactions.
"Invent a system for tracking your opinions of the writing. For example, you can underline a keyword, note a paragraph topic in a couple words in the margins, or use a smiley symbol to note that you agree, :) disagree, :( or are confused :?. For example, you can use the margins to note good (+) or bad (-) evidence.
"Avoid highlighters. They do not help you to track an argument or the organization of a paper clearly, and reviewing the material is difficult. If you like highlighters, try using two techniques at the same time or read once with one method then again with another.
"Another technique for active reading is mind mapping. Check out this video to learn more.
"Beyond a pencil and paper, you can use tools like mind42 and Visio to organize your mind maps. However, hand writing notes usually leads to better learning & retention. Sharing mind maps is a great way to divide & conquer in a study group. For example, make each person responsible for mapping a single paper and leading a discussion board thread on the topic."
Active Reading Suggestions by Susan Burroughs is licensed under a Creative Commons Attribution-Noncommercial-Share Alike 3.0 United States License.
Based on a work at docs.google.com.
Subscribe to:
Posts (Atom)
