Why Peer Review?

According to a study by Leadership IQ, 26% of new hires fail because they can’t accept feedback (Murphy, 2015). Most students are trained to study for grades and have seldom been given enough training on how to receive feedback and how to make feedback work for them. By the time they enter workforce, they will have a hard time facing feedback from coworkers and supervisors. As instructional designers and instructors, we can help by training students in peer review skills. In doing so, we are preparing our students to be successful in their future career on the one hand; on the other hand, instructors will spend less time grading peer reviewed submissions because of the improved quality of work submitted. It’s a win-win solution. Nothing could get better than this in teaching, right?

How to Create Peer Review Assignments in Canvas?

There are two types of peer review assignments.

Writing assignments with peer review process where peer review effort is not heavily graded. The focus is on improved writing.
Writing assignments with peer review process where peer review efforts is heavily graded. The focus is on training students in peer reviewing.

If your students lack peer reviewing skills, instructors can provide tutorials on how to provide feedback constructively. And instructors can also set up practice assignments where peer review is graded, for at least one or two assignments so that students are given the proper training and practices they need. Here is a video tutorial on how to provide constructive feedback.

If your students have been trained in peer reviewing, I recommend the type of assignments where peer review is not heavily graded. Peer review can be extra credit points, or a small portion of the grade.

To set up peer review in Canvas for an assignment, Log into Canvas course as an instructor/designer, go to the assignment, click “Edit” button to edit the settings for the assignment.

It will greatly help students if you provide clear directions for how you expect students to conduct peer review.

banner image of BA 347 International Business

For example, in BA 347 Research Writing Assignment, the instructor provided the following directions: “Peer Review Feedback guidelines: As you conduct your peer review, remember to praise, criticize appropriately, and be specific with revision strategies.

Identify and describe three strengths in this draft.
Identify and describe three weaknesses in this draft.
How does this draft meet the requirements of the assignment? If not, what is missing?
What should be revised in this writing? Why?
After reading, I was left wondering….”

To set up details for peer review, first we set up a due date for when the draft writing will be due and enter the date in the assignment “Due” area. Secondly, check “Required Peer Reviews” box to enable peer review, and how to assign peer review (manually or automatically), if automatically, enter a number for how many peer reviewers will be automatically assigned for each submission and enter a date for when the peer reviewers will be assigned in the “Assign Reviews” area. Lastly, enter a date in the “Available from … until” area for a “until” date as the date for when the peer review will be due. And explain to your students what these dates mean if this is the first time you assign peer review assignment in your course.

Peer Review Set up in Canvas — How to set up peer review assignment in Canvas. date 1: Individual draft due date; date 2: Peer Reviewers assigned date; date 3: Peer Review due date and assignment becomes unavailable after this time.

Image 1: Canvas Assignment Peer Review Option Set Up

Grading peer review

Once all of the peer reviews have been submitted, if instructors would like to access the actual comments, assess, or add comments of their own, they can do so by going to the speedgrader function. To grade the original submission, simply enter grade point in the Assessment “Grade _____ out of 10” area.

Image 2: Grading Assignment

To grade peer review effort, the instructor would need to set up a separate assignment and name it something like “Peer Review Grade”. Some instructors attach peer review rubric forms so students can attach the forms in the submission for Peer Review Grades. See a youtube video example and its web instructions.

References:

Murphy, M. (2015). Why New Hires Fail. Retrieved on November 17, 2015, from http://www.leadershipiq.com/blogs/leadershipiq/35354241-why-new-hires-fail-emotional-intelligence-vs-skills

Alisa Cooper’s post “Conducting Peer Review Assignments in Canvas” at http://freshmancomp.com/2013/02/19/conducting-peer-review-assignments-in-canvas/

TEXT(S)

There’s been some discussion recently about students and lectures and attention spans.* As conversation about this turned to how students grapple with long form texts in an online course, I thought it might be useful to gather some ideas on course design and working with texts in online courses.

The initial page of the Peterborough Chronicle, marked secondarily by the librarian of the Laud collection. The manuscript is an autograph of the monastic scribes of Peterborough. The opening sections were likely scribed around 1638 (See document). The section displayed is prior to the First Continuation. — The initial page of the Peterborough Chronicle (annotated monastic text)

Structure your course around the texts not the weeks

Some instructors structure their reading intensive course around the books that are read. A module per book with multiple discussion boards and prompts per book. This subtly shifts the focus from what am I doing this week to what is this book doing. For example, in ENG 210 Literatures of the World: Asia, Jeff Fearnside structured the course around the five books they were reading.

Banner depicting a 3D model of the Blue Mosque in Turkey — Banner for the module and book on Turkey

Other course elements allowed summative reflections and integrative questions to address themes throughout the course.

Shift format away from only reading and writing.

If reading, multiple books you might change how students interact with one or more of the books.

In engaging with a text, students might:

watch a play rather than read it
listen to a poem rather than read it (For example, this recording of The Waste Land – poem begins at 16:10)

In responding to a text, students might:

create a video journal or podcast as they progress through text(s)
tweet (or write within 140 characters) summaries of characters or plot themes (similar to the idea of Tweet your thesis)
build a timeline of the narrative (for example, this timeline of Russian History created with student entries for Betsy Ehler’s RUS 233: 20th Century Russian Culture course)

Check for understanding

I’m wary about the idea of reducing student engagement with a text to quizzes but as a feedback mechanism as part of a process and on the way to richer engagement I think quick polls, quizzes, or surveys have a role. You could have quick short post reading quiz or survey to figure out if the students have followed the reading. This would let you respond before the students get further along and further behind.

Other interpretative tools

Pie Charts (Shakespeare) - Link out to a collection of images on flickr analyzing word counts of characters in the plays — Collection of Images Analyzing Word Counts for Characters in Shakespeare’s Plays

There’s a whole range of interpretive tools Digital Humanities tools and engaging with them is well beyond this blog post. However, many projects have available outputs in some form that instructors or students can draw on as they grapple with texts.

A great starting point to explore digital humanities would be to take a look at this overview & contact Jane Nichol the emerging technologies librarian. And as you think through how to use these tools in your course

One example of the type of output you might find is this collection of visualizations of the word counts in Shakespeare’s plays

Collaborative writing and commenting

There are also other ways to dialogue around a text – especially when it’s a short dense text with lots of debate and discussion around the text. There are examples of creating your thesis on a blog as you write or using github or a federated wiki as collaborative authoring tools. However, for many courses google docs (which is integrated into canvas) offers a fantastic, known, and private tool for a course to create a document.

A manifesto for teaching online

As a open experiment let me invite you to engage with a text. The Digital Education team at the University of Edinburgh recently released their 2015 Manifesto for teaching online. There’s a lot to think about in their manifesto and how it works or doesn’t work in our context. Please join in and add your voices this copy for comment – an annotated ‘manifesto for teaching online’. **

*The statement spun off a conversation triggered by the NYT opinion piece on lectures, (which I can’t mention without also noting this thoughtful response)

**If you want to author a new pathway through the text or add resources beyond what comments allow request an account and I’ll add you to the project.

Reflecting

How do you help your students reflect on your course and integrate what they’re learning into their subject knowledge and worldview? If you want your students to develop metacognition and self-understanding, or to articulate professional identity or a disciplinary perspective – reflection and reflective practice can help them integrate what they learn in your course into how they think.

Work found at https://www.flickr.com/photos/cogdog/4502048268/ / undefined

The Theory

The role of reflection in personal development and academic practice is widely acknowledged as a part of higher order thinking in general and also particularly in AACU’s VALUE rubric for Integrative Learning and rubric for Foundations and Skills for Lifelong Learning . The question is how we incorporate reflection in course design.

Adding it in

Adding reflection as a self-contained activity can be a great step, but we often add such activities as small items at the end of a course, or – from the student point of view – as an afterthought or the extra bit they need to do after they’re finished. Reflections done this way, though of potential benefit, can often easily lapse into superficial form-filling. A better approach is to build reflection into the course, and to scaffold student engagement with the process. This can be much more effective and changes how a student interacts with the reflective activity.

Work found at https://www.flickr.com/photos/cogdog/4708291454/ / undefined

An example: Current Problems in Sustainable Living (PS 399)

In PS 399 Current Problems in Sustainable Living (in the future to be offered as PS 374) Dr. Erika Wolters set out to engage students with the issue of their personal role in sustainability within the context of huge global political systems. The course description is as follows:

“Exploration of the role of individuals in sustainability practices and policies. Special focus is given to an examination of how individuals can make sustainable lifestyle choices in light of policy regulations, technologies, socio-economic conditions, and cultural values.”

The Final Paper

Dr Wolters had set set up her course with three major papers alongside other activities and assessments. Originally, the reflective activity was contained in the final paper which required

“By the end of week 2, please select three personal behaviors […] that you will try to change in order to live more sustainably. Document your starting point and each step along the way. Your final paper will require you to discuss your step-by-step attempts where you were successful, where you met with unexpected difficulties, or any other surprises along the way. Place your personal sustainability experience into the context of your readings about individual actions and impacts.“

This paper sought to integrate practice, reflection, and critical disciplinary analysis. As Dr. Wolters and I discussed the course design and how to help students engage with this activity in an online environment, we were aware of two pitfalls to avoid: students reaching the end of the course and struggling to remember their experience and students spending all of their final paper recounting their experience rather than critically engaging with it.

The redesign

The solution we came up with was to ask students to create journal entries throughout the course documenting and beginning to reflect on their practice. In the ten-week course, they identified their sustainable practice by week 2 and journaled about it in weeks 4, 6, and 8 before writing their final paper in week 10. The journal could either be in written or video diary format. There were any number of tools that could be used to support the video option, but using Canvas’ integrated tools and video recorder enabled students to do so easily and without the cognitive overhead of learning an external tool.

The journal could have been set up in a Canvas discussion board. This setup would have created a shared experience across the class in which students reflected and shared together. However, because the focus of this course was personal reflection, the journal activity was set up using the assignment tool. The video or text reflection was shared only with the professor. The reason for doing this was to create the opportunity for more personal reflections than the student might have felt comfortable posting in a forum.

Work found at https://www.flickr.com/photos/cogdog/5570581236/ / undefined

Work in progress

The course is still underway but halfway through I was able to catch up with Dr. Wolters to find out how it was going. Her key observations so far relate in large part to the changes developed through the availabilty of video as an option for this journal activity. She reported the following:

“I do think it is helping them think about the course differently. It is great having them undertake behavior/habit changes and reflect on the costs vs. benefits. It is really exciting to see how they are embracing the project!”
“[I]t is definitely helping me connect with the students differently. I really enjoy seeing and hearing them vs. just having the one-dimensional responses of the discussion boards.“
“[The video posts] were so much fun I responded with a video comment and then posted bi-weekly announcements as a video. It was fun! I definitely feel more connected to the students this way.”

Interim Conclusion

Although this activity needs further evaluation, it illustrates a way to engage students with academic reflection through encouraging dialogue early and throughout the course. From the initial feedback it seems clear that from the instructor’s perspective it offers opportunities to connect with students throughout the course and enable them to engage with the topic.

Image Credits

All images by Alan Levine (Flickr user Cogdog), used under a CC- BY licence.

“We do not learn from experience… we learn from reflecting on experience.”

― John Dewey

Reflection can be a powerful addition to any module or course both for instructors and for students. Instructors can inform themselves about student learning and whether their teaching is effective. Students can deepen their learning through reflection. Reflecting both on knowledge gained as well as areas of confusion can be valuable.

What types of reflective activities are of use in an online course? Two of the simplest activities to incorporate in a course are the Muddiest Point activity or the One Minute Paper activity. Both are short activities in which students answer questions after a brief reflection on their learning.

Muddiest Point:

What concept was the “muddiest” to you during this week, that is, which concept was most unclear?

Minute Paper:

What was the most important thing you learned during this week?
What important question remains unanswered?

Reflection questions can be general or can be more specific. An instructor may want general feedback on a module in the course or they may want students to reflection on a specific field experience, collaborative group project, difficult concept, lecture, reading, etc. Reflective questions can be general or specific.

In the online classroom in which there are many active learning opportunities, adding in extra reflection activities to an already busy schedule can seem overwhelming. One solution to effectively create reflection activities online is to use the Graded Survey option within Canvas (under Quizzes). Canvas will automatically give the student full credit for submitting the survey.

Reflection does not have to add significant time to the student’s workload, does not have to add significant time to the faculty workload, and can teach students the value of reflection which can be applied to their own lives and to their workplace.

Why Modular Course Design

The Course Development & Training team at Oregon State University Ecampus promotes modular course design in our online courses. Laura Crowder (2011) defines modular content as “a collection of learning resources developed as a single learning object”. The major benefits of modular course design include:

Saving time in the development and updating of course content
Modular components are easily repurposed across courses
Student learning is improved since the content is presented in smaller chunks

Modular Course Design in Online Education

Modular course design has been highly recommended by various pioneers in online education. Stephen Downes (Downes, 1998) stated in “The Future of Online Learning” that, “…Online courses will be modular. A course – especially from the designer level – will no longer be seen as a single unit, but rather, as a collection of component parts, each of which may be replaced or upgraded as the need arises.” Andrea Henne (Kelly, 2009) recommended that “modular course design benefits online instructors and students.” The Institute-wide Task Force on the Future of MIT Education Final Report (2014) suggests “an action plan that includes the goals to identify any new or existing MITx course that could be produced as modules; produce the “sticky” modules associated with these subjects; define a limited set of standalone (“smooth”) modules and produce these; put in place a well-organized repository of existing and new modules and define guidelines for building and credentialing customized courses.”

Modular Course Design at OSU Ecampus

With Ecampus’s move to Canvas, modular course design is even easier to implement. Our online courses are generally formatted into 11 weeks as 11 modules. We used a modular course design template for creating each week’s learning content, which includes:

weekly overviews
learning objectives
pre-quiz
assigned readings
lectures
resources
appropriate activities such as graded and non-graded assignments
discussions
assessments
post-quiz
wrap-up

Our template is very similar to Henne’s template (Kelly, 2009), which consists of learning objectives, see table 1 for comparison of the two templates.

Table 1. Comparison of Global Public Health – H 333’s course design template and Henne’s course design template.

This weekly modular template, however, should not limit us from organizing learning content into even smaller units within a weekly module. Here is an example of two modular learning content units within one week in Global Public Health – H 333. The highlighted boxes show two modular content units within Week 1.

Image 1. Screenshot of Global Public Health – H 333 online course Week 1 Learning module

Therefore, if you have a course that has heavy content within each week, feel free to break them into smaller learning modules instead of putting them together as a long big piece.

Enjoy designing and teaching online in Canvas.

References:
Crowder, L. (2011). How to develop modular content in 4 easy steps. retrieved from http://www.learninghouse.com/blog/publishing/how-to-develop-modular-content-in-4-easy-steps on July 28, 2015.

Downes, S. (1998). The Future of Online Learning. Online Journal of Distance Learning Administration, Volume I, Number 3, Fall 1998. State University of West Georgia, Distance Education Center.

Kelly, Rob. (2009). A Modular Course Design Benefits Online Instructor and Students. Faculty Focus. September 2009. Retrieved on July 24th, 2015 from http://www.facultyfocus.com/articles/online-education/a-modular-course-design-benefits-online-instructor-and-students/

Institute-wide Task Force on the Future of MIT Education: Final Report, July 28, 2013, pp. 49–50.

When developing course material for online learning environments—especially narrated presentations—it is important to consider not only the content, but also the design of the material. If material is designed in a way that minimizes visual and cognitive distractions it will be easier for your students to engage with the content.

A particularly effective paradigm in understanding the relationship between content and design can be found in cognitive load theory articulated by John Sweller (1988, 1999). When students are processing learning materials, their total cognitive load is made up of intrinsic cognitive load and extraneous cognitive load. The intrinsic cognitive load is the amount of mental energy required to process the content of the learning at hand. The extraneous cognitive load is the amount of additional mental energy required by the form (design) of the material. Our goal in designing materials should be to minimize the extraneous cognitive load.

Heavy Extraneous Cognitive Load Learning Materials:

Minimal Extraneous Cognitive Load Learning Materials:

Here are a few easy-to-follow principles for designing narrated online presentations (as well as other learning materials) which minimize extraneous cognitive load…

Font

Select fonts that are easy to read. Sometimes we have the tendency to use ornamental or “fun” fonts because we think they will increase engagement. Unfortunately, these fonts increase extraneous cognitive load greatly. The simplest example would be the use of an inappropriate font. Compare the following identical bits of text from Michelle Cook’s article:

Use no more than two fonts in one piece of learning material, and if you do use two fonts, make sure those fonts are quite different, such as a sans serif and a serif font.

Color

Fonts and graphics should both make use of contrast in color. For example, you wouldn’t want to have yellow text on a white background. In addition to causing an increase in extraneous cognitive load, this also causes accessibility issues, especially for your colorblind students.

Avoid the use of more than two colors of text. Sometimes we get the urge to make our slides “pretty” by decorating them with lots of colors. Although this may be a wonderful idea for a work of art, it is counterproductive for narrated slides.

Text

When creating materials such as PowerPoint slides with audio narration, remember that all authority comes from what you are saying. Also, remember the Cook quote above: use the slides to present the visual information while your voice presents the textual (verbal) information. In other words, the slides should have the absolute minimum in terms of text.

Here’s an example of information students learn in MB 480—General Parasitology, created by Sascha Hallett. Note that the text which appears in the first slide becomes narrated content in the second slide:

Non-Narrated Version	Narrated Version

Bullets

Bulleted material is good for documents, but not for narrated presentations. Avoid bullets like the plague. Let’s say you have a slide with five bullet points. What could you do? One possibility would be to create five slides—one for each point. One benefit to this method is that often the text can be eliminated completely and replaced with a visualization (graphic) of the idea you are discussing. Another method would be to use the “SmartArt” function in PowerPoint.

Pictures and Graphics

A picture is worth a thousand words. Therefore, selection of the appropriate picture or graphic is essential. If we use a picture that decorates a slide, the extraneous cognitive load will increase. If we use a picture that illustrates the message of the slide, the extraneous cognitive load will decrease. The ideal, however, is to use pictures that embody the message.

For more ideas on reducing extraneous cognitive load in your narrated presentations, I highly recommend Presentation Zen by Garr Reynolds.

Please share your own tips for effective presentations in the “Leave a Reply” area below! Your insight would be greatly appreciated.

The 2013 Survey of Online Learning Report was released last week on the Sloan Consortium website. This annual series tracks trends in online higher ed in the U.S. through use of institutional data and responses from chief academic officers at colleges and universities nationally. Among key findings of the 2013 survey, three-fourths of chief academic officers think the learning outcomes in online education are the same or better than with face-to-face instruction, but more than 40% of them say that retaining students in online courses is more difficult than in face-to-face courses.

Fortunately, Blackboard has built-in tools to help you monitor student progress. In addition to the Grade Center, check out the Performance Dashboard and Retention Center in the Evaluation section of your Blackboard course Control Panel. These tools can be used with no set up, though you do have the option to customize the Retention Center. The Performance Dashboard gives you a quick overview of each student’s online course activity (for example, days since last Blackboard course access, and level of discussion board participation). The Retention Center provides a more detailed picture of which students may be struggling or at risk in your course. A glance at the Performance Dashboard and Retention Center can give you a heads-up at any point in the term about student engagement and success, so that you can take steps to communicate with students about your concerns and offer strategies for improvement.

Beyond this, make sure your students know about the Ecampus Student Success links, which direct students to Ecampus Success Counselors, online tutoring support, Academic Success (ALS) courses, personalized coaching services from InsideTrack, and other services available to OSU online students.

What strategies do you find most successful in retaining online students in your courses?

There is an exciting new feature in Blackboard which will help instructors provide more detailed feedback in less time. This is the rubrics feature.

What are they?

Rubrics are tables of assessment. Blackboard uses the most common layout, which has columns of proficiency with the greatest levels of achievement on the right moving down to the lowest levels on the left. The rows indicate what is being measured. For example, a piece of writing may be assessed on measures of grammar, structure, clarity, formatting, and citations. Points are attached to each aspect being assessed, with the highest possible points in the right column. Different aspects can have different values. For example, perhaps the focus of this assignment was proper citations, so these would have higher values than grammar or structure, but in another assignment in a future week another rubric could be used in which clarity is the focal point.

Why should I use them?

Measure multiple aspects on one assignment
Save instructor time during grading
Ensures fairness while grading
Guidance for students while completing assignment
Ability to be re-used for multiple assignments

How do I make it happen?

Rubrics can be built right into Blackboard and utilized time and time again. Once you create a rubric, that same rubric can be modified to work for other assignments so there’s not a need to start from scratch. Learn how! (linked) http://oregonstate.edu/instruct/dce/walkthroughs/Rubrics/story.html

Need inspiration?

Here are some examples from the Center for Teaching and Learning to help.

Thinking Rubric (Linked) http://oregonstate.edu/ctl/thinking

Communication Rubric (Linked) http://oregonstate.edu/ctl/communication

Collaboration Rubric (Linked) http://oregonstate.edu/ctl/collaboration

Share your experience with rubrics! Click on “Leave a Reply” below.

Providing feedback to students is a critical component in any course and perhaps even more important in an online course where the instructor and students are not in the same physical space. Although written feedback is the primary method used when providing feedback to students, some instructors are turning to the use of audio feedback and finding that it is both easy to do and effective. Research has shown that audio feedback can allow for more nuanced messages to the student. It has also been shown to involve the student more deeply in a class and make them feel that the instructor really cares. One study even found an association between the use of audio feedback and better retention of course content.

There are several online tools that allow you to create and share audio clips easily. One that I’ve used recently is Chirbit. You only need a microphone and you can record clips up to five minutes in length. There is no limit to the number of audio posts that you can share on Chirbit. Once you create an audio clip you can mark it as private and then share the link that is provided with your student. Chirbit has a number of other capabilities for sharing clips that you can explore even further, including the ability to attach transcripts or QR codes directly to audio clips.

Consider choosing one assignment next term that you could experiment with by providing audio feedback to students. Some instructors have reported that giving audio feedback is actually more efficient for them than giving written feedback. It is definitely another way to extend your presence in the online classroom.

What does the latest research say about the most effective approaches to online and blended learning? Consider adding one or more of these peer-reviewed journals to your summer reading list:

International Review of Research in Open and Distance Learning – The current issue of this twice-a-year journal is a special edition on the hot topic of open educational resources.

Journal of Asynchronous Learning Networks – Articles in the latest issue delve into mobile learning, e-portfolios, and student engagement. JALN is published by the Sloan Consortium, whose website has a wealth of resources about online and blended learning.

Journal of Interactive Online Learning – Recent articles cover learning analytics as predictive tools, the challenge of establishing a sense of community in an online course, and a comparative study of student performance in online and face-face chemistry courses.

Journal of Online Learning and Teaching – The current issue of JOLT (the best journal acronym here!) includes such diverse topics as instructor-made videos as a tool to scaffold learning, comparative usefulness of web-based tech tools in online teaching, and student perceptions of online discussions. JOLT is published by MERLOT, the Multimedia Educational Resource for Learning and Online Teaching, a great collection of peer-reviewed open educational materials that could be useful in your online or classroom teaching.

Ecampus Course Development and Training

Providing inspiration for your online class

Category Archives: Uncategorized

Peer Review: Why and How

Working with Texts in Online Courses

TEXT(S)

Structure your course around the texts not the weeks

Shift format away from only reading and writing.

Check for understanding

Other interpretative tools

Collaborative writing and commenting

A manifesto for teaching online

Building reflection into your course design.

Reflecting

The Theory

Adding it in

An example: Current Problems in Sustainable Living (PS 399)

The Final Paper

The redesign

Work in progress

Interim Conclusion

Image Credits

Reflecting on Learning

Modular Course Design at OSU Ecampus

Why Modular Course Design

Modular Course Design in Online Education

Modular Course Design at OSU Ecampus

Effective Narrated Presentations

Retaining Students Online: Pathways to Success

Rubrics

Can You Hear Me Now?

What Are We Learning about Online and Hybrid Education?

Contact Info