Just over a year ago I was tasked with creating an online algebra 1 course for my school. I’m finishing that up now. One of the portions of the course that I’ve been putting off until the end is assessment. I have already built in a lot of self-checking and automatic digital feedback. I’ve utilized online resources like Khan Academy and IXL where possible. If I was unable to find what I needed, I created practice tasks in Quizizz, Quizlaize, Desmos, and other platforms. The thing that has stumped me the most is formal assessment. This is especially relevant now, teaching during a pandemic.
I think the root of the problem is that some math teachers, myself included, have overly relied on poor/easy summative assessment methods. It’s so simple to write and grade multiple-choice, fill in the blank, and free-response questions that have only one correct answer. Many of my tests were no more than the same types of Khan Academy and IXL questions but on paper. I pondered over how I could create assessments for algebra 1 that could be meaningful. How can I prevent learners from ‘cheating?’ We all know that students are using apps (photo math, mathway, cymath, etc) to solve problems for them. How can I prevent kids from using these tools on an online test measuring DOK 1 skills? How can I prevent kids from having someone else take the online test for them? How can I prevent kids from googling the answers to the test?
The answer is that I can not prevent any of this. These tests will be taken in unsupervised settings. The problem is not the learner, but the assessment tool. Here are a few ideas that I have considered.
1) Have students use a lockdown browser to complete the assessment.
This method would keep an honest student honest. There is nothing to stop the learner from accessing another device (phone, personal laptop) while taking my test on their school-issued device. Additionally, the software that my school uses to monitor and block content can only be used during the regular school day (M-F from 7 am - 3 pm on 180 days from late August through early June). The course is meant to be asynchronous, so there is a lot of time on the clock when I have no ability to monitor students.
2) Set time limits on the test so that students don’t have time to look-up answers.
This would work a bit better, but I don’t love the idea. In the classroom, I’ve always tried to allow plenty of time for all students to finish the assessment. I’ve aimed at writing assessments that I think should take 30 minutes to complete, but allow 90 minutes so that everyone has all the time they need. I’m not a fan of timed tests in general. And of course, there are concerns for IEP accommodations here.
3) Require students to turn on the cameras of their school device and submit a screencast (including their faces) of themselves taking the test.
This is another improvement. Kids could still use other devices and they could edit the video before uploading it. Also, we use free screencasting tools that are limited to 15 minutes in length. So learners would need to submit multiple videos.
So here is what I’m now aiming for. My goal is to create very short formal assessments each week. The process of viewing and providing feedback for these weekly assessments will be very time-intensive, but it should be doable by providing most of the informal assessment in self-checking formats. I’d like to lean heavily on screencasting a single solution, with a time limit of 60 seconds (or maybe two minutes tops). I’m expecting to have as many as 150 learners to provide weekly feedback, so keeping the time limit low will be important.
I’m hoping that for algebra 1, I can find worksheets or textbook problem sets with many problems. I could assign each student a different problem and they could each submit a screencast of themselves showing and explaining their solution. By limiting the time to 1-2 minutes, students would need to rehearse their submissions. I’m thinking that I would be able to get a good idea of their understanding based on these video explanations.
I’ve got some ideas for the more complicated topics and higher DOK levels that I’ll blog about in the near future.
In the meantime, I’d love to hear your ideas for formal assessment in an asynchronous online setting. Share your questions, comments, and compliments :)
Thank you so much for sharing your thinking! I keep thinking about this and then shying away. I'm so curious about people who already do this. My limitations is that in a week, I hope to cover more content than one problem can assess. And 60 or 120 seconds isn't *that* long. And then watching all the videos will take time but it's the feedback that I imagine will take even more time that is in limited supply. But in the back of my mind that this is all my making excuses for not doing it.
ReplyDeleteIf I did go down this road... The one thing I think would be important for me to do is to "train" kids in how to make these video solutions. I can imagine a kid giving an algebraic solution - and just saying "so we add 3 to both sides, we divide by 7, etc." and then saying I'm done! But if my goal is to check understanding, I'd need to make sure to pick good problems -- but also show them ways they can demonstrate understanding in how they talk about their solution beyond giving a list of steps. I'm guessing we'd probably have to look at a few together and I'd show them critiques and commendations.
Good luck with this! I maaaaay join you - though I'm not sure. (One thing I thought of doing is for part of a formal regular written assessment having kids chose to write an answer for a conceptual question OR record a short video.)
Always,
Sameer
Thanks for your thoughts Sameer. I appreciate the opportunity to hone my thinking :)
DeleteI totally agree that it will be challenging or sometimes even impossible to write a one-question formal assessment each week. I’m not sure what the solution is there. This would especially be the case for an algebra 1 class where the pace of the class is pretty quick and many of the topics are unrelated. Some weeks (like week 1 where we cover classifying numbers, ordering numbers, and finding lcm/gcf of monomials) would be tough. I could, however, do this for other weeks. For example by skipping to formal assessment on solving one- and two-step equations and only focusing on multi-step equations with variables on both sides. The assumption being that if kids can prove mastery on the more complex task, they don’t need to show mastery on the simpler tasks.
I also agree with your concern about the feedback. One thing that our LMS does is that we can leave feedback via recorded voice comments. I’ve not tried that yet, but I’m hoping to try it soon. Maybe speaking the feedback will take less time than writing it out. Also maybe there is a way to do some peer-feedback with rubrics here. Although, I’ve had trouble getting good results from that in the past.
I love your ideas for teaching learners how to create screencasts and what makes a good screencast. As always, finding good questions can be challenging. Maybe I could get away with assigning fewer unique questions if I reuse some, but somehow keep kids from knowing who has been assigned which problem during the initial phase of the assignment.
I have one other idea along the lines of your conceptual questions, but I’ll come back to it later in the week when I’ve had more time to flush it out :)
Thanks again for your engagement. I enjoy learning this way!