sep 15, 2015 - MAGIC: Massive Automated Grading in the Cloud
Description:
Authors:
- Armando Fox
- David Patterson
- Samuel Joseph
- Paul McCulloch
Journal:
European Conference on Technology Enhanced Learning 2015 (EC-TEL'15)
Abstract:
We describe our experience developing and using a specific category of cloud-based autograder (automatic evaluator of student programming assignments) for software engineering. To establish our position in the landscape, our autograder is fully automatic rather than assisting the instructor in performing manual grading, and test based, in that it exercises student code under controlled conditions rather than relying on static analysis or comparing only the output of student programs against reference output. We include a brief description of the course for which the autograders were built, Engineering Software as a Service, and the rationale for building them in the first place, since we had to surmount some new obstacles related to the scale and delivery mechanism of the course. In three years of using the autograders in conjunction with both a software engineering MOOC and the residential course on which the MOOC is based, they have reliably graded hundreds of thousands of student assignments, and are currently being refactored to make their code more easily extensible and maintainable. We have found cloud-based autograding to be scalable, sandboxable, and reliable, and students value the near-instant feedback and opportunities to resubmit homework assignments more than once. Our autograder architecture and implementation are open source, cloud-based, LMS-agnostic, and easily extensible with new types of grading engines. Our goal is not to make specific research claims on behalf of our system, but to extract from our experience engineering lessons for others interested in building or adapting similar systems.
Keywords:
automatic grading, programming, software engineering, on-line education
Link:
file:///C:/Users/yeray/Zotero/storage/WLFRBVQP/Fox%20et%20al.%20-%202015%20-%20MAGIC%20Massive%20Automated%20Grading%20in%20the%20Cloud.pdf
Added to timeline:
Automatic Assessment Systems Timeline
Date:
