A Web-based system for supporting feedback-enabled teaching and learning of computer programming

S Lam, L Yuen, M Y Choy, C K Poon, F L Wang and Y T Yu
City University of Hong Kong
Hong Kong SAR, China

The assessment of programming exercises and assignments by manual inspection of codes on paper is known to be tiring, inefficient and error-prone. Alternatively, students may be required to perform demonstrations or to submit their program source codes so that the instructor can execute the programs one by one. The latter methods allow a more reliable evaluation of the correctness of students' programs, but they are still labour-intensive and time-consuming. More importantly, such a practice is educationally undesirable because students generally have to wait for a long time before they receive feedback from their instructors to help them improve their work.

To address this problem, the authors have developed a Web-based system called Programming Assignment aSsessment System (PASS) for automating the testing of students' programs. Students can now submit their programs directly to PASS, which executes their codes automatically with test cases that are pre-defined by the instructor and students then receive instant feedback on the results of the executions. If the program fails, PASS can pinpoint exactly the position at which its actual output in the test run differs from the correct output according to the program's specification. In addition, the instructor can provide relevant comments to students by annotating each test case with its intended purpose. This kind of advice can help students to understand in which scenarios their programs work incorrectly, thus indirectly giving hints and directions for desirable improvements.

PASS has been used in programming classes since 2003, and has received favourable responses from both students and instructors. It has recently evolved into its third version with a substantial upgrade in its overall performance and usability, together with major enhancements of functionalities to provide a greater variety of educationally useful feedback that facilitates the teaching and learning process.

Specifically, PASS now preserves all students' submissions and can analyse the information to facilitate monitoring of the progress and performance of the whole class, selected groups of students, or each individual student. The computed statistics can be grouped by students to help identify those with outstanding performance, as well as those who consistently lag behind. Also, by grouping the submission results by individual programming problems, the instructor can easily discover the common mistakes made or difficulties encountered by students; and he/she can inspect finer details of the problems by checking the effectiveness of each test case in detecting students' program faults. Temporal statistics, which are all displayed in easy-to-read charts, are also available to view the usage and activity profiles of the students. To promote students' interest in learning, selected statistics can also be disclosed to them so that they can better evaluate their own performance individually, as well as relative to their peers. With this handy pedagogical tool, instructors are seeking to experiment on new ways of constructing programming exercises for continuous improvement in their teaching and learning.