ACM DL

ACM Transactions on

Computing Education (TOCE)

Menu
Latest Articles

An Improved Grade Point Average, With Applications to CS Undergraduate Education Analytics

We present a methodological improvement for calculating Grade Point Averages (GPAs). Heterogeneity in grading between courses systematically biases... (more)

How do Gender, Learning Goals, and Forum Participation Predict Persistence in a Computer Science MOOC?

Massive Open Online Courses (MOOCs)—in part, because of their free, flexible, and relatively anonymous nature—may provide a means for... (more)

NEWS

 

ACM Reappoints Editor-in-Chief

The ACM Publications Board has appointed Dr. Chris Hundhausen, current ACM TOCE editor-in-chief, to a second three-year term, which runs from August 31, 2018 through August 30, 2021. Chris is excited to continue his collaboration with the computing education community on the important work of maintaining, growing, and improving our premier archival publication for computing education research.

Submit Special Edition Proposals

ACM TOCE seeks proposals from researchers and educators interested in guest-editing special issues on topics of interest and relevance to the computing education research community. If you have an idea for a special issue, please email the editor-in-chief.

Special Issues on 'global software engineering' and 'capstones and projects' coming soon.

ACM TOCE will soon publish accepted manuscripts from our two special issues: "Global Software Engineering" and "Capstones and Projects".

Special Issue on machine learning education coming in early 2019

ACM TOCE will publish a special issue on machine learning education in early 2019. Authors interested in submitting articles to this special issue need to submit article abstracts by April 6, 2018, and full papers by June 15, 2018. For more information, please see the call for papers.

ACM TOCE review process is now double-blind 

Authors who submit papers to ACM TOCE should see the paragraphs on double-blind review in the newly-updated author guidelines for details on how to anonymize their papers for submission. Papers that are not properly anonymized will be returned to authors, thus delaying the review process. 

Authors: Submit by 8:00 am on first day of month for fastest review 

ACM TOCE's monthly peer review process begins on the first day of each month. On this day, we begin reviewing all papers received by our "soft" deadline:  8:00 a.m. U.S. Pacific Time on the first day of the month. If you want your paper to be reviewed as quickly as possible, submit it just before our monthly "soft" deadline.

About TOCE

The ACM Transactions on Computing Education (TOCE) publishes high quality, peer-reviewed, archival papers in computing education. Papers published in TOCE take a scholarly approach to teaching and learning, establish a clear connection to student learning, and appeal to a broad audience interested in computing education: instructors, researchers, curriculum designers, and administrators.  READ MORE

Forthcoming Articles

Multiple Choice Questions in Programming Courses: Can We Use Them and Are Students Motivated By Them?

Low performance of engineering students in programming courses is a problematic that remains unsolved. Over the years, many authors have tried to identify the multiple causes for that failure, but there is unanimity on the fact that motivation is a key factor for the acquisition of knowledge by students. To better understand motivation, a new evaluation strategy has been adopted in a second programming course. Comprising 91 students, the goals of the study were to identify if those students felt more motivated to answer multiple-choice questions in comparison to development questions, and what type of question better allows for testing student knowledge acquisition. The results show that students felt more motivated to answer multiple-choice questions and overall obtained 8.5% better performance in those questions when compared to development questions. Additional analysis related to students performance was also performed, targeting gender and first enrollment as differentiating factors. Regarding to gender, the analysis did not show any kind of relation; however, the analysis revealed a correlation between first enrollment status and performance, and also between students who obtained success in the first programming course and performance. In conclusion, it is clear that student performance varies according to the type of questions. Our study points out that multiple choice questions can be seen as a motivational factor for engineering students and it also might be a good way to test acquired programming concepts. Because of that, this type of questions should be included in evaluation points.

A Robust Machine Learning Technique to Predict Low Performing Students

As enrollments and class sizes in postsecondary institutions have increased, instructors have sought automated and lightweight means to identify students who are at risk of failing a course. This identification must be performed early enough in the term to allow instructors to assist those students before they fall irreparably behind. This study describes a modeling methodology that predicts student final exam scores in the third week of the term by using the clicker data that is automatically collected for instructors when they employ the Peer Instruction pedagogy. The modeling technique uses a support vector machine binary classifier, trained on one term of a course, to predict outcomes in subsequent terms. We applied this modeling technique to five different courses across the computer science curriculum, taught by three different instructors at two different institutions. Our modeling approach includes a set of strengths not seen wholesale in prior work, while maintaining competitive levels of accuracy with that work. These strengths include being lightweight, affording early detection of struggling students, and predicting outcomes across terms in a natural setting (different final exams, minor changes to course content), across multiple courses in a curriculum, and across multiple institutions.

A Systematic Literature Review of Automated Feedback Generation for Programming Exercises

Formative feedback, aimed at helping students to improve their work, is an important factor in learning. Many tools that offer programming exercises provide automated feedback on student solutions. We have performed a systematic literature review to find out what kind of feedback is provided, which techniques are used to generate the feedback, how adaptable the feedback is, and how these tools are evaluated. We have designed a labelling to classify the tools, and use Narciss' feedback content categories to classify feedback messages. We report on the results of coding 101 tools. We have found that feedback mostly focuses on identifying mistakes and less on fixing problems and taking a next step. Furthermore, teachers cannot easily adapt tools to their own needs. However, the diversity of feedback types has increased over the last decades and new techniques are being applied to generate feedback that is increasingly helpful for students.

Students' Experience of Participation in a Discipline -- A Longitudinal Study of Computer Science and IT Engineering Students

Research suggests that the notion of social identity can be useful to investigate learning in a broader sense and to better understand the problem of retention, low engagement and diversity in science, technology, engineering, and mathematics (STEM) education. There is little prior work in computing and engineering education research that builds on theories of social identity. This article summarises the findings of a longitudinal study exploring identity development in computing higher education. Students of two computing study programmes, computer science (CS) and IT Engineering (IT), were followed over a three-year period. In semi-structured interviews, the students reflected on their engagement in CS/IT. Phenomenographic analysis produced an outcome space describing increasingly broad ways in which students experience participation in CS/IT, i.e. engaging in doing, thinking, feeling, in relation to CS/IT, and in relation to other people. Experiences described by two categories were of increasing relevance as the students proceeded in their studies, participation as problem solving and problem solving for others. Another result is two outcome spaces providing nuanced descriptions of experiences that fall into these two categories. These results give us new insight into the students reflections on their engagement in CS/IT over the years. Participation as described by the category problem solving becomes the predominant way of participating in CS/IT, as is experienced by the students. Social aspects, introduced in the category problem solving for others, become more visible among third year students. However, some third year students express scepticism for less-technical ways of engaging in CS/IT.

Searching for Employability: Can Students Capitalize on Enabling Learning Environments?

Literature on employability signifies "enabling" learning environments where students encounter ill-formed and open-ended problems and are required to adapt and be creative. Varying forms of "projects" have populated computing curricula for decades and are generally deemed an answer to this call. We performed a qualitative study to describe if and how project course students are able to capitalize on the promise of enabling learning environments. This critical perspective was motivated by the circumstance of the present-day education systems being heavily regulated for the precipitated production of human capital. The students involved in our study described education system-imposed and group-imposed narratives of lost opportunities, as well as many self-related challenges. On the other hand, students welcomed autonomy as an enjoyable condition and linked it with motivation. Self-related attitudinal attributes such as taking care of one's own learning and whole-group commitment appeared as important conditions. The results highlight targets for interventions that can counteract unfruitful behaviors and continue the march of projects as a means to foster professionalism in global software engineering.

Building LEGO Towers: An Exercise for Teaching the Challenges of Global Work

Global software engineering has changed the way software is developed today. To address the new challenges, many universities have launched specially tailored courses to train young professionals to work in globally distributed projects. However, a mere acknowledgment of the geographic, temporal, and cultural differences does not necessarily lead to a deep understanding of the underlying practical implications. Therefore, many universities developed alternative teaching and learning activities, such as multi-university collaborative projects and small-scale simulations or games. In this paper, we present a small-scale exercise that uses Lego blocks to teach skills necessary for global work. We describe the many different interventions that could be implemented in the execution of the exercise, and share the results from two runs of the exercise with a group of international students. Our results suggest that the exercise can be a valuable tool to help students dealing with troublesome knowledge associated with global software engineering and a useful complement to the courses dedicated to this subject.

How Else Should It Work?

In order to apply their computing knowledge to everyday situations and systems, students first need to be able to identify those situations in which computing even plays a role. This task becomes increasingly difficult, however, as computing systems become more and more ubiquitous and invisible. Based on the analysis of 28 semi-structured focus interviews, we present a grounded theory of students' conceptions and reasoning related to the identification of computing within technical devices. At its core is the finding that students seem to differentiate technical artifacts with respect to three conceived levels of capability. Many household appliances, for instance, may very well be seen as electronic and programmed, but still as too limited in their capability to warrant the presence of a real computer or to be related to informatics. Given the increasing versatility and power of modern embedded systems and the advent of the internet of things, this issue should clearly be addressed. Based on our grounded theory, we propose some first ideas for how this might be done.

Does Computer Game Design and Programming Benefit Children? A Meta-Sythesis of Research

It is widely believed that there are educational benefits to making computer games, but there is no systematic review of research on this topic. This paper describes a meta-synthesis of research on children designing and programming computer games to determine the extent to which there is evidence of benefits for learning and motivation. Over four hundred articles were identified, and eighty-four articles met the inclusion criteria. A systematic analysis and synthesis across studies showed some evidence that computer game design and programming can lead to changes in programming knowledge, problem solving, computer science attitudes and confidence, and other academic knowledge. However, most of the evidence described engagement in computing-related practices and did not measure learning. The findings were similar across different pedagogical approaches. In addition, the synthesis resulted in a list of design elements for studying computer game design and programming activities; these can be used to increase the availability of evidence about learning. The article concludes with the identification of gaps in the research and suggestions for additional research.

Digital and Physical Fabrication as Multimodal Learning: Understanding Youth Computational Thinking When Making Integrated Systems Through Bidirectionally Responsive Design

This paper proposes and explores the kinds of computational thinking, creative practices, design activities and equitable opportunities provided to diverse high school youth when designing simultaneously physically and digitally responsive wearable games. Previous work in this area coined the term bidirectionally responsive design, or the design of dual-feedback systems using multiple interfaces, which are both digital and physical, utilizing simplified digital and physical toolkits common in makerspaces. While some initial studies have explored the computational concepts, practices and perspectives involved in coding and circuitry, particularly with Scratch and e-textiles, this study extends this work by providing detailed analysis of one case study of a bidirectionally responsive project produced during one of the workshops. This study provides a framework to analyze computational concepts, practices and perspectives in the bidirectionally responsive project adding to, building upon and integrating previous analytical approaches to looking at Scratch coding, physical computing and e-textiles. Utilizing a detailed case study of one such youth team and project, we conduct a multimodal analysis of bidirectionally responsive making activities to discuss the ways they present novel understanding of integrating diverse interests, and fostering collaborative and distributed computational thinking.

The effects of adding non-compulsory exercises to an online learning tool on student performance and code copying

This study evaluates the impact of a new review-exercises module that has been incorporated into an online tool in a software engineering degree with the goal of avoiding plagiarism by enhancing study commitment. Two groups of students were quantitatively compared: the former used the tool exclusively during lab sessions, whereas the latter had the possibility of employing the tool for enhancing their study. The tool allows us to collect interesting data related to the focus of this study: supplementary work completed voluntarily by students and the percentage of students copying others work during lab sessions. In the obtained results, we have observed that the students in the enhanced study group plagiarized less and achieved better academic results. Moreover, in the enhanced study group the students that copied less and put in more effort, got better results; and, interestingly, the effort invested in completing review exercises did not seem to compensate for the learning effort avoided by copying others exercises during lab sessions. The results show the advantages of a tool used with a dual orientation: compulsory and voluntary. Mandatory usage in lab sessions establishes some milestones that, eventually, act as an incentive fostering learning, while voluntary use reinforces students perception of the tool usefulness to achieve learning.

Taking a Studio Course in Distributed Software Engineering from a Large Local Cohort to a Small Global Cohort

One of the challenges of global software engineering courses is to bring the practices and experience of large geographically distributed teams into the local and time-limited environment of a classroom. Over the last six years, an on-campus studio course for software engineering has been developed at The University of Queensland that places small teams of students on different features of a common product. This creates two layers of collaboration, as students work within their teams on individual features, and the teams must interoperate with many other teams on the common product. The class uses continuous integration practices and predominantly asynchronous communication channels (Slack and GitHub) to facilitate this collaboration. The original goal of this design was to ensure that students would authentically experience issues associated with realistically-sized software projects, and learn to apply appropriate software engineering and collaboration practices to overcome them, in a course without significant extra staffing. Data from the development logs showed that most commits take place outside synchronous class hours, and the project operates as a temporally distributed team even though the students are geographically co-located. Since 2015, a course adapted from this format has also been taught at the University of New England -- an Australian regional university that is also Australia's longest continuous provider of distance education. In this course, most students study online, and the class has to be able to work globally, because as well as students taking part from around Australia, there are also typically a small number of students taking part from overseas. Transfering the course to a smaller but predominantly online institution has allowed us to evaluate the distributed nature of the course, by considering what aspects of the course needed to change to support students who are geographically distributed, and comparing how the two cohorts behave. This has produced an overall course design, to teach professional distributed software engineering practices, that is adaptable from large classes to small, and from local to global.

Assessing Students' IT Professional Values in a Global Project Setting

This exploratory study examines the development of an instrument for assessing dispositional knowledge in the context of global software engineering. It focused on the development and use of a set of instruments for assessing affective domain student learning of global Information Technology (IT)/Software Engineering (SE) professional values. The effort needed to generate the questionnaires was relatively low, and were deployed to students and alumni from an open-ended global software engineering project course. The project included making explicit the IT professional values of interest among the participating faculty in the form of actionable value statements. It also included validation of these statements with an expert panel as question roots, and the use of these questions to investigate student and alumni receiving, responding and valuing of these professional values. Students responding reported significant agreement receiving these global values, but more mixed responses in responding and valuing them. The effort helped identify several actionable values worth reinforcing in future course offerings.

A Fringe Topic in a Fragile Network: How Digital Literacy and Computer Science Instruction is Supported (or not) by Teacher Ties

In this NSF CSforALL funded research study, the authors sought to understand the extent to which an urban districts teacher instructional support network enabled or constrained access to social capital and their capacity to implement and diffuse DLCS instructional practice throughout the K-12 curriculum. Social network analysis was used to investigate informal teacher advice-seeking and advice-giving patterns of DLCS support. Network measures of cohesion and centrality were computed. Findings revealed that DLCS focused teacher support networks tend to exhibit very low density, have relatively few ties, include a high number of isolates (teachers with no connections), and centralize around a particular actor. In addition, results revealed that teacher sense self-efficacy (a belief in their own ability to implement DLCS instruction) was significantly lower than their self-efficacy related to the implementation of more general instructional practices. Overall, study findings suggest that district capacity for the implementation of K-12 digital literacy and computer science curriculum and instruction may be quite low. Authors conclude that examining and strengthening teacher access to social capital and networks of instructional support may be a crucial step for educators concerned with school improvement and the diffusion of digital literacy and computer science curricula in US schools. CCS Concepts: " Human Centered Computing ’ Collaborative and social computing design and evaluation methods ’ Social Network Analysis; Social and Professional Topics ’ Computing Education ’ Computing literacy, K-12 education

Assessing the Impact of the Distributed Software Development Course on the Careers of Young Software Engineers

Various Software Engineering curricula in Higher Education started including courses on Global Software Engineering, carried out as international distributed project-based courses. These courses, known for their closeness to real world work experience, emphasize the importance of involving industry partners as customers and focus on soft skills important for employment, often neglected in engineering education. However, not many such courses are long-lived or of a consistent form throughout the years, making their impact and relevance hard to assess. The Distributed Software Development course (DSD), currently run among three universities from Croatia, Italy and Sweden, is now carried out for 14 years consecutively, providing a rich source of in-course and post-graduation data. In order to evaluate the students experiences of the course after they graduate and start working, a survey among former DSD students from University of Zagreb, Croatia has been carried out, to understand how useful this course was in students careers, both in first and current employment, depending on several factors during the course (such as project role), as well as factors at the workplace (such as the level of global teamwork). The survey results show the relevance of such distributed course experiences for future employment, as well as importance of building up on soft skills as part of the engineering curricula. Higher Education institutions are invited to consider including such courses in the Software Engineering curriculum, to the benefit of their students and, indirectly, their future employers.

Managing diversity in distributed software development education - a longitudinal case study

Teaching Distributed Software Development with real distributed settings is a challenging and rewarding task. Distributed courses are idiosyncratically more challenging than standard local courses. We have experienced this during our distributed course, which has now been run for 14 consecutive years. In this paper, we present and analyze the emerging diversities specific to distributed project-based courses. We base our arguments on our experience and we exploit a three-layered distributed course model, which we use to analyze several course elements throughout the 14-years lifetime of our distributed project-based course. In particular, we focus on the changes that the course underwent throughout the years, combining findings obtained from the analyzed data with our own teaching perceptions. Additionally, we propose insights on how to manage the various diversity aspects.

Exploring and Expanding GSE Education with Open Source Software Development

Global software engineering (GSE) courses traditionally require cooperation between at least two universities so as to provide distributed development environment to the students. This study presents an alternative way of delivering such courses by incorporating open source software development (OSSD) in the absence of multiple universities alliance. The results show that the new setup provides challenges similar to multi-site software development as well as challenges associated with software development outsourcing. The present paper further compares a combined GSE and OSS course against a traditional GSE course in terms of students learning outcomes and course organization. The authors found that a combined GSE and OSS course provides similar learning opportunities to the students as a traditional GSE course, but with the added benefit of relative ease of organizing the course within a single university setting.

An empirical investigation on the benefits of gamification in programming courses

Context: Programming courses are mandatory in most Engineering degrees, and students performance in these courses is all too often not as good as expected. Programming is difficult for students to learn, given that it includes a lot of new, complex and abstract topics. All this leads us to acknowledge that fresh teaching techniques are needed if students are to be motivated and engaged in learning in programming courses. Gamification has come to be an effective technique in education in general, and it is especially useful in programming courses. This fact motivated us to develop a gamified platform, called UDPiler, for use in a programming course. Objective: The main goal of this paper is to obtain empirical evidence on the learning effectiveness of UDPiler in comparison to a non-gamified compiler. Method: A quasi-experiment was performed with two groups of first-year Engineering students at Diego Portales University in Chile, using a non-gamified compiler and a gamified platform respectively. Moreover, a post-experiment questionnaire was subsequently handed out, in order to obtain qualitative data about the gamified platform. Results: The results reveal that the students obtained better marks when the gamified platform for learning C programming was used. In addition, there is statistical significance in favor of a positive effect on the learning effectiveness of the students who used the gamified platform. Conclusions: The results allow us to conclude that gamification is an encouraging approach for teaching C programming. This finding is aligned with previous empirical studies about gamification, carried out in academic contexts.

Comparing Computing Professionals' Perceptions of Importance of Skills and Knowledge on the Job and Coverage in Undergraduate Experiences

The purpose of this paper is to discuss the findings of a survey of nearly 300 computing professionals who are involved in the design and/or development of software across a variety of industries. We report on the surveyed professionals perceptions of the importance of a range of topics and skills, and the degree to which recent graduates felt that each topic or skill was stressed in their undergraduate experience. Our findings highlight the value of breadth and flexibility in technical skills, and the universal importance of critical thinking, problem solving, on-the-job learning, interpersonal skills, and the ability to work well in cross-disciplinary teams. However, the recent graduates we surveyed report inconsistent coverage of these most important topics in their degree experiences. We discuss implications for education and for future research.

RecurTutor: An Interactive Tutorial for Learning Recursion

Recursion is one of the most important and hardest topics in lower division computer science courses. As it is an advanced programming skill, the best way to learn it is through targeted practice exercises. But the best practice problems are hard to grade. As a consequence, students historically have completed only a small number of recursion programming exercises as part of their coursework. We present a new way for teaching such programming skills. Students view examples and visualizations, then practice a wide variety of automatically assessed, small-scale programming exercises that address the sub-skills required to learn recursion. The basic recursion tutorial (RecurTutor) teaches material typically encountered in CS2 courses. Experiments showed that RecurTutor supports recursion learning for CS2 level students. Students who used RecurTutor had significantly better grades on recursion exam questions than did students who used typical instruction. Students who experienced RecurTutor spent significantly more time on solving programming exercises than students who experienced typical instruction, and came out with a significantly higher confidence level.

Evaluating GSD-Aware: a Serious Game for Discovering Global Software Development challenges

Global Software Development (GSD) is a strong industry trend at the present time. This means that if computer science engineers are to be trained with respect to this paradigm, it is very important to include the topic in software engineering courses, attempting to ensure that students learn about GSD and become familiar with its advantages and challenges. However, software engineering curricula courses do not always consider this paradigm. It must also be recognized that it is also difficult to find a suitable method with which to teach/develop the different skills that are advisable to have in GSD, given that it is frequently the case that there is a lot of content and not a great deal of time available to deal with it all. In this paper we propose the use of a serious game called GSD-Aware, with which students can suffer some of the typical challenges of GSD by interacting with avatars and by using several means of communication to solve some given problems. The paper focuses on the empirical validation of the game, for which an experiment was conducted, the aim being to analyze whether the game helps students to be aware of GSD challenges. It was discovered that after playing the game the students gave the same degree of importance as before playing it to these factors: lack of communication, lack of experience in the use of communication tools, and language difference. On the other hand, after playing the game the students were aware of the greater influence that the following factors can have: lack of coordination, trust, cultural differences, lack of face to face and informal communication, time difference and lack of team spirit. In their final analysis, students agreed that the serious game scenario helped them to understand the importance of GSD challenges. As a result, it can be concluded that the serious game seems to be an appropriate tool to help teach GSD features, as these games are often more affordable and entertaining than other traditional methods. Data indicate that by using GSD-Aware students can in a short period of time realize the importance of different factors that influence GSD projects. This method may be a better way of explaining these factors to them than by means of explanations given from a merely theoretical point of view.

Second Special Issue on Learning Analytics in Computing Education

All ACM Journals | See Full Journal Index

Search TOCE
enter search term and/or author name