ACM Transactions on

Computing Education (TOCE)

Latest Articles

An Improved Grade Point Average, With Applications to CS Undergraduate Education Analytics

We present a methodological improvement for calculating Grade Point Averages (GPAs). Heterogeneity in grading between courses systematically biases... (more)

How do Gender, Learning Goals, and Forum Participation Predict Persistence in a Computer Science MOOC?

Massive Open Online Courses (MOOCs)—in part, because of their free, flexible, and relatively anonymous nature—may provide a means for... (more)

Transfer-Learning Methods in Programming Course Outcome Prediction

The computing education research literature contains a wide variety of methods that can be used to identify students who are either at risk of failing... (more)

The Academic, Social, and Professional Integration Profiles of Information Technology Students

Low retention rates in higher education Information Technology (IT) studies have led to an unmet... (more)

Comparing Computing Professionals’ Perceptions of Importance of Skills and Knowledge on the Job and Coverage in Undergraduate Experiences

This article discusses the findings of a survey of nearly 300 computing professionals who are involved in the design and/or development of software across a variety of industries. We report on the... (more)

A Fringe Topic in a Fragile Network: How Digital Literacy and Computer Science Instruction Is Supported (or Not) by Teacher Ties

In this NSF CSforALL funded research study, the authors sought to understand the extent to which an urban district's teacher instructional support network enabled or constrained capacity to implement and diffuse Digital Literacy and Computer Science (DLCS) instructional practices throughout the K-12 curriculum. Social network analysis was... (more)



ACM Reappoints Editor-in-Chief

The ACM Publications Board has appointed Dr. Chris Hundhausen, current ACM TOCE editor-in-chief, to a second three-year term, which runs from August 31, 2018 through August 30, 2021. Chris is excited to continue his collaboration with the computing education community on the important work of maintaining, growing, and improving our premier archival publication for computing education research.

Submit Special Edition Proposals

ACM TOCE seeks proposals from researchers and educators interested in guest-editing special issues on topics of interest and relevance to the computing education research community. If you have an idea for a special issue, please email the editor-in-chief.

Special Issues on 'global software engineering' and 'capstones and projects' coming soon.

ACM TOCE will soon publish accepted manuscripts from our two special issues: "Global Software Engineering" and "Capstones and Projects".

Special Issue on machine learning education coming in early 2019

ACM TOCE will publish a special issue on machine learning education in early 2019. Authors interested in submitting articles to this special issue need to submit article abstracts by April 6, 2018, and full papers by June 15, 2018. For more information, please see the call for papers.

ACM TOCE review process is now double-blind 

Authors who submit papers to ACM TOCE should see the paragraphs on double-blind review in the newly-updated author guidelines for details on how to anonymize their papers for submission. Papers that are not properly anonymized will be returned to authors, thus delaying the review process. 

Authors: Submit by 8:00 am on first day of month for fastest review 

ACM TOCE's monthly peer review process begins on the first day of each month. On this day, we begin reviewing all papers received by our "soft" deadline:  8:00 a.m. U.S. Pacific Time on the first day of the month. If you want your paper to be reviewed as quickly as possible, submit it just before our monthly "soft" deadline.

About TOCE

The ACM Transactions on Computing Education (TOCE) publishes high quality, peer-reviewed, archival papers in computing education. Papers published in TOCE take a scholarly approach to teaching and learning, establish a clear connection to student learning, and appeal to a broad audience interested in computing education: instructors, researchers, curriculum designers, and administrators.  READ MORE

Forthcoming Articles

Equitable Learning Environments in K-12 Computing: Teachers' Views on Barriers to Diversity

The current efforts to expand computer science (CS) education in K-12 schools, such as the CS for All initiative, highlight the need for all students to get an opportunity to study computing. However, as recent research has shown, diversity in computing at the K-12 level remains problematic, and little has been done to look at how computer science learning environments can impact minority student interest and retention in CS. In this paper, we report results from an in-depth qualitative study of high school computer science teachers perspective on barriers to increasing diversity in their classes. Based on teachers experiences, we provide practical recommendations on how to encourage equitable learning environments in K-12 computer science courses.

Multiple Choice Questions in Programming Courses: Can We Use Them and Are Students Motivated By Them?

Low performance of engineering students in programming courses is a problematic that remains unsolved. Over the years, many authors have tried to identify the multiple causes for that failure, but there is unanimity on the fact that motivation is a key factor for the acquisition of knowledge by students. To better understand motivation, a new evaluation strategy has been adopted in a second programming course. Comprising 91 students, the goals of the study were to identify if those students felt more motivated to answer multiple-choice questions in comparison to development questions, and what type of question better allows for testing student knowledge acquisition. The results show that students felt more motivated to answer multiple-choice questions and overall obtained 8.5% better performance in those questions when compared to development questions. Additional analysis related to students performance was also performed, targeting gender and first enrollment as differentiating factors. Regarding to gender, the analysis did not show any kind of relation; however, the analysis revealed a correlation between first enrollment status and performance, and also between students who obtained success in the first programming course and performance. In conclusion, it is clear that student performance varies according to the type of questions. Our study points out that multiple choice questions can be seen as a motivational factor for engineering students and it also might be a good way to test acquired programming concepts. Because of that, this type of questions should be included in evaluation points.

A Robust Machine Learning Technique to Predict Low Performing Students

As enrollments and class sizes in postsecondary institutions have increased, instructors have sought automated and lightweight means to identify students who are at risk of failing a course. This identification must be performed early enough in the term to allow instructors to assist those students before they fall irreparably behind. This study describes a modeling methodology that predicts student final exam scores in the third week of the term by using the clicker data that is automatically collected for instructors when they employ the Peer Instruction pedagogy. The modeling technique uses a support vector machine binary classifier, trained on one term of a course, to predict outcomes in subsequent terms. We applied this modeling technique to five different courses across the computer science curriculum, taught by three different instructors at two different institutions. Our modeling approach includes a set of strengths not seen wholesale in prior work, while maintaining competitive levels of accuracy with that work. These strengths include being lightweight, affording early detection of struggling students, and predicting outcomes across terms in a natural setting (different final exams, minor changes to course content), across multiple courses in a curriculum, and across multiple institutions.

Searching for Global Employability: Can Students Capitalize on Enabling Learning Environments?

Literature on employability signifies "enabling" learning environments where students encounter ill-formed and open-ended problems and are required to adapt and be creative. Varying forms of "projects" have populated computing curricula for decades and are generally deemed an answer to this call. We performed a qualitative study to describe if and how project course students are able to capitalize on the promise of enabling learning environments. This critical perspective was motivated by the circumstance of the present-day education systems being heavily regulated for the precipitated production of human capital. The students involved in our study described education system-imposed and group-imposed narratives of lost opportunities, as well as many self-related challenges. On the other hand, students welcomed autonomy as an enjoyable condition and linked it with motivation. Self-related attitudinal attributes such as taking care of one's own learning and whole-group commitment appeared as important conditions. The results highlight targets for interventions that can counteract unfruitful behaviors and continue the march of projects as a means to foster professionalism in global software engineering.

A Framework for Teaching Security Design Analysis Using Case Studies and the Hybrid Flipped Classroom

With ever-greater reliance of the developed world on information and communication technologies, constructing secure software has become a top priority. In order to produce secure software, security activities need to be integrated throughout the software development lifecycle. One such activity is security design analysis (SDA), which identifies security requirements as early as the software design phase. While considered an important step in software development, the general opinion of information security subject matter experts and researchers is that SDA is challenging to learn and teach. Experimental evidence provided in literature confirms this claim. To help solve this, we have developed a framework for teaching SDA, by utilizing case study analysis and the hybrid flipped classroom approach. We evaluate our framework by performing a comparative analysis between a group of students which attended labs generated using our framework and a group that participated in traditional labs. Our results show that labs created using our framework achieve better learning outcomes for SDA, as opposed to the traditional labs. Secondary contributions of our paper include teaching materials, such as lab descriptions and a case study of a hospital information system to be used for SDA. We outline instructions for using our framework in different contexts, including university courses and corporate training programs. By using our proposed teaching framework, with our or any other case study, we believe that both students and employees can learn the craft of SDA more effectively.

Building LEGO Towers: An Exercise for Teaching the Challenges of Global Work

Global software engineering has changed the way software is developed today. To address the new challenges, many universities have launched specially tailored courses to train young professionals to work in globally distributed projects. However, a mere acknowledgment of the geographic, temporal, and cultural differences does not necessarily lead to a deep understanding of the underlying practical implications. Therefore, many universities developed alternative teaching and learning activities, such as multi-university collaborative projects and small-scale simulations or games. In this paper, we present a small-scale exercise that uses Lego blocks to teach skills necessary for global work. We describe the many different interventions that could be implemented in the execution of the exercise, and share the results from two runs of the exercise with a group of international students. Our results suggest that the exercise can be a valuable tool to help students dealing with troublesome knowledge associated with global software engineering and a useful complement to the courses dedicated to this subject.

How Else Should It Work?

In order to apply their computing knowledge to everyday situations and systems, students first need to be able to identify those situations in which computing even plays a role. This task becomes increasingly difficult, however, as computing systems become more and more ubiquitous and invisible. Based on the analysis of 28 semi-structured focus interviews, we present a grounded theory of students' conceptions and reasoning related to the identification of computing within technical devices. At its core is the finding that students seem to differentiate technical artifacts with respect to three conceived levels of capability. Many household appliances, for instance, may very well be seen as electronic and programmed, but still as too limited in their capability to warrant the presence of a real computer or to be related to informatics. Given the increasing versatility and power of modern embedded systems and the advent of the internet of things, this issue should clearly be addressed. Based on our grounded theory, we propose some first ideas for how this might be done.

Does Computer Game Design and Programming Benefit Children? A Meta-Synthesis of Research

It is widely believed that there are educational benefits to making computer games, but there is no systematic review of research on this topic. This paper describes a meta-synthesis of research on children designing and programming computer games to determine the extent to which there is evidence of benefits for learning and motivation. Over four hundred articles were identified, and eighty-four articles met the inclusion criteria. A systematic analysis and synthesis across studies showed some evidence that computer game design and programming can lead to changes in programming knowledge, problem solving, computer science attitudes and confidence, and other academic knowledge. However, most of the evidence described engagement in computing-related practices and did not measure learning. The findings were similar across different pedagogical approaches. In addition, the synthesis resulted in a list of design elements for studying computer game design and programming activities; these can be used to increase the availability of evidence about learning. The article concludes with the identification of gaps in the research and suggestions for additional research.

Digital and Physical Fabrication as Multimodal Learning: Understanding Youth Computational Thinking When Making Integrated Systems Through Bidirectionally Responsive Design

This paper proposes and explores the kinds of computational thinking, creative practices, design activities and equitable opportunities provided to diverse high school youth when designing simultaneously physically and digitally responsive wearable games. Previous work in this area coined the term bidirectionally responsive design, or the design of dual-feedback systems using multiple interfaces, which are both digital and physical, utilizing simplified digital and physical toolkits common in makerspaces. While some initial studies have explored the computational concepts, practices and perspectives involved in coding and circuitry, particularly with Scratch and e-textiles, this study extends this work by providing detailed analysis of one case study of a bidirectionally responsive project produced during one of the workshops. This study provides a framework to analyze computational concepts, practices and perspectives in the bidirectionally responsive project adding to, building upon and integrating previous analytical approaches to looking at Scratch coding, physical computing and e-textiles. Utilizing a detailed case study of one such youth team and project, we conduct a multimodal analysis of bidirectionally responsive making activities to discuss the ways they present novel understanding of integrating diverse interests, and fostering collaborative and distributed computational thinking.

Learning IS Child's Play: Game-Based Learning in Computer Science Education

Game-based learning has received significant attention in educational pedagogy as an effective way of increasing student motivation and engagement. The majority of the work in this area has been focused on digital games or games involving technology. We focus on the use of traditional game design in improving student engagement and perception of learning in teaching computer science concepts in higher education. In addition, as part of an interdisciplinary effort, we discuss the interplay between game-based learning in higher education and disciplinary cultures, addressing the lack of empirical evidence on the impact of game design on learning outcomes, engagement, and students' perception of learning.

The effects of adding non-compulsory exercises to an online learning tool on student performance and code copying

This study evaluates the impact of a new review-exercises module that has been incorporated into an online tool in a software engineering degree with the goal of avoiding plagiarism by enhancing study commitment. Two groups of students were quantitatively compared: the former used the tool exclusively during lab sessions, whereas the latter had the possibility of employing the tool for enhancing their study. The tool allows us to collect interesting data related to the focus of this study: supplementary work completed voluntarily by students and the percentage of students copying others work during lab sessions. In the obtained results, we have observed that the students in the enhanced study group plagiarized less and achieved better academic results. Moreover, in the enhanced study group the students that copied less and put in more effort, got better results; and, interestingly, the effort invested in completing review exercises did not seem to compensate for the learning effort avoided by copying others exercises during lab sessions. The results show the advantages of a tool used with a dual orientation: compulsory and voluntary. Mandatory usage in lab sessions establishes some milestones that, eventually, act as an incentive fostering learning, while voluntary use reinforces students perception of the tool usefulness to achieve learning.

Assessing Students' IT Professional Values in a Global Project Setting

This exploratory study examines the development of an instrument for assessing dispositional knowledge in the context of global software engineering. It focused on the development and use of a set of instruments for assessing affective domain student learning of global Information Technology (IT)/Software Engineering (SE) professional values. The effort needed to generate the questionnaires was relatively low, and were deployed to students and alumni from an open-ended global software engineering project course. The project included making explicit the IT professional values of interest among the participating faculty in the form of actionable value statements. It also included validation of these statements with an expert panel as question roots, and the use of these questions to investigate student and alumni receiving, responding and valuing of these professional values. Students responding reported significant agreement receiving these global values, but more mixed responses in responding and valuing them. The effort helped identify several actionable values worth reinforcing in future course offerings.

Transformative and Troublesome? Students' and Professional programmers' perspectives on difficult concepts in programming

Programming skills are an increasingly desirable asset for higher education students across disciplines; however, learning to program continues to be difficult to master for many students. The heterogeneity of student cohorts in most computer science courses makes it challenging to isolate the concepts within programming that are particularly hard for all students to understand (i.e., 'threshold concepts'). As a result, there is a lack of consensus over the threshold concepts that are relevant to those learning programming. This paper draws on a qualitative study using focus groups with undergraduates and professional software developers to identify potential threshold concepts in programming. Data from the focus groups were thematically coded and analyzed using a theoretical framework based on established criteria for threshold concepts. In particular, we focused on concepts that were both troublesome and transformative and included other characteristics such as participants' partiality of understanding. Six potential threshold concepts were identified in the majority of the focus groups, including abstract classes, data structures and designing objects. Further analysis of the data identified additional concepts that may hinder rather than help the learning of these threshold concepts, which we have called 'accidental complexities'.

Assessing the Impact of the Distributed Software Development Course on the Careers of Young Software Engineers

Various Software Engineering curricula in Higher Education started including courses on Global Software Engineering, carried out as international distributed project-based courses. These courses, known for their closeness to real world work experience, emphasize the importance of involving industry partners as customers and focus on soft skills important for employment, often neglected in engineering education. However, not many such courses are long-lived or of a consistent form throughout the years, making their impact and relevance hard to assess. The Distributed Software Development course (DSD), currently run among three universities from Croatia, Italy and Sweden, is now carried out for 14 years consecutively, providing a rich source of in-course and post-graduation data. In order to evaluate the students experiences of the course after they graduate and start working, a survey among former DSD students from University of Zagreb, Croatia has been carried out, to understand how useful this course was in students careers, both in first and current employment, depending on several factors during the course (such as project role), as well as factors at the workplace (such as the level of global teamwork). The survey results show the relevance of such distributed course experiences for future employment, as well as importance of building up on soft skills as part of the engineering curricula. Higher Education institutions are invited to consider including such courses in the Software Engineering curriculum, to the benefit of their students and, indirectly, their future employers.

Exploring and Expanding GSE Education with Open Source Software Development

Global software engineering (GSE) courses traditionally require cooperation between at least two universities so as to provide distributed development environment to the students. This study presents an alternative way of delivering such courses by incorporating open source software development (OSSD) in the absence of multiple universities alliance. The results show that the new setup provides challenges similar to multi-site software development as well as challenges associated with software development outsourcing. The present paper further compares a combined GSE and OSS course against a traditional GSE course in terms of students learning outcomes and course organization. The authors found that a combined GSE and OSS course provides similar learning opportunities to the students as a traditional GSE course, but with the added benefit of relative ease of organizing the course within a single university setting.

An empirical investigation on the benefits of gamification in programming courses

Context: Programming courses are mandatory in most Engineering degrees, and students performance in these courses is all too often not as good as expected. Programming is difficult for students to learn, given that it includes a lot of new, complex and abstract topics. All this leads us to acknowledge that fresh teaching techniques are needed if students are to be motivated and engaged in learning in programming courses. Gamification has come to be an effective technique in education in general, and it is especially useful in programming courses. This fact motivated us to develop a gamified platform, called UDPiler, for use in a programming course. Objective: The main goal of this paper is to obtain empirical evidence on the learning effectiveness of UDPiler in comparison to a non-gamified compiler. Method: A quasi-experiment was performed with two groups of first-year Engineering students at Diego Portales University in Chile, using a non-gamified compiler and a gamified platform respectively. Moreover, a post-experiment questionnaire was subsequently handed out, in order to obtain qualitative data about the gamified platform. Results: The results reveal that the students obtained better marks when the gamified platform for learning C programming was used. In addition, there is statistical significance in favor of a positive effect on the learning effectiveness of the students who used the gamified platform. Conclusions: The results allow us to conclude that gamification is an encouraging approach for teaching C programming. This finding is aligned with previous empirical studies about gamification, carried out in academic contexts.

RecurTutor: An Interactive Tutorial for Learning Recursion

Recursion is one of the most important and hardest topics in lower division computer science courses. As it is an advanced programming skill, the best way to learn it is through targeted practice exercises. But the best practice problems are hard to grade. As a consequence, students historically have completed only a small number of recursion programming exercises as part of their coursework. We present a new way for teaching such programming skills. Students view examples and visualizations, then practice a wide variety of automatically assessed, small-scale programming exercises that address the sub-skills required to learn recursion. The basic recursion tutorial (RecurTutor) teaches material typically encountered in CS2 courses. Experiments showed that RecurTutor supports recursion learning for CS2 level students. Students who used RecurTutor had significantly better grades on recursion exam questions than did students who used typical instruction. Students who experienced RecurTutor spent significantly more time on solving programming exercises than students who experienced typical instruction, and came out with a significantly higher confidence level.

Identifying Pathways to Computer Science: The Long-Term Impact of Short-Term Game Programming Outreach Interventions

Short-term outreach interventions are often conducted to raise young students awareness of the field of computer science (CS). Typically these interventions are targeted towards K-12 students, attempting to engage them in academic studies of CS later. The current study is based on a series of extra-curricular outreach events that introduced students to the discipline of computing, nurturing creative computational thinking through problem solving and game programming. To assess the long-term impact of this campaign, the participants were contacted and interviewed three or four years after they had attended an outreach event. We studied, in particular, how participating in the outreach program affected the students views of CS as a field, and more importantly, how it affected their educational choices. We found that in general the outreach program had an impact on the educational choices. The most prominent finding was that students who already possessed a maintained situational interest in CS found the event to strengthen their choices of further studies. On the other hand, there were many students whose plans were not affected by attending the program, but their image of the discipline of CS did change. Our results emphasize the need to provide continuing possibilities for interested students to experiment with computing-related activities, to maintain their emerging individual interests.

All ACM Journals | See Full Journal Index

Search TOCE
enter search term and/or author name