This paper reports on a sequential mixed-methods research study, which compared different approaches on how to guide students through a semester long data science project. Four different methodologies, ranging from a traditional just assign some intermediate milestones to other more Agile methodologies, were first compared via a controlled experiment. The results of this initial experiment showed that the project methodology used made a significant difference in student outcomes. Surprisingly, the Agile Kanban approach was found to be much more effective than the Agile Scrum methodology. Based on these initial results, in the second semester, we focused on use of the Kanban methodology. The findings in the second, more qualitative phase, confirmed the methodologys usefulness and scalability. A key issue when using the scrum methodology was that the students had a very difficult time estimating what could be completed in each of their two-week sprints. The Kanban board, which visually shows and limits work-in-progress, was found to be an effective way for students to communicate with each other as well as with their instructor. In addition, Agile Kanban also streamlined the work required for instructors to efficiently provide guidance to student teams, and to understand each teams status. In summary, a scalable Kanban methodology, which can be applied to a wide variety of student computing projects, was found to an effective methodology to guide and manage student projects, improving student outcomes and minimizing instructor workload.
We performed a controlled experiment on comparing a C versus a Python Introductory Programming course. Three faculty at our university taught the same CS1 course for the same majors in two different semesters, one version in Python and one in C, with a total of 391 students involved in the experiment. We measured dropout rate, the failure rate, the grades on the two exams, the proportion of completed lab assignments, and the number of submissions per completed assignment. There was no difference in the dropout rate. The failure rate for Python was 16.9\% against 23.1\% for C. The effect size on the comparison of Python against C on the midterm exam was 0.27, and 0.38 for the final exam. The effect size for the proportion of completed assignments is 0.39 and the effect size for the number of submissions per assignment is -0.61 (Python has less submissions per completed assignments). All effect sizes are significantly different that 0. Thus for all measures, with the exception of dropout rate, the version of the course in Python yielded better student outcomes than the version in C.
Our cybersecurity workforce needs surpass our ability to meet them. These needs could be mitigated by developing relevant curricula that prioritize the Knowledge, Skills, and Abilities (KSAs) most important to cybersecurity jobs. To identify the KSAs needed for performing cybersecurity jobs, we administered survey-interviews to 44 cyber professionals at the premier hacker conferences Black Hat 2016 and DEF CON 24. Questions concerned 32 KSAs related to cyber-defense. Participants rated how important each KSA was to their job and indicated where they had learned that KSA. Fifteen of these KSAs were rated as being of higher-than-neutral importance. Participants also answered open-ended questions meant to uncover additional KSAs that are important to cyber-defense work. Overall, the data suggest that KSAs related to networks, vulnerabilities, programming, and interpersonal communication should be prioritized in cybersecurity curricula.
Massive Open Online Courses (MOOCs)-in part, because of their free, flexible, and relatively anonymous nature-may provide a means for helping overcome the large gender gap in Computer Science (CS). This study examines why women and men chose to enroll in a CS MOOC and how this is related to successful behavior in the course by (a) using k-means clustering to explore the reasons why women and men enrolled in this MOOC and then (b) analyzing if these reasons are related to forum participation and, ultimately, persistence in the course. Findings suggest that women and men have different reasons for taking this CS MOOC, and they persist at different rates, an outcome which is moderated by forum participation.
The introductory programming course is one of the very first courses that aspiring computer scientists take at a University. It is, unfortunately, also known for its low retention rates. In-time identification of those students who are likely to fail the course or to drop out would provide opportunities for early interventions. Given additional support and sufficient practice, students who might initially lean towards dropping out could be persuaded to continue study the subject. The computing education literature contains a wide variety of methods that could be used to identify students who are at risk of failing their studies, many of which are based on machine learning models that learn to make predictions based on previously observed data. However, in educational contexts, differences between courses set huge challenges for the generalizability of these methods. For example, traditional machine learning methods assume identical distribution in all data -- in our terms, that all teaching contexts are alike. In truth, data collected from different courses can be very different as there are many factors that may change, including grading, materials, teaching approach and -- for sure -- the students. Transfer learning methodologies have been created to address this challenge. They relax the strict assumption of identical distribution for training and test data, but naturally some similarity of the context is still needed for efficient learning. In this work, we review the concept of transfer learning methods especially for the purpose of the identification of students who are at risk of failing an introductory programming course, and contrast the results with those from traditional machine learning methods. The methods are evaluated using data collected in-situ from two separate introductory programming courses. The main focus is to see how these methods perform in the first weeks of the course, which are often the most critical for students. We demonstrate empirically that transfer learning methods are able to improve the predictions especially in cases with limited amount of training data, for example when making early predictions for a new context. Traditional machine learning models are, however, also sufficiently accurate assuming the contexts are closely related and the features describing the student activity are carefully chosen to be insensitive to the fine differences.
Problem: The involvement of external stakeholders in capstone projects and project courses is desirable due to its potential positive effects on the students. Capstone projects particularly profit from the inclusion of an industrial partner to make the project relevant and help students acquire professional skills. In addition, an increasing push towards education that is aligned with industry and incorporates industrial partners can be observed. However, the involvement of external stakeholders in teaching moments can create friction and could, in the worst case, lead to frustration of all involved parties. Contribution: We developed a model that allows analysing the involvement of external stakeholders in university courses both in a retrospective fashion to gain insights from past course instances and in a constructive fashion to plan the involvement of external stakeholders. Key Concepts: The conceptual model and the accompanying guideline guide the teachers in their analysis of stakeholder involvement. The model is comprised of several activities (define, execute, and evaluate the collaboration) and the guideline provides questions that the teachers should answer for each of these activities. In the constructive use, the model allows the teachers to define an action plan based on an analysis of potential stakeholders and the pedagogical objectives. In the retrospective use, the model allows the teachers to identify the issues that appeared during the project and which causes were underlying the issues. Drawing from ideas of the reflective practitioner, it contains an emphasis on reflection and interpretation of the observations made by the teacher and other groups involved in the courses. Key Lessons: Applying the model retrospectively to a total of eight courses reveals that it is possible to reveal so far implicit risks and assumptions and to gain a better insight into interaction between the external stakeholder and the students. The empirical data has revealed seven recurring risk themes that categorise the different risks that appeared in the analysed courses. These themes can also be used to categorise mitigation strategies to address these risks proactively. Additionally, aspects not related to external stakeholders, e.g., about the interaction of the project with other courses in the study program, have been revealed. The constructive use of the model for one course has proved helpful in identifying action alternatives and finally deciding to not include external stakeholders in the project due to the perceived cost-benefit-ratio. Implications to practice: Our evaluation shows that the model is a viable and useful tool that allows teachers to reason about and plan the involvement of external stakeholders in a variety of course settings, and in particular in capstone projects.
In computer science, students could benefit from exposure to critical programming concepts from multiple perspectives. Peer review is one method to allow students to experience authentic uses of the concepts in an activity that is not itself programming. In this work, we examine how to implement the peer review process in early, object-oriented, computer science courses as a way to increase the students knowledge of programming concepts, specifically Abstraction, Decomposition, and Encapsulation, and to develop their higher-level thinking skills. We are exploring the peer review process, the effects of the type of review on the reviewers, and the results this has on the students learning. To study these ideas, we used peer review activities in CS2 classes at two universities over the course of a semester. Using three groups (one reviewing their peers, one reviewing the instructor, and one completing small design or coding assignments), we measured the students conceptual understanding throughout the semester with concept maps and the reviews they completed. We found that reviewing helped students learn Decomposition, especially those reviewing the instructors programs, but we did not find that it improved the students level of thinking. Overall, reviews (peer or otherwise) are beneficial for teaching Decomposition to CS2 students and can be used as an alternative method for teaching other object-oriented programming concepts.
The introduction of Computing to the National Curriculum in England has led to a situation where in-service teachers need to develop subject knowledge and pedagogical expertise in computer science, which presents a significant challenge. Professional learning opportunities can support this; these may be most effective when situated in the teachers' own working practices. This paper describes a project to support Computing teachers in developing pedagogical skills by carrying out classroom-based research in their schools. A group of 22 primary (Grades K-5) and secondary (Grades 6-10) teachers from schools across England planned, designed and implemented research projects either individually or in small groups, supported by a team of university colleagues. Inter and intra group progress was shared online and face-to-face within a distributed community of inquiry. Data collection included surveys, video data, and the projects completed by the teachers. The findings from the project are analysed using Clarke and Hollingsworth's Interconnected Model of Teacher Professional Growth (IMTPG), which enables an identification and exploration of teacher change. Results of the analysis demonstrate that the approach can foster ``growth networks" - the construct used within IMTPG to indicate teacher change which is likely to be sustained and fundamental to teachers' understanding. The individual nature of this change indicates that the approach supports personal change related to each teacher's specific situation. Although there is a huge literature on action research as part of teacher professional learning, we believe this to be the first time this has been carried out in the context of computer science education. We conclude by critically reflecting on the lessons that we have learned in leading this project.
Modern capstone courses use agile methods to deliver and demonstrate software early in the project. However, a simple demonstration of functional and static aspects does not provide real-world software usage context, although this is integral to understand software requirements. Software engineering is experimental knowledge work: it involves creativity, imagination and interaction, that are typically not emphasized in its education. Therefore, a more engaging, dynamic way of presenting software prototypes is needed that creates a basis for communication about requirements and the context in which the software is used. We combine agile methods, scenario-based design and theatrical aspects into software theater, an approach to present visionary scenarios using techniques borrowed from theater and film, including props and humor. In this article, we describe our approach and provide examples and patterns to demonstrate its potential. We present two large case studies in which we teach students with varying levels of experience to apply software theater: a capstone course involving industrial customers with 100 students and an interactive lecture-based course with 400 students. We empirically evaluated the use of software theater in both courses using quantitative and qualitative methods. Our evaluations show that students understand and apply software theater and that this technique increases their motivation to prepare demonstrations already early in the project. Software theater demonstrations are more creative, memorable, dynamic and engaging than traditional techniques and bring fun into education.
Many computer science programs have capstone project courses which allow students to integrate knowledge from the full breadth of their major. Capstone projects may be student designed, instructor designed, designed in conjunction with outside companies, or integrated with ongoing free and open source (FOSS) projects. The FOSS approach has attracted a great deal of interest, in particular when integrated with projects that have humanitarian goals (HFOSS). There is literature that indicates that HFOSS projects increase student engagement and interest. However, instructors who wish to adopt an open source approach may worry that they cannot adapt this approach to the specific needs and goals of their institution and students. In this paper, we describe five unique models for incorporating significant FOSS or HFOSS (``H/FOSS'') projects into capstone project courses, from five distinct types of institution. All of the models presented are based on sustained H/FOSS engagement with projects that last at least one semester. The goal is to provide a guide for instructors wondering how to integrate significant H/FOSS project work into their curriculum. Each model is described in terms of its characteristics and how it fits the needs of the institution using the model. Assessment of each model is also presented. We then discuss the themes that are common across the models, such as project selection, team formation, mentoring, and student assessment. We examine the choices made by each model, as well as the challenges faced. We end with a discussion of initiatives that leverage institutional efforts by collaborating with outside organizations in order to address the challenges of these projects.
We present a methodological improvement to the ``Grade Point Penalty'' technique of detecting discrimination in large data sets by using a logistic model to account for heterogeneity in grading between courses. We then use this improved method to examine the grades of undergraduates for evidence of gender bias in Computer Science (CS) courses at a large university. We find that using a logistic model to simulate student performance across all courses removes course selection effects inherent in recorded Grade Point Averages (GPAs), and creates a much more accurate measure of student academic ability. This new measure greatly reduces the observed gender bias in academic achievement in CS and other STEM courses. We find that gender is not an important factor in academic performance in CS at this institution, and that the impact of gender on course performance for CS students is at most very small (less than 1/20th of a GPA point) and of uncertain sign.