Working Group 4: A Multi-National Investigation of a Possible Hierarchy of Programming Skills - Proposal PDF
In New Zealand and Australia, the BRACElet project has been investigating the possibility of a hierarchy of programming skills. At the bottom of the hierarchy is knowledge of basic programming constructs (e.g. what an "if" statement does). At the top of the hierarchy is the ability to write non-trivial, correct code. The question studied in BRACElet is thus: What, if any, are the intermediate levels in that hierarchy?
This working group will bring the BRACElet project to ITiCSE, where it had its origins. The BRACElet project began as a direct consequence of the 2004 "Leeds" Working Group. Consequently, this 2009 working group will complete a five year circle that begins and ends at ITiCSE.
The members of this ITiCSE 2009 working group will collect data from their respective students. As ITiCSE attracts delegates from many countries, the data collected will include a wide spectrum of students, in their ability to program, their cultural backgrounds, and in the way they are taught to program. Thus this working group will pose a strong test to the generality of the claims made in the BRACElet project, which are based only on data collected from students at a small number of New Zealand and Australian universities. Our research goals for the working group are encapsulated in the following questions:
- Are there intermediate skills and knowledge in programming? If so, then...
- Is it possible to assess explicitly a student's grasp of these intermediate skills and knowledge?
- Is it possible to teach explicitly these intermediate skills and knowledge?
Working Group participants will collect data based on the BRACElet 2009.1 common core. The core consists of exam-type questions which fall into one of three categories, described below: (1) Basic Knowledge & Skills; (2) Reading / Understanding; and (3) Writing.
Working group participants need not use exactly the same questions, but their questions must fit the above three categories. Past questions have focused on procedural concepts, but working group members are free to use object-oriented questions (after having the questions reviewed by the working group leaders).
Most data will be collected from students, but the working group will also collect data on those same questions from some accomplished programmers (most likely academic colleagues), to provide a contrast for the data from the students.
Expected prior experience of members
To qualify for participation in the working group, data must be collected either through (1) an exam given to students, or (2) as a paper-based test given to student volunteers and/or accomplished programmers, or (3) through a "think out loud" protocol with students and/or accomplished programmers.
In cases where data is gathered as an exam given to students, or as a paper-based test given to student volunteers, data from at least 20 students is required, to enable statistical analysis. Where it is not possible to collect data from 20 students, working group membership can be obtained by conducting ’think out loud‘ sessions, with less than 20 student volunteers and/or colleagues.
Data may be collected from students at any stage of the undergraduate degree, but we anticipate that most questions used in the study will be aimed at students who are in the first two or three semesters of learning to program.
Working Group membership may also be granted, without data being collected, to anyone who can demonstrate a strong background in methods suitable for analyzing this type of data.
Please also see the information about Applying to be a Working Group Member on the Working Group page.
Working Group 4 leaders
- Tony Clear - Auckland University of Technology, New Zealand
- Raymond Lister - University of Technology, Sydney, Australia
