Evaluating graduate qualities
Evaluation is essentially concerned with:
exploring the relationship between the intentions of a program and the actual outcomes (both intended and unintended) of that program and
using this evaluative information to improve the program.
Because graduate qualities are such a central part of the curriculum, their systematic evaluation is an essential aspect of program evaluation and should occur within the wider evaluation activities as set out in the University policy A-35.A
To facilitate this evaluation, there are two main sources of evaluative information about your program—the national graduate Course Experience Questionnaire (CEQ) and the University of South Australia’s Course Evaluation Instrument (CEI).
The two instruments provide information about a program which varies in the scope, specificity and time lag.
However, the combination of data provides a rich source of information which can be used to plot developments over time and to address particular issues of quality improvement.
The Graduate Course Experience Questionnaire (GCEQ) is a nationally based questionnaire administered to all graduates of undergraduate programs in Australian universities by the Graduate Careers Council of Australia (GCCA). It is sent out in the April following a cohort’s final year and results are collated and sent to individual universities who make it available to staff. In addition, the results are published (among other things) in the Good Universities Guide and in the national press.
The CEQ has 25 questions that are aggregated into scales. One of these scales is called Graduate Skills and consists of the following 6 items from the questionnaire.
- The course developed my problem-solving skills.
- The course sharpened my analytical skills.
- The course helped me to develop my ability to work as a team member.
- As a result of my course I feel confident about tackling unfamiliar problems.
- The course improved my skills in written communication.
- My course helped me to develop the ability to plan my own work.
By the time the CEQ is available through the University it is about 1 year after students have left the University and by the time it is published in the Good Universities Guide it is about 18 months after they have left. Further, some response rates are extremely low. However, the value of this information is that it can be compared nationally across institutions and disciplines and is used to determine the level of funding the University recieves from the Federal Government.
The Course Evaluation Instrument (CEI) collects data at a course level, however the data from any courses can be aggregated so that comparative information can be obtained for a program, year level or strand.
The compulsory core of 10 questions has the following 2 Likert Scale response items:
- The course enabled me to develop and/or strengthen a number of the qualities of a University of South Australia graduate
- The assessment tasks were related to the qualities of a University of South Australia graduate
The CEI allows you to create up to 10 of your own Likert Scale response items. You could use this facility to gather data on any aspect of the graduate qualities in your course. Possibilities include:
The graduate qualities were clearly identified in the aims and objectives of this course.
- The graduate qualities were integrated into the teaching and learning of this course.
- This course made clear the relevance of the graduate qualities for my professional needs.
- The assessment items incorporated the graduate qualities.
- The marks and feedback given for the assessment reflected the emphasis placed on the graduate qualities.
- I found the teaching methods were effective in helping me develop graduate qualities.
- This course developed my ability to solve problems.
- This course developed my ability to think creatively.
- This course encouraged students to take responsibility for their own learning.
- In this course I was able to develop my ability to collaborate with others.
- This course helped me to develop a greater sense of social responsibility.
- This course helped me to develop my ability to write in ways appropriate to my discipline.
- This course provided me with opportunities to develop my abilities in oral communication.
- This course helped me to appreciate cultural diversity.
- This course enabled me to develop internationally relevant practices.
The CEI allows you to create up to 3 text response items. You could use this facility to gather qualitative data on any aspect of the graduate qualities in your course.