Research Supervision
Information and Tools
Print PDF

Institutional quality assurance

Overview and research

Quality Assurance mechanisms tend to evaluate whole of Faculty, or whole of institution, performance on standardised survey instruments. These tools can be useful at the institutional level in highlighting areas of excellence and areas requiring attention, but this information is generally not so useful at the level of the individual supervisor. Furthermore, results from quality assurance strategies are really only useful when acted upon by the institution and /or faculty and those actions shared with candidates and staff.

One example of a quality assurance tool is the Postgraduate Research Experience Questionnaire (PREQ). This instrument was developed by the Graduate Careers Council of Australia in the late 1990s amidst considerable controversy, particularly when the Federal Government suggested that it might use the PREQ results as a basis for funding universities.

The survey instrument, which is sent to all recent research graduates across Australia at the time of graduation, consists of 28 items clustered into six scales:

  • Supervision (Items #1, 7, 13, 21, 24)
  • Skill development (Items #6, 10, 14, 20, 26)
  • Intellectual climate (Items #5, 9, 16, 22, 23)
  • Infrastructure (Items #3, 8, 12, 18, 27)
  • Thesis Examination (Items #2, 15, 25)
  • Goals and Expectations (Items #4, 11, 19)
  • Overall Satisfaction (Item #28)

The results are analysed centrally and each university receives its own results compared with the national average across each scale.

Several universities e.g. the University of Sydney, the Australian National University, and Oxford University in the UK use the PREQ scales and items as the basis of their own, internal surveys.

Another example of quality assurance it to examine graduate destinations data also conducted by the Graduate Careers Council of Australia. At the same time graduates are surveyed with the PREQ, they are also asked to complete the Graduate Destinations survey. These data provide universities with a picture of where their graduates are employed, if in fact they are, and the main employers of doctoral graduates. For example, across the four main types of Australian universities, in 2004, the majority of graduates from each type were employed in some form of education. Furthermore, the majority of graduates from virtually every Field of Study, was employed in education.

The difficulty with data of this type is that they survey graduates soon after graduation when many of them are not in stable employment. Studies such as 'Ph.D.s--Ten Years Later' undertaken in the USA by Nerad and Cerny can provide enhanced understandings of doctoral graduates and their destinations after completion due to the extended period between doctoral completion and survey.

The above text was based on the following research:

Ginns, P. (2004, 7-9 July). Quality assurance and improvement of the Research Higher Degree Experience at the University of Sydney. Paper presented at the Australian Universities' Quality Forum, Adelaide.

Guthrie, B., & Trembath, R. (1998). Researching the researchers: The pilot Postgraduate Research Experience questionnaire. In M. Kiley & G. Mullins (Eds.), Quality in Postgraduate Research: Managing the new agenda (pp. 121-138). Adelaide: The University of Adelaide.

Marsh, H. (2000, June 28-July 4). When PREQ fails to measure up. Campus Review, p. 10.

Nerad, M. (2002, August-September). Postdocs: What we know and what we would like to know. Paper presented at the NSF/CPST/Professional Societies workshop, Washington, DC.

Nerad, M, & Cerny, J. (2002, August-September). Postdoctoral appointments and employment patterns of science and engineering doctoral recipients after Ph.D. completion: Selected results from the 'Ph.D.s--Ten Years Later' study. Council of Graduate Schools Communicator, pp. 1-11.

Neumann, R., Kiley, M. & Mullins, G. (2008). Australian Doctoral Graduates: Where are they going? In M. Kiley & G. Mullins (Eds) Quality in Postgraduate Research: Research education in the new global environment. CEDAM The ANU pp. 84-89

Trigwell, K., & Dunbar-Goddet, H. (2005). The research experience of postgraduate research students at the University of Oxford. Oxford: University of Oxford.

Ideas and tools

The Postgraduate Research Experience Questionnaire (PREQ) survey instrument can be downloaded.

ANU information

Since 2005 all PhD candidates at the ANU have been asked to complete the ANU Doctoral Experience Questionnaire (ANUDEQ). The request is made at about the same time as candidates are asked to submit their annual review and plan. A confidential report for each ANU College is forwarded to the College Dean and his/her nominee regarding HDR supervision matters. A collated, University-wide report is made available to the Pro Vice-Chancellor (Research)

For the five years prior to 2005 the ANU had only asked completing students to respond to the ANUDEQ as an exit survey, however, response rates were such that the results were not considered reliable and hence were not widely reported.

 

For College-specific information contact:

College of Engineering & Computer Science

College of Law

College of Arts and Social Sciences

College of Asia & the Pacific

College of Business & Economics

College of Medicine and Health Sciences

Alternatively, contact the College Associate Dean (HDR) for more general college information.