Academic staff and the process of subject review: some experiential qualitative accounts from 'backstage' and 'under the stage'
Margo Blythman
London College of Printing

This proposal is a report of work in progress. It follows from work by Newton and by Henkel looking at academic staff perceptions of the process of Quality Assurance Agency subject review in the United Kingdom. It reports on research on qualitative experiential accounts of subject review by academic staff. Three groups of staff have been interviewed, one group on the run-up to subject review, another group after one year and a third group after three years. In total 19 people have been interviewed and the interviews are being analysed using a modified form of grounded theory. The paper will outline the debate on subject review, the theoretical background to the research and some preliminary findings.

Impact of External Quality Assurance on Educational Standards
An Australian Case Study
Rob Carmichael
Swinburne University of Technology

Unlike institutions in the UK and NZ, Australian universities have had limited experience of external Quality Assurance. However, the Australian federal government has announced the establishment of a new 'Australian Universities Quality Agency'. The Agency will commence auditing universities in 2002.

In response to Lee Harvey's request for feedback from academics around the world about the 'quality control culture' and a purported 'decline' in higher education standards, a formal survey of Australian higher education staff was organised. The survey results will be presented and analysed at the session and their implications discussed in relation to the seminar's three main themes.

Teaching quality assessment scores: measuring quality or confirming hierarchy?
Dr Lynn Drennan and Professor Matthias Beck
Glasgow Caledonian University

Utilising data from the Times Higher Education Supplement league tables for the United Kingdom universities, covering the years 1993 to 2000, this paper examines the relationship between teaching quality assessment (TQA) scores, reputational factors and resourcing indicators. The analysis identifies a significant correlation between TQA scores, student entry standards and research assessment exercise (RAE) results. A principal component analysis of this data identifies a high multi-collinearity between these variables, such that the reputational factors (student entry standards and RAE results) exhibit the highest loadings. The conclusion is that, rather than representing an independent indicator of teaching quality, TQA scores largely serve to confirm existing hierarchies, which may be driven by factors

Lessons to be learned?: The Irish experience
Stuart Garvie
Dublin Institute of Technology

Higher Education in the Republic of Ireland has experienced unprecedented growth and change over the past thirty years. In Britain, a similar process was experienced soon after World War II and accelerated in the 1960s. The expansion and diversification in Ireland has been particularly complex in that, over the same period, there have been rapid and significant developments in technology and communications, as well as in Irish society itself. Within this context of expansion and change, issues of academic quality have come to the fore. In the event, many of the problems of comparability of standards and strategic planning in higher education have been revealed and are being investigated under the general banner of quality assurance and improvement. This paper seeks to describe the emergence of quality assurance in education in Ireland and considers the issues that have arisen in that process. In particular, and in the light of the Bologna Declaration, it questions whether experiences in other countries have influenced the policies and strategies adopted in Ireland.

A perspective on tensions between external quality assurance requirements and institutional quality assurance development: a case study
Ethney Genis
Technikon Pretoria

This paper reviews the development of quality assurance in technikons, and identifies some of the tensions between institutional quality assurance approaches and the approach of the external accreditation body. The account is given against a background of a higher education sector formerly characterised by a distinctive differentiation between technikons and universities, and which have followed significantly different quality assurance roads. The development of a unifying qualifications framework in South Africa has drawn all higher education institutions together in a single band, with one body responsible for quality assurance of all these institutions. The experience of technikons regarding quality assurance has lessons for the future.

Anything goes? The concept of quality revisited
Birgitta Giertz
Uppsala University

This paper argues that to be able to decide whether quality in higher education is coming to an end or not we must know what the important aspects of quality are and what they stand for. The concept of quality in higher education is still vague and confused. A model for analysing quality in higher education is proposed. Three perspectives (1) intrinsic quality, (2) extrinsic quality, and (3) politically correct quality are related to three groups of key stakeholders - the academic community, the market and the state. Each perspective has to be well defined and clearly described in order to reveal points of agreement and disagreement, thus providing a useful starting point for negotiations about a common platform for quality work. The state often uses quality to promote political and ideological purposes.

The roles of leadership and ownership in building an effective quality culture
George Gordon
Strathclyde University

The search for consensual, effective and meaningful agreements over the purposes, means and outcomes of quality assurance continues to tax the inventiveness (and at times, patience) of academics, administrators and other stakeholders higher education. Studies into several systems reveal that there are significant differences of opinion between key stakeholders, particularly about the effectiveness, appropriateness and insightfulness of operating schemes and new proposals. Yet internal and external monitors of quality assurance claim that progress is being made, quality assured, even enhanced, albeit whilst simultaneously identifying weaknesses and gaps which need to be addressed. Strategy and tactics, appear to be highly influential in the way departments and institutions respond to external agencies. Tactical responses may succeed in relation to the outcomes of external evaluations of quality assurance but are unlikely to build either an institutional or system-wide culture of quality assurance and continuous improvement. The evidence that has been gained over the past decade of quality assurance in higher education points to the centrality of strategy over tactics, and within the former, to the need to align leadership with ownership, and internal cultures with quality cultures.

Quality development: a new concept for higher education
David Gosling and Vaneeta D'Andrea
University of East London and City University

Despite the enormous growth in national quality assurance processes in the UK, serious doubts remain about their effectiveness in achieving lasting quality improvement. This paper suggests that the quality of students' experience of higher education can more effectively be improved by combining educational development with quality assurance to create a more holistic approach. The concept, which we call 'quality development', is explained and four examples of how this approach can work are described.

Running the maze: interpreting review recommendations
Margarita Jeliazkova
CHEPS, University of Twente

The paper presents the major findings and the theoretical insights gained by the latest study carried out by CHEPS on the impact of external review reports in Dutch higher education. A model of argumentation around review recommendations is presented and illustrated by cases from Dutch higher education institutions. Some suggestions are made about the value of the model as a predictive tool for success and failure and as a basis for the development of a practical instrument for optimisation of the effects of external quality reviews.

Disabled Students in Higher Education and Quality: Current public policy issues
Ozcan Konur
City University, London

Public policy initiatives in the United Kingdom have encouraged the widening participation in higher education. In line with these initiatives, disabled students have increasingly participated in higher education, currently comprising 4% of the total student population. In parallel with these developments there have been increasing concerns expressed about the academic standards and the related quality issues. This presentation aims to deal with the current policy and legal issues with regard to the quality issues, such as faculty development, academic achievement of disabled students, and the new 'duties not to discriminate against disabled students' and duties 'to make adjustments for disabled students' proposed by the Special Educational Needs and Disability Education Bill (2000).
This paper will be available to delegates although the author is no longer able to attend the seminar.

The Achilles' heel of quality: the assessment of student learning
Peter T. Knight,
Educational Research, Lancaster University

This paper explores the dependability of assessments of student achievement when used as performance indicators for internal and external quality monitoring (IQM and EQM). Problems are identified that jeopardise attempts to monitor, control and enhance quality in higher education. Responses are suggested with preference being given to a radical approach based on accepting that reliable national data about complex student achievements are not to be had. It is argued that this means that reliance on EQM is unwise and that more attention should be paid to internal quality enhancement.

Does the development of mass education necessarily mean the end of quality?
Dr Laurie Lomas,
Canterbury Christ Church University College

Four of Harvey and Green's definitions of quality are used as an analytical framework to examine whether the massification of higher education is bringing about the end of quality. Recent small-scale research with a sample of senior managers in higher education institutions revealed that fitness for purpose and transformation were the two most appropriate definitions of quality. Problems of measuring quality as transformation would suggest this interpretation of quality is at an end. However, the gauging of fitness for purpose through Quality Assurance Agency subject reviews indicates that the quality of mass higher education, in these terms, is not at an end.

Quality Assurance in Vietnamese Higher Education
Dr. Nguyen Phuong Nga and Dr. John J. McDonald

Although Vietnam has millennia of experience with higher education, its systems have been redesigned twice in the last 200 years. Now comes the third reorganisation, based in the on-going renovation ('doi moi') of its social organisation into a socialist market economy. In such conditions, with meagre resources but a high commitment to education, western 'quality' systems retain a freshness that might no longer be felt in more developed market economies. Therefore, Vietnam is now considering an opportunity to choose among alternatives, which have already been worked through and evaluated by developed countries, seeking to combine effective methods from different systems.

Quality: a picture tells a thousand words
Barbara O'Connor,
UNITEC Institute of Technology, Auckland, New Zealand

The non-university higher education sector in New Zealand has implemented a 'quality management systems' approach based on 'the wedge and the wheel'. All institutions have derived a quality management system that adheres to this approach.
There is a growing perception that the language associated with quality monitoring in higher education is becoming a major barrier to improvements in teaching and learning, and maybe even inhibiting the learning experience of the student. During 2000, the author used two key groups of people associated with quality in higher education - quality managers and programme leaders - to depict in pictorial form their perceptions of quality and to compare the pictorial perceptions with the 'wedge and wheel' model. This was an attempt to remove the barriers associated with the language, and to work at the conceptual level rather than the operational level. The various pictorial models are summarised and discussion of feasible alternative conceptual models for deriving effective quality assurance are invited.
This paper will be available to delegates although the author is no longer able to attend the seminar.

Statistical Analysis of Academic Programmes as an Indicator of Quality: A longitudinal study of MBA candidates
Godfrey Pell
Open University

The paper investigates programme quality issues using simple statistical techniques to analyse student grades. The data has been obtained from MBA examination boards, and includes marks for each module and for the dissertation. The findings indicate, inter alia, that candidates from the action learning development route significantly out perform the candidates from the DMS development route and women out perform men.

This paper demonstrates the inference that may be obtained from a basic analysis of student data, which ought to be recorded in any higher education institution's management information system. Secondly, the data represents the first quantitative comparative study of the performance of action learning candidates versus traditional candidates at MBA. Thirdly, the paper looks at the characteristics which make the action learning programme a high quality development route

Shorter outline

Quality: the end or just the beginning?
Mike Pupius
Sheffield Hallam University, UK

In higher education , 'quality' has been variously described as fitness for purpose, meeting customer expectations and needs or the achievement of the goals of an institution in an efficient and effective way. But does a score of 24 out of 24 for a subject review meet these requirements? If institutions seek to 'continuously improve' what lies beyond 24 points? Many organisations in the private and public sector have faced similar challenges and many now seek to measure themselves against the European Quality Award run by the European Foundation for Quality Management (EFQM). Organisations are using the EFQM excellence model to provide a health check, as a planning tool and as a strategic tool. This paper will report on the early findings from a study that is being funded by the Higher Education Funding Council for England to evaluate the benefits of applying the EFQM excellence model in an higher education context.

The quality issue: challenges and responses in Hungary
Christina Rozsnyai
Hungarian Accreditation Committee

Hungary opted to use accreditation as its tool for quality assurance in higher education. The author explores the implications of this approach and argues that, while almost all countries in Central and Eastern Europe conduct accreditation rather than quality evaluation alone, the regional and even country-by-country differences behind the term are considerable. After a brief history of, and foreseen future developments in, quality assurance in Hungary, the author concludes that quality, both as pertaining to that of higher education and the profession of quality assurance, will continue to exist, but is not a static concept.
This paper will be available to delegates although the author is no longer able to attend the seminar.

Quality assurance in South African higher education: a new beginning
Michael Smout and Sandra Stephenson,
Rhodes University, Grahamstown, South Africa

This paper aims to demonstrate that, far from being near the end, quality assurance in South African higher education is about to embark on a new beginning. A brief overview of approaches to quality assurance in South Africa up to 2000 is provided, with specific reference to the impact of the historical legacy of apartheid and the university/technikon divide. The new beginning relates to the establishment in 2001 of the country's first sector-wide external quality agency, the Higher Education Quality Committee. The challenges that face this statutory authority are critically examined.

Quality improvement alive and vital in United States accreditation
Dr. Stephen D. Spangehl
North Central Association of Colleges and Schools

The United States (US) prizes heterogeneous institutions responding creatively to the diverse needs of American society and economy. No assurance system that measures every institution against one quality standard will support this goal. In a higher education market where institutions struggle for advantage, the largest US accreditor has implemented an alternative accrediting process (AQIP) that helps an institution incorporate quality-improvement principles and strive toward self-chosen improvement targets to strengthen its competitive position. Fundamentally different from traditional quality assurance, which uses sporadic inspections to judge institutions against fixed standards, AQIP uses institutions' ongoing documentation of quality improvement to provide public quality assurance.

Quality and Mass Higher Education in Australia: A Matter of Purpose, Not Standards
Kim Watty
RMIT University, Australia

In Australia, the government has achieved its goal of mass education. The numbers prove it! Quality on the other hand is a more contentious concept and not so easily measured in quantitative terms. It seems, however, that this has not deterred those 'directing the traffic' in higher education from using numbers as a measure of quality. Whether the current quality agenda in higher education is viewed as a political project or legitimate goal will ultimately be determined by stakeholder views of the purpose of higher education.

Teaching skills, academic rewards and promotion
Staffan Wahlén
Swedish National Agency for Higher Education

The quality of a university depends on the quality of the conditions for learning, teaching and research which it offers and on the value it attaches to all these three aspects. Traditionally, universities have focused on research qualifications when promoting and rewarding their teachers and recruiting new teaching staff and paid lip service to teaching qualifications. One reason, it is maintained, is that there are reliable criteria for assessing research but not teaching. A Swedish commission of inquiry argues that there exist valid criteria but that they are not used sufficiently by either applicants or recruitment committees. Regulations to ensure that such criteria are developed and used are proposed.

Quality Assurance Policy in South Africa - the end of the beginning or the beginning of the end?
Denyse Webbstock

Quality assurance has been on the agenda in South African higher education for some years now. In introducing a quality assurance framework, South Africa found itself in the unique position of being able to learn from developments in other countries, as well as having the opportunity to introduce large-scale policy changes (in all areas, not just higher education) in the post-apartheid era. The time was ripe, therefore, for developing a system which embodied best practices from elsewhere, and which avoided some of the pitfalls, such as overregulation and bureaucratisation, that have bedevilled some other quality assurance regimes. However, in an analysis of policy developments undertaken in this paper, it will be argued that, despite early good intentions of introducing a development-orientated system, the push from major players in the system, such as the Department of Labour, has led to greater and greater emphasis on the accountability aspects of QA. In the latest National Plan on Higher Education, the tasks and functions which have been delegated to the new Higher Education Quality Committee indicate that the purpose and orientation of national QA in higher education has radically departed from the vision put forward in earlier policy documents. Furthermore, the recently announced new funding formula has the potential to engineer the system in such a way that quality issues will become secondary to the rush to increase student numbers in selected areas. It has become a moot question whether quality assurance will indeed lead to the improvements in higher education originally intended, or whether it will demand such a high level of compliance that institutions will either resist, or comply in minimal fashion, leaving fundamental quality levels unchanged.

From quality to standards in higher education: does the new UK 'standards infrastructure' offer a distinctively new - 'expositional' - approach to standards
Dr Peter W.G. Wright,
Quality Assurance Agency for Higher Education, UK

The new method of external quality assurance for higher education in the UK represents a fundamentally new strategic approach which, at first glance, is concealed by various superficial similarities with previous methods of review. Hitherto external quality assurance had limited itself to judging the quality of programmes of study against the ends and standards that an institution had set itself. The new method uses the five elements of a 'standards infrastructure' (subject benchmarking, the qualifications frameworks, programme specifications, the Code of practice and progress files) to make explicit the standards already implicit in good academic practice. They make it possible to begin to establish an open, public discourse about higher education standards in which all parties share common, overt reference points, and in which non-academics are no longer marginalised. Thus, the new quality assurance method, far from involving increased external prescription (as some have claimed), is an attempt to identify, promote and make visible the largely covert standards embodied in academic practice.

From quality control to quality assurance: how to change behaviour
Roos Zwetsloot and Veronica Bruijns
Hogeschool van Amsterdam, Amsterdam, The Netherlands

The Hogeschool van Amsterdam was nationally widely acknowledged for their system of quality control. However, this did not lead to high ranking in quality surveys. We concluded that we put to much emphasis on controlling quality instead of assuring quality. Assuring quality means that every employee is aware of the quality standards we want to achieve. Therefore, attention should be directed from instruments to behaviour. To help employees in thinking and acting according to quality standards, we developed a model with explicit criteria. This enables groups of employees to discuss about quality standards in concrete terms.