Quality Assurance in e-Learning – Tom Prebble

Issues:

1. QA in e-learning within NZ – the current situation
   
   There are two agencies with statutory responsibility for programme approval, institutional accreditation and ongoing programme validation within the tertiary sector in New Zealand. In the University sector, these responsibilities are carried by the New Zealand Vice-Chancellors’ Committee, and specifically by its sub-committee, the Committee on University Academic Programmes. The New Zealand Qualifications Authority discharges this responsibility on behalf of the remainder of the tertiary sector. In recent years, NZQA has delegated this responsibility to the NZ Polytechnics Programme Committee of the Association of Polytechnics of NZ for the polytechnics sector. In order to attract public funding, all tertiary qualifications must be accredited by the appropriate agency for that sector, and offer programmes that have also been approved by these agencies.
   
   Tertiary institutions are also subject to periodic review, by the Academic Audit Unit in the case of the University sector, and by the NZQA in the case of the remainder of the sector.
   
   Currently these quality assurance appraisals (institutional accreditation, programme approval, and ongoing review and audit) are performed against a single set of standards and expectations that apply regardless of delivery medium or mode. All quality agencies are aware of the particular quality challenges presented by virtual education and have moved to develop appropriate guidelines for considering such institutions and programmes.
   
   The New Zealand Universities Academic Audit Unit sought to support this endeavor by commissioning a publication on the subject in 1999: External Quality Assurance for the Virtual University. It is the expectation of the AAU that the guidelines indicated in this publication should inform any audit being carried out on virtual programmes delivered by NZ universities. This publication takes the view that the basic principles of QA should apply in all educational institutions regardless of delivery mode. For any course or programme, an institution must be able to demonstrate that:
   - Learning outcomes have been set at the appropriate level and clearly communicated to students.
   – Content and design of the curriculum and teaching methodologies are effective in enabling the student to achieve the outcomes in terms of both the acquisition of knowledge and the development of related practical skills and abilities.
   – Assessment is appropriately designed and rigorously administered to measure the achievement of the outcomes.
   All institutions are also expected to have quality assurance systems in place to deliver on these objectives and to be able to demonstrate the success of these systems.
   
   The Association of Polytechnics of New Zealand has recently carried out a similar exercise. In this case, APNZ was seeking to modify its own well-established 12 Quality Standards to take account of online and mixed mode delivery. As with the AAU initiative, the approved modifications to these standards statements simply direct the attention of quality auditors to particular aspects of online delivery, rather than setting a specific benchmark or target for these areas of activity.
   
2. Quality standards for e-learning
   
   There are arguments for and against introducing a set of quality standards specific to virtual education for New Zealand providers. In favour of the proposition, it could be argued that virtual education is such a new phenomenon that all stakeholders require greater assurance of its quality than might be delivered by existing medium-neutral standards and system. It could also be argued that, as virtual education has opened up a global educational market, international students will be looking to compare virtual providers with one another, rather than finding any assurance in a comparison of institutional providers within a particular nation or jurisdiction.
   
   Against the argument, it can be claimed that quality standards should related to core educational processes and learning outcomes rather than focus on the matter of delivery medium. An equally powerful argument is that ‘online learning’ is an ill-defined activity, and that a set of quality standards applying to any particular definition of this mode will not be applicable to variations of the mode. For example: a ‘standard’ relating to the presentation of study material online is unlikely to be applicable to a programme which restricts its online activity to interaction with students.
   
   On balance, the Group finds that the approach being taken by the various quality assurance agencies for the tertiary sector are appropriate for the purposes of institutional accreditation, programme approval, and ongoing audit of quality. This approach has been to expect an online programme to meet all the standards applying to more conventional modes of delivery, but to alert both providers and quality auditors to the particular challenges posed to quality by the online mode.
   
   Recommendation: that existing accreditation and quality assurance agencies be expected to apply their normal standards to programmes offered online, modifying these standards only where necessary to meet the particular characteristics of the online mode.
   
   
3. Assuring the quality of e-learning programmes within institutions
   
   It is currently a requirement for all public educational institutions to have quality assurance systems in place that are designed to meet the requirements of their particular business. Any provider engaged in online teaching needs to be aware of the special requirements of this mode in terms of quality assurance. Given the varied interpretation of virtual education, it may be preferable to continue to challenge each institution to arrive at its own quality standards. In doing so institutions need to be alerted to existing guidelines such as External Quality Assurance for the Virtual University, 1999 and Quality on the Line: Benchmarks for Success in Internet-based Distance Education, 2000. Each institution’s quality assurance plan, and its progress against that plan, is then the focus for regular audits by external quality agencies.
   
   There is scope for institutions to commit to a voluntary code of standards for virtual education that will encourage them to engage in constant improvement and allow them to publicly benchmark themselves in the market. Institutions may opt to work to international codes of practice in this regard, or to work collectively to arrive at a NZ code.
   
4. Verification of quality by external agencies
   
   The framework for external quality audit is well entrenched within the New Zealand education sector. As has been indicated already, each quality agency should provide guidance to providers of virtual education of the sorts of measures and standards that will be expected of them, but leave providers relatively free as to how they demonstrate compliance with these standards.
   
5. Institutional requirements for guidance/training/ in developing QA systems and delivering quality outcomes
   
   Most New Zealand tertiary institutions are committed to delivering at least some of their educational services online within the next very few years. For many conventional institutions, virtual delivery introduces quality challenges that they have never encountered before. These could relate, for example, to intellectual property and copyright; to access by students and teachers to hardware and networks; to the advice and guidance of far-flung students about course selection; to ‘version control’ of course material through the period of study; and to the security and privacy of communications systems, to name just a very small selection.
   
   Teachers, course development teams, and institutions throughout the country will need to identify and address these challenges. There is a real need to provide them with authoritative advice and guidance as they go about planning, developing delivering and assessing their virtual programmes. Most institutions are too small to provide comprehensive training for their staff, and may also find it difficult to develop adequate quality standards. This would be a case where a single agency could be given the mandate for coordinating the training and quality assurance efforts of the tertiary sector. The aim of such a unit should be to identify the dimensions of quality assurance in virtual education, to propose indicative standards and measures, and to showcase best practice from around the sector. The primary mode of communication and service delivery should be online. Such an agency might be funded through a combination of central funding and institutional purchase of services.
   
   Recommendation: that the Government commission an agency to provide advice and guidance to tertiary institutions on the development delivery and assessment of online programmes.
   
   
6. Protecting New Zealand students from poor quality offshore virtual programmes
   
   An unresolved policy issue is whether UTTA funding should be available for offshore educational providers delivering educational services to New Zealanders online. The Government has indicated support in principle for such a development but concerns over the quality of such provision (quite apart from the impact of such provision on the New Zealand education sector) have delayed its introduction. Other jurisdictions have moved to address the issue of the quality of offshore providers if not the question of a public subsidy. The Australian Government has moved recently to introduce the National Protocols for Higher Education Approval Processes (www.detya.gov.au/highered/mceetya_cop.htm). The protocols provide for virtual universities to be prosecuted in the jurisdiction in which their activities have an adverse effect as well as in their home countries.
   
   Recommendation: That the Government introduce legislation enabling prosecutions to be brought against offshore providers of virtual education in cases where students are adversely effected by the actions of those providers.
   
7. Information/guidance/protection required by New Zealand students in selecting services from virtual providers
   
   The ideal of virtual education is that students will be able to select virtual programmes from anywhere in the world, that they will receive a series of high quality learning experiences, and that they will be able to aggregate these completed study units into a recognized qualification. At present, a New Zealand student is confronted with a bewildering array of online options, and is frequently unable to determine their relative quality or the likelihood that study at any of them will earn them credit from another.
   
   The Canadian Government has funded the development of a consumers’ guide to distance and online learning. These guidelines identify the desirable features of high quality online courses and allow a prospective student to assess the quality of programmes before commitment to them. The draft guidelines at available at http://www.futured.com The mandate to publish and promote such a guide could be added to the service recommended under section 5 above.
   
   Recommendation: that the Government commission an agency to develop and publish a consumers’ guide to online learning opportunities.
   
   One of the key pieces of information that students need to know when selecting online programmes is the mix of media that will be used. This may be unproblematic in the case of programmes that are studied entirely on-line. More commonly, programmes are delivered by a mix of modes. In addition to their online study, students may be expected to attend block course sessions or final examinations at a particular time and place; there may be a requirement for scheduled, real-time, on-line tutorials; or in some cases, the use of online media may not even be a compulsory requirement for completing the course. There is a real need for New Zealand institutions to agree on a standard way of describing these features of course delivery and study to inform students unambiguously about the way the course will be delivered, and the requirements this places on students.
   
   Recommendation: that a priority for such an agency should be to arrive at a common set of descriptors to indicate the study requirements of each programme offering.
   
8. Dimensions of quality that are particularly sensitive in e-learning
   
   Institutions and quality agencies need to be aware of the pitfalls of online learning as well as the promises. In The Business of Borderless Education (Cunningham et al., 2000) a series of ‘hotspots’ to which quality agencies need to pay particular attention are identified. These include -
   - standards of online information and library resources;
   - verification of student identity;
   - the use of part-time contract staff as opposed to full-time tenured staff
   - subcontracting of administrative and ICT functions to separate commercial companies;
   - corporate management prevailing over academic governance;
   - no or little research being undertaken by teaching staff;
   - de-coupling of research and teaching/course development;
   - limited range of courses;
   - trans-border coverage;
   - discrepancies between measures of attendance and face-to-face modes.
   
   These are all areas to which quality assurance systems and quality auditors should pay special attention. It would be premature to suggest ‘standard’ solutions to these challenges.
   
9. Common barriers to quality in virtual education
   
   At a broader level, there are a number of common barriers that continue to affect the achievement of the objectives of online providers. Hope (2001, 136-7) lists these as-
   - inequality of access by students to the technology itself;
   - the temptation by funding agencies to underfund online learning with the result that providers are forced to withdraw the human intervention in their programmes;
   - the challenge confronting academic staff in our tertiary institutions to adopt new approaches to teaching with technologies they may be unfamiliar with;
   - the threat facing many academics that their formerly integrated role of course developer, teacher and assessor will be distributed across a team of specialists, and the resistance academic demonstrate to this threat.
   
   Many of these barriers will only be addressed by greater knowledge and understanding of online media and how they are best employed in the service of education. The need for this knowledge is not restricted to those who develop and deliver online courses. Those involved in institutional governance, in policy making both at a systems and institutional level, and in funding at a national and sector level, all need to be better informed about what is and is not possible through online learning. Recommendations elsewhere in this report about training and education requirements, about researching and promoting best practice , and about ensuring adequate national infrastructure begin to address this need.
   

References:

Advisory Committee for Online Learning, The e-Learning e-Volution in Colleges and Universities: A Pan-Canadian Challenge, Government of Canada, 2001 (http://www.schoolnet.ca/mlg/sites/acol-ccael/en/report.html)

Butterfield, S et al. External Quality Assurance for the Virtual Institution, New Zealand Universities Academic Audit Unit, July 1999.

Cunningham, S. et al. The Business of Borderless Education, Canberra: Department of Education, Training and Youth Affairs, 2000.

Hope, Andrea. "Quality Assurance", pp 125-140, The Development of Virtual Education: A Global Perspective (ed. Glen Farrell), Commonwealth of Learning, Vancouver, Canada, 2001.

Prebble, Tom K., "Quality Assurance in Dual Mode Institutions", in Achieving Quality: Examples of Good Practice in New Zealand Universities, AAU Series on Quality, Number 3, Academic Audit Unit, Wellington, 1999.

Quality on the Line: Benchmarks for Success in Internet-based Distance Education, Institute for Higher Education Policy, National Education Association, Washington, April, 2000 (http://www.ihep.com/quality.pdf)

Web-Based Education Commission to the President and Congress of the United States, The Power of the Internet for Learning: Moving from Promise to Practice, Washington, 2000 (http://interact.hpcnet.org/webcommission/index/htm)

Tom Prebble

August 2001