Chapter 16

The Quality Dilemma in Online Education

Nancy K. Parker
Athabasca University

Introduction

With the proliferation of online learning providers and the challenges presented by the distance education sector to state regulators and accrediting bodies, it is not surprising that “buyer beware” is the watchword for students, institutions, and public agencies alike. In the current environment, it is incumbent on organizations to demonstrate the quality of their services in ways that are intelligible to potential students and their employers, faculty and staff, regulators, and government agencies. The admirable attempts to define quality standards and best practices for online education have done little to assuage the skepticism of representatives in the academy, who are more accustomed to face-to-face delivery directed to bounded communities. Fully addressing the roots of such skepticism is beyond the scope of this paper; however, its presence informs much of the technical discussion around quality assurance frameworks in higher education in general, and in online delivery in particular.

Purveyors of online learning programs maybe inclined to attribute a lack of broad acceptance among their colleagues to the paradigm shift that higher education has been undergoing in the past 15 years. In many cases, however, it must be admitted that potential of electronic delivery modes has not been fully realized in the execution of online courses. Some have suggested that these shortcomings are the result of trying to replicate the classroom environment, instead of maximizing the new configurations of knowing and community formation possible in an interactive online environment (Schank, cited by Caudron, 2001). Moreover, finding appropriate comparators for the efficacy of any particular mode of delivery is difficult when the broader questions of quality assurance in higher education are far from settled. An examination of definitional issues points to a long-standing conflict in values between business modeling and public services. It is important to acknowledge these tensions fully before turning to the more technical, but admittedly value laden, exercise of reviewing the standards proposed by different quality assurance agencies.

After a discussion of the contexts of quality assurance activities in higher education in general, and of the competing paradigms highlighted by online learning, this chapter examines quality standards that have been proposed for the delivery of online instruction in four jurisdictions. The full range of state licensing, voluntary accreditation, and market driven seals of approval reveals tensions between externally driven compliance and internally driven improvements. Although the regulatory frameworks for quality assurance vary dramatically in Australia, the United Kingdom, Canada, and the United States, there is still enough common ground to establish some general characteristics for a scholarly approach to online teaching and learning. At a basic level, the characteristics of quality educational delivery demonstrated in these frameworks include 1) providing clear statements of educational goals; 2) sustaining the institutional commitment to support learners; and 3) engaging in a collaborative process of discovery, which contributes to 4) improving the teaching and learning environment.

Another area of commonality is the fact that, while self-review can be a key component for any of the frameworks, to a large degree they are being driven by external demands. One area of contention, the degree to which quality assurance activities can or should be targeted to outcomes as opposed to internal processes, is addressed in a separate section. On a wider level, each of the projects seeking to establish quality standards for online education appears to aim toward inculcating a set of values that prizes management by measurement. Recognizing that the terms “quality” and “online education” are so burdened with assumptions as to create their own problematic is a necessary prelude to what follows.

Definitional Issues

top

The greatest challenge for trying to define quality in any product or service is that quality remains a relative experience, realized in large part through an individual's level of expectation. Since quality necessarily rests in the eye of the beholder, at first glance, systems developed around the concept must necessarily be exercises in systematic subjectivity. In higher education, quality is a construct

relative to the unique perspectives and interpretations of different stakeholder groups (students, alumni, faculty, administrators, parents, oversight boards, employers, state legislatures, local governing bodies, accrediting associations, transfer institutions, and the general public). (Cleary, 2001, p. 20)

It follows, therefore, that the effectiveness of any quality improvement activities will be as much a function of the ability to foster agreement around common goals as of any substantive input or process adjustments attempted by an institution. Fostering agreement, however, is much more difficult when the term “quality” is burdened with the legacy of failed management fads.

In many circles, the term “quality” is understood as shorthand for Total Quality Management (TQM), or its close cousin, Continuous Quality Improvement (CQI). Some may believe that these fads peaked and retreated in the last century; however, recent modeling (Widrick, Mergen, & Grant, 2002), and examples of the pursuit, by individual institutions, of the Malcolm Baldridge Awards (Spahn, 2000) or ISO9000 recognition in distance education (Benjamin Franklin Institute, 2001) suggest that TQM still has a foothold in higher education, in spite of the problems posed by the fact that its language carries a corporate flavor (Banta & Associates, 2002). The International Standards Organization makes the central principle of the pursuit of quality clear—to establish processes that will maximize service to customers. The pressure to apply management techniques to higher education came from a perceived crisis in confidence with post-secondary systems, and from the growth of state-sponsored accountability systems.

For supporters it “has long been understood in organizations that when you want to improve something, you first must measure it” (Widrick et al., 2002, p. 130). But measurement systems are about much more than the technical specifications of various indicators—they are about control. The first iteration of TQM/CQI provoked a debate about its social as well as technical implications, and demonstrated the “disconnect between the philosophy of the management process and the purposes of the institution[s] for which it was being proposed” (Birnbaum, 2001, p.107). The engineering (or re-engineering) of systems designed to guarantee that manufacturing processes would meet technical specification might seem to imply a uniformity that may not be possible, or even desirable, in the dynamic and heterogeneous environment of higher education.

The annual race past the post in national press or government scorings, while clearly suspect, still draws a surprising amount of attention from audiences within the academy and government. This is a bitter pill for academics, who are accustomed to casting their institution's function as being nothing less than to maintain the foundations of the civil state. From that perspective, the vision of customer satisfaction is a pale substitute for contributions to knowledge and community formation. It also suggests an ideological congruence with the reduction of “citizens” to “taxpayers,” and as the focus moves to “value added” activities, the terrain of the debate is being narrowed to shorter and shorter transactional terms. To many within the academy, the “learner as consumer/information as commodity” world presupposed by the business model of higher education remains antithetical to independent scholarship in pursuit of the advancement of knowledge.

If these observations hold true, then it is fair to ask why one would engage in a discussion of quality in relation to any form of education. Why not just stand on the self-evident virtues of the “sacred spaces” of the classroom, library, and laboratory? The answer to that question resides in how contested meanings can be both signals of power relations and tools of resistance. Traditionally, universities achieved quality in intellectual endeavors through the professionalism of academics, the principles of scholarship, and the rigors of peer review; and they gained standing in society by communicating those standards to political and social elites. More recently, massification, diversity, and cuts to funding, along with a wider political movement to demonstrate efficiency and responsiveness, have spawned different conceptions of accountability (Brennan & Shah, 2000). The attempt to lift the meaning of quality education to something beyond short-term fiscal efficiencies and taxpayer benefits is a matter of trying to regain some of the ground lost in previous decades. It is also an encounter with what has been represented as a paradigm shift in higher education that has been highlighted by the advent of online education.

It must also be admitted at the outset that, with the shift to wireless technologies, “online” education may well appear to be an outmoded shorthand for computer or Web-enabled activities. The term has appeal, however, since it carries the sense of a linked community of learners. It still resonates of bounded communities with the possibilities of transformative experiences, rather than the sporadic or strictly utilitarian viewing of information on screens. This feature distinguishes online teaching and learning systems from Web-based publishing. It also presumes a qualitative difference from the convergence of distance and face-to-face delivery techniques being experienced on many campuses. With the proliferation of easily accessed Web tools, mounting a course's syllabus on its own home page, directing students to relevant electronic journals or Web-sites, and accepting assignment submissions by e-mail are no longer enough to allow one to lay claim to providing a distinct online teaching and learning environment. Textbook publishers are adapting, providing an increasing range of electronically delivered content systems and customized companion Web sites to supplant traditional student guides. When these resources are coupled with course management systems, the online environment begins to take shape. Until students and instructors engage, however, it is still just a shell.

It has been suggested that online learning is best conceptualized as “an environment that integrates collaboration, communication, and engaging content with specific group and independent learning activities and tasks” (Sims, Dobbs, & Hand, 2002, p. 138). More particularly, the ability of students to engage in “asynchronous interactive learning activities” has been described as the “signature characteristic of this technology” (Phipps & Merisotis, 2000, p. 6). The importance of the flexibility inherent in asynchronous activities challenges the assumption that emulating the classroom constitutes best practice in online teaching and learning environments. However, the degree to which technology has driven, or simply enabled, the paradigm shift in higher education, is debatable. Whether their adherents have overstated the changes that have taken place as a result of Web-enabled learning technologies is another question worthy of consideration.

Paradigm Shift

top

Although there had been many examples of applications of computer technology in classrooms for at least a decade before 1995, Michael Dolence and Donald Norris have been credited with issuing a “wake up call” for higher education administrators. In Transforming Higher Education, they purported to offer ways for colleges and universities to survive the transition from the Industrial Age to the Age of Information. Even though their vision for the future has not be realized on a wide scale, it is clear that many of the conceptual juxtapositions they offered have gained currency in higher education. These juxtapositions include a shift from episodic access, to clusters of instructional resources, to integrated perpetual learning, with a separation of teaching and certification of mastery, and a reconceptualized role for faculty—from deliverers of content to mentors and facilitators of learning. The most pervasive of these changes is the shift from a “provider focus” to a “learner focus,” with its attendent mass customization through individualized learning systems.

More recent elaborations on this theme have indicated that the capabilities of the Internet have overturned “the traditional roles of the college or university as the leading (1) research source and knowledge creator, (2) archivist and gateway to knowledge, (3) disseminator of advanced knowledge, and (4) referee and evaluator of truth” (Quinn, 2001, p. 32). If the production and dissemination of knowledge are no longer the restricted purview of higher education, the roles of postsecondary institutions in the worldwide network are increasingly vulnerable. The need to be more open and to promote capacities to analyze, interrelate and communicate about facts gleaned from network-based knowledge will become essential for students and faculty alike.

The traditional quality measures associated with accreditation or state administered quality assurance frameworks do not match this new climate of teaching and learning. One of the most common measures, and a unit of analysis for costing, is “seat-time,” which does not translate to an online environment. Yet Lawrence N. Gold, higher-education director for the American Federation of Teachers, claims that the amount of time a student spends in class is still an important measure of quality (cited in Carnevale, 2002). Even when adapted to an online environment, other common measures rely on inputs (averages of entering students, number of students, qualifications of instructors, systems development) or outputs (numbers completing courses; satisfaction ratings by students and alumni; revenue generated from tuition, intellectual property, or commercial partnerships), but lack in measures to address the fundamental integrity of the online learning environment.

A recent article by Wallace Pond summarized some of the old and new paradigms for accreditation and quality assurance as follows. According to Pond the old paradigm measures could be characterized by the following words and phrases: teacher-institution centred, centralized, hegemonistic, “one-size-fits-all,” closed, “us versus them,” quantitative, prescriptive, time as constant with learning as variable, teacher credentials, consolidated experience, regional/national, static, single delivery mode, process, infrastructure. In contrast, the new paradigm measures can be seen as learner centred, local, deferential, tailored, open, collaborative, qualitative, flexible, learning as constant/time as variable, teacher skills, aggregated experience, international/global, dynamic, distributed delivery model, outcomes, services (Pond, 2002, p. 4). The degree to which these measures might apply is discussed the next section, but they do not address some of the other questions generated by the entry into online course delivery.

The first questions must relate to the degree to which online learning environments have delivered or can deliver on their promises. The greater access afford through Web-based delivery systems has been one of the key advantages cited by observers of the technological transformation in higher education. Whether depicted as an advantage in developing greater economies of scale for delivery systems or in ameliorating social inequalities, broader access has been lauded as a key feature of the new paradigm. However, electronic learning systems are not always as billed. From the perspective of Thomas Booth, president of the Canadian Association of University Teachers (CAUT), the access argument carries very little weight, because people who have less education to begin with are less likely to have access to the tools and services they would need to study online. He also suggests that “Students traditionally excluded from post-secondary education are the most dependent on face-to-face interaction and least able to deal with the frustration and isolation of Web-based distance education” (Booth, cited in CAUT, 2001). Another caution rests in the comparative completion rates between online and classroom delivery. If intended economic and social transformations are to be realized, access must be examined at more than just the point of entry.

The promise that economies of scale will make education more affordable is perhaps even less persuasive to most academics. That “proprietary institutions are likely to enter the market by contracting with the best professors to provide video-based courses with exclusive rights to their distribution and use” was a vision of higher education in the 1990s (Hooker, 1997, p. 8). It is painfully obvious that the proponents of such models have missed the significance of interactive technologies. Providing more efficient delivery of “lectures by famous faculty” would recreate in cyberspace the “world of the passive listener and single speaker that has marked much of what passes for higher education” (Lairson, 1999, p. 188). Making the doubtful system of mass lectures more efficient does not appear to be much of an advancement over the correspondence school's traditional course-in-a-box. Another tension emanates from the fact that the bulk of what is delivered in the online environment consists of discrete training modules directed to particular job skills or competencies. While there seems to be slippage between what is being articulated in the realm of learning outcomes (the skills we expect graduates to demonstrate) and expectations around the values associated with the liberal arts, it is fair to say that higher education aims should be broader than the goals of the corporate training sector.

Critics such as David Noble have presented almost apocalyptic views on the incursion of educational technologies into the classroom (Noble, 2001). The Web's “dark side” has been depicted as the “rapidly growing trend of university corporatism” and the exploitation of knowledge workers (Kompf, 2001). Challenges from the for-profit sector, the influence of corporate training agendas, and “the ‘rush to serve’ different clienteles” have been described as jeopardizing the position of the post-secondary sector as the “source of objective analysis of the society in which it exists” (Crow, 2000, p. 2). Acting as the conscience of civil society speaks to a much broader purpose than meeting the immediate training needs of corporations. If this ideal is taken seriously, then one should expect that faculty would lead the debate from a perspective broader than their own protectionist instincts.

An alternative vision of democratic ideals in the digital age would have education enabling “people to learn about, with, and beyond technology” to open the “doors of economic, educational, and personal empowerment” (Milliron & Miles, 2000, p. 61). The reconceptualization of higher education should be done by (not to) the academy. Establishing the terms through which online education should be assessed should not be left to the marketplace or to self-perpetuating bureaucracies. Taking back some of the momentum will be a challenge, however, since attempts to establish standards and best practices are well under way.

Standards from Four Jurisdictions

top

The formulation of quality assurance systems for online education, while most frequently regulated at a regional or national level, has in recent years been driven by international developments. The global reach of the Internet and the lack of ways to regulate transnational commercial activities allow fraudulent operators to spring up. One response has been the promotion of consumer education by sites such as AboutEducation (http://www.about.com/education); another possibility is freelance course reviews from former students, similar to the book reviews found on the sites of online booksellers such as Amazon.com (Carnevale, 2000). Not surprisingly, the appetite for allowing the marketplace to determine the outcomes in a wide-open, for-profit model is not large. Simply stated, it does not seem either ethical or efficient to leave students to bear the full risks for the product testing of online education ventures. Alternative responses have involved trying to develop quality assurance and accreditation systems, or regionally harmonized systems, such as those proposed under the Bologna Declaration (see European Ministers of Education, 1999). Responses from national and local quality assurance interests have varied. Some of the differences rest in the degree to which state-sponsored quality auditing procedures have become entrenched in the past decade; others reflect the suspicions or traditions associated with distance education in general.

Australia has a national instrument, in the form of its Qualifications Framework, for protecting the quality of its educational and training programs. Even the use of the term “university” is restricted by State or Territorial legislation, and universities must demonstrate that they have appropriate quality assurance procedures in place. Within this framework, “universities are expected to engage in a pro-active, rigorous and ongoing process of planning and self-assessment which will enable them to ensure the quality outcomes expected by their students and the wider community” (Department of Education, Training and Youth Affairs, 2000). The Australian government policy framework has been presented as a marketing tool to address the advantages that global competitors enjoy by having “centralized, separate, and highly visible” bodies responsible for quality assurance (Vidovich, 2001). The rationale for the development of the national system was explicitly framed in terms of competitive challenges, domestic and international, and of policies that have encouraged the universities to “align themselves more closely with industry needs” (DETYA, 2000). Under the revised regime, creditable quality assurance systems providing evidence of the quality of service and skills of graduates were explicitly intended to make the universities more attractive to business investors. The systems include national qualification schemes that communicate expected standards for each level of post-secondary achievement.

Quality assurance for online offerings by Australian universities are within the self-accrediting models that include peer assessment processes. Registered training organizations (RTOs) have also been affected by quality audits, and draft guidelines have been developed for auditing online learning. The standards include statements on organizational commitment, learner support systems, learning designs, learning outcomes and assessments, and technology plans (Flexible Learning Advisory Group, 2002). Other features relate to managing risks by arranging for the stability of Web sites, and to cross-jurisdictional regulations. A key principle in this framework is that online delivery standards are incorporated within comprehensive criteria, and do not operate as stand-alone schemas. A general institutional standard can easily encompass online learning; for example, having written policies and procedures to ensure that educational offerings and assessment processes are consistent with the scope of the institution. The full ramifications of these elements for post-secondary institutions will depend on how the audit processes and guidelines are implemented in Australia over the next three years.

The examples of quality assurance frameworks from the United Kingdom are all centered around open and distance learning, with e-learning issues being acknowledged variables within a spectrum of delivery mechanisms. Three different external approaches to assessing the offerings by individual institutions include licensing procedures under the auspices of a government agency, a voluntary accreditation association, and a scheme for certification through quality marks. Again, much of the drive to enhance quality assurance schemes has been presented in the context of potential regional and global competition. Each of these examples also demonstrates ongoing tensions between external regulatory approaches and internal aspirations for improvement.

It has been suggested that the Quality Assurance Framework in the United Kingdom is not just comprehensive, it is “the most complex anywhere in the world” (Brown, 2000). The Quality Assurance Agency for Higher Education (QAA) was incorporated in 1997, with the aim of reducing some of the reporting burdens created by a combination of external assessments by funding agencies, and quality assurance processes driven by peer review. Its mission is to “promote public confidence that the quality of provision and standards of awards in higher education are being safeguarded and enhanced” (QAA, 2000). The QAA has developed codes of practice for ten areas: postgraduate research programs; collaborative provision; students with disabilities; external examining; assessment of students; program approval, monitoring and review; career education, information and guidance; placement learning; recruitment; and admissions (QAA, N.d.a). Further regulation has developed in the form of benchmark information for different subject areas, linked to the national frameworks for higher education qualifications. These are the explicit learning outcomes meant to communicate to the public and to potential employers the attainments to be expected from program graduates.

The main thrust of the QAA guidelines for distance learning is integration between distance delivery and the general quality standards for teaching and learning activities expressed in the other codes of practice. While not addressing online education directly, the six sections of the guidelines (system design, program design, program delivery, student development and support, student communication and representation, and student assessment), include examples of questions that address electronic delivery systems (QAA, N.d.b). The guidelines (paraphrased in Table 1 of Appendix 16A), are notable in that, whereas the codes of practice for higher education apply equally in distance education, the guidelines are intended to deal with aspects of quality assurance that are “likely to require attention in a particular way” when study is undertaken at a distance (QAA, N.d.b). The elaborate state licensing approach has been depicted as excessive and a sign of the erosion of the autonomy of higher education. To some, these measures have demonstrated the drive to “harness the universities to perceived economic priorities” (Greatrix, 2001). In that light, it is interesting that the criteria of the state licensing agency have largely subsumed standards that had been developed for a peer review model of accreditation.

The QAA distance learning guidelines reference the work of the voluntary association in the United Kingdom's distance education sector, citing the Open and Distance Learning Quality Council (ODL QC) standards. These accreditation standards, revised in 2000, are organized into the following discrete operational areas:

• course objectives and outcomes (providing clear statements of what will be achieved on successful completion; having objectives that are compatible with the method of delivery);

• course contents (providing sufficient content to enable the target group of learners to meet the course objectives);

• publicity and recruitment (providing accurate materials and direct communication to give potential learners the best basis for deciding to take the course);

• admission procedures (communicating the terms of the course, including its scope and the requirements needed to realize the intended outcomes, and providing applicants with enough information to assess their own needs and level of preparation);

• learning support (monitoring student progress, providing appropriate supports, including supplementary material, and facilitating peer group interactions);

• open learning centers (identifying groups of users and ensuring that sufficient resources are available to serve them);

• learner welfare (maintaining accurate records, providing appropriate guidance, demonstrating learner satisfaction and appropriate completion rates);

• providers (having appropriate plans and resources to meet their mission, adopting good business and employment practice, adhering to relevant legal requirements, ensuring that staff have appropriate qualifications, monitoring performance);

• joint provision (specifying respective rights and responsibilities, including procedures to meet standards); and

• accreditation (having procedures for application, limitation, and review).

The operational recommendations around learning support resonate with the models developed by the Open University, and technical supports appear almost incidental in the entire scheme. This model demonstrates an integrated understanding of the practice of distance learning, in contrast with the market model for applying quality marks, which draws on more generic business process analyses.

The British Association for Open Learning (BAOL) is an external marketing agency that applies self-assessment and external review processes. Of all the frameworks summarized here, it is the one most explicitly tied to TQM precepts. Using the Business Excellence Model developed by the European Foundation for Quality Management, BAOL has developed five “enabler” criteria that cover how an organization is managed: leadership, people, policy and strategy, partnerships and resources, and processes. BAOL had also developed four “results” criteria: people, customer, society, and key performance results. Together, these nine generic aspects of an organization “contribute to and enable” the achievement of quality (BAOL, 2002). BAOL has developed quality marks to cover materials development, advice and guidance, learner support, and learning centers. It is claimed that earning these quality marks provides a market advantage for e-learning providers, especially as they face the prospects of competing internationally. BAOL targets a broader range of education providers than does the university sector, and so its criteria have points in common with the suggestions for the registered training organizations in Australia. The existence of BAOL is an explicit acknowledgement that the market sector can be a powerful external influence on attempts to inculcate internal quality assurance values.

With such an array of quality assurance prospects, it is noteworthy that, in their study of “borderless education,” the higher education agencies in the United Kingdom have acknowledged that public accountability arrangements and elements of the credentialing or qualification schemes have been challenged by developments in for-profit, virtual, and corporate providers in the domestic and international higher education market. They have proposed that the quality frameworks addressing these developments would include

currency and security of qualifications; audit of the system for design and approval of curricula or appropriate learning contracts; an internationally recognized system of educational credit; licensing of staff; security of assessment; adequate and accurate public information about learning opportunities; approved guidance and complaints systems for learners; transparent quality management processes for each agent in the educational supply chain; access to learning resources assured by the provider; and publication of guidance relevant to different modes of provision. (Committee of Vice Chancellors et al., 2000, p. 30)

It has also been suggested that the thinking on quality assurance will have to shift dramatically, from external “compliance-based approaches” toward “comparative benchmarking” and mutual recognition arrangements for international quality standards. Attempts to integrate an array of international standards have been made in other jurisdictions.

In Canada, the responsibility for education rests at the provincial, not the national, level. Each province has its own quality assurance framework or approach to determining whether post-secondary programs are eligible for student funding or to receive public money. The degree to which a province might regulate or even provide subsidies to private or for-profit educational institutions varies widely. It is fitting, then, that the Canadian example of quality guidelines originates with a private corporation sponsored by community and government-funded agencies (Barker, 2002a).

The Canadian Recommended e-Learning Guidelines bill themselves as “consumer-oriented, consensus-based, comprehensive, futuristic, distinctively Canadian, adaptable, and flexible.” The last feature admits that “not all guidelines will apply to all circumstances” (p. 2). This qualification is only realistic, as the list is exhaustive. The 138 items are organized into three distinct sections: “Quality Outcomes from e-Learning Products and Services” includes 15 items related to how students acquire content skills, knowledge, and learning skills; “Quality Processes and Practices” includes 20 items on the management of students, and the delivery and management of learning, using appropriate technologies; and “Quality Inputs and Resources” includes the remaining 103 items, which range through intended learning outcomes, curriculum content, teaching and learning materials, product and service information, learning technologies, technical design, personnel, learning resources, comprehensive courses packages, routine evaluation, program plans and budgets, and advertising, recruitment, and admissions information. A more succinct adaptation issued under the same initiative was the Consumer's Guide to e-Learning, which structured 34 questions into basic, discerning, and detailed levels. These questions have been paraphrased in Table 2 of Appendix 16A, to allow for comparison with the other frameworks, but the instructions to consumers provided with the Consumer's Guide are more telling.

Before you sign up of an e-learning course or program ask yourself:

• What is my purpose for taking this course? Do I know what I want or need to learn?

• Do I need a credit or certificate when I finish . . . or do I just want to know more?

• How much can I afford to spend? How much time can I invest?

• What hardware and software do I have, and is it enough?

• Where will I access the Internet, what will it cost, and how convenient will it be?

• Are my computer and Internet skills good enough for the course I have in mind? Will I need technical help? (Barker, 2002b)

Institutions intending to adapt their offerings to the online teaching and learning environment would be well advised to rephrase these questions along the following lines:

• What is our purpose for offering this course?

• Do we know what we expect students to learn?

• Do we have the technological infrastructure to support our students? Is it up-to-date?

• How skilled are our course developers and instructors in the online environment?

• What technical assistance do we have available?

Such questions are at the heart of the two models proposed in the United States.

In an analysis of the impact of electronically delivered distance education, undertaken for the American Council of Education, Judith Eaton suggested that the emergence of electronically delivered degrees, programs, courses, and services, has the potential to undo the delicate balance between “accreditation to assure quality in higher education, the self-regulation of higher education institutions, and the availability of federal money to colleges and universities” (Eaton, 2002, p. 1). Although higher education institutions are subject to state funding and regulatory bodies, and although the systems of accountability may vary from state to state, the federal government relies on accredited status to signal that institutions and programs are of sufficient quality to allow the release of federal funds in the forms of student grants and loans, research grants, and other federal program funds. Under traditional approaches to accreditation, the focus was on the verification of site-based resources contributing to a learning environment (e.g., the number of volumes in the library). To address some of the concerns raised by electronic delivery, the eight regional accrediting commissions in the United States developed the Statement of Commitment for the Evaluation of Electronically Offered Degree and Certificate Programs, which declares the resolve of the commissions to sustain the following values:

• that education is best experienced within a community of learning where competent professionals are actively and cooperatively involved with creating, providing, and improving the instructional program;

• that learning is dynamic and interactive, regardless of the setting in which it occurs;

• that instructional programs leading to degrees having integrity are organized around substantive and coherent curricula which define expected learning outcomes;

• that institutions accept the obligation to address student needs related to, and to provide the resources necessary for, their academic success;

• that institutions are responsible for the education provided in their name;

• that institutions undertake the assessment and improvement of their quality, giving particular emphasis to student learning;

• that institutions voluntarily subject themselves to peer review. (reprinted in Eaton, 2002, p. 26)

The regional commissions also committed themselves to a common statement titled Best Practices for Electronically Offered Degree and Certificate Programs, developed by the Western Cooperative for Educational Telecommunications (WCET). The statement was organized into five discrete sections: 1) institutional context and commitment; 2) curriculum and instruction; 3) faculty support; 4) student support; 5) evaluation and assessment. Taken together, the Statement of Commitment and the Best Practices propose a consistent framework for developing quality standards. How those standards might translate into benchmarks was the subject of a study prepared by the Institute of Higher Education Policy (Phipps & Merisotis, 2000).

In Quality on the Line, Phipps and Merisotis surveyed the literature to compile a list of 45 possible benchmarks. They then determined whether those benchmarks were recognized at various institutions delivering online courses, and examined the importance of each benchmark to administrators, staff, faculty, and students at those institutions. The result was a list of 24 benchmarks that should be considered “essential to ensure the quality in Internet-based distance education” (p. 2). The elements (see Table 3 in Appendix 16A) include institutional support, course development, teaching and learning, course structure, student support, faculty support, and evaluation and assessment benchmarks. The similarities between these benchmarks and the proposals from the accrediting agencies clearly demonstrate a common conceptualization of distance education in the United States. Where they diverge is in the degree to which the actual curriculum elements are prescribed, and in the relative weights given to institutional structures. It is also apparent that both sets of standards are designed more for traditional face-to-face institutions introducing distance education programs than for distance education providers updating their mode of delivery. The provider focus remains a strong orientation under both U.S. schemes, and unlike the accreditation standard for open and distance learning in the United Kingdom, neither U.S. scheme speaks to the importance of encouraging learners to take responsibility for their own learning.

Process versus Outcomes

top

One of the first principles in all of the quality assurance schemes considered here is guaranteeing consistency in the product's results. In the view of TQM advocates “many quality management initiatives, especially in service industries, die because we fail in measurement of the outcomes” (Widrick et al., 2002, p. 130). The dangers of presenting higher education outcomes as strictly utilitarian competencies are familiar features in the debate about quality assurance activities. However, even if outcomes could be framed in wider terms, there is also a hazard of sliding into what has been aptly described as a variation on the “naming fallacy”; that is, assuming that “explicitness about standards” somehow provides assurance that the standards have been or can be achieved (Greatrix, 2001).

Major efforts have been directed to identifying “quality in undergraduate education,” but according to Ernest Pascarella, some of these efforts are “based on a naive understanding of just how difficult it is to accomplish in a valid manner” (Pascarella, 2001, p. 19). Most notably, he argues that institutional reputation and resources, and student or alumni outcomes are “potentially quite misleading,” and that results based on either of these common approaches are more likely to be driven by inputs than by effective educational practices (pp. 19-21). The solution to this problem should rest in careful measures that address the integrity of the teaching and learning processes within institutions. The seemingly insatiable appetite for comparable measures, regardless of their validity, is a dimension of the operating environments of most post-secondary institutions. While it is clear that the rhetoric of accountability and the bureaucratic systems it has spawned are not likely to disappear, it may be possible to present a framework for quality online teaching and learning that attends to more than short-term transactional or monetary values.

Reshaping the Debate

top

Whether or not the demands of stakeholder groups (however ill-defined), the threat of fraud, or the intensification of competition from local or international providers are behind the current impulses for elaborating quality assurance mechanisms, a dual challenge is being presented to the providers of online teaching and learning. The common thread across quality assurance schemes in the four jurisdictions is the need to address the concerns from both inside and outside the academy. Even if online and distance delivery institutions have been made the scapegoats for a wide range of changes, not the least of which being the erosion of the power of institutions of higher education to regulate themselves, there may still be an opportunity to address some of the concerns presented by colleagues in more traditional institutions. It follows that an overarching principle of any proposal to address quality assurance in online teaching and learning environments must be to recognize the integrity of higher education—no matter how it is delivered. The rhetoric of both the Australian and British qualification frameworks suggests just such an integrated approach, but the regulatory burdens they have spawned do little to reassure those who value the independence of higher education.

In the process of taking back some of the momentum in the debate, the academy must provide clear statements of educational goals. Such goals need not be restricted to technical mastery in specific subjects; the opportunity to pursue ideas beyond the needs of corporate sponsors should not be ignored. The measure of the effectiveness of the articulation of the educational goals should be the ways a course, program, and institution's goals align with one another. The demonstration of a consistency of purpose should be persuasive to internal and external stakeholders alike, but should not presuppose that students are responsible for seeking their own learning outcomes. This suggestion returns to the essential need for quality to be constructed through consensus building among a range of institutional stakeholders, who must, at the same time, not promise, or be promised, more than can be delivered.

A second theme running through all of the frameworks presented here is the need for sustained institutional commitment to support distance learners. The precise nature of that support would be determined by the nature of the programs and by what students need in order to have a reasonable chance of attaining their aspirations in a given program. All too often, online delivery of courses and programs has been presented in an experimental mode, without long-term, planned infrastructure development. Whether it involves investing in technical systems, or in training for support and instructional staff, the process of developing robust online teaching and learning environments should not be attempted as “one-offs.” Some observers have gone so far as to suggest that digital technology may hamper rather than promote educational change, because the focus of investment becomes the short life-cycle technologies, rather than the longer view needed for effective education (Ehrmann, N.d.). An institutional commitment to supporting learners will go a long way to satisfying other stakeholders, without displacing the fundamental project of scholarship. This can only be true, however, if students and educators are engaging in a collaborative process of discovery; that is, if academics are not simply dispensers or interpreters of content for passive students.

Learning technologies can promote powerful connections to content, context, and community. Unfortunately, they can also offer broad access to poorly designed and executed courseware. There are deliberate choices to be made in how to accommodate a generation of students who expect independent investigation, collaboration, and peer contacts to be facilitated in an online environment.

The threats to traditional delivery, and most especially the disaggregation of tasks associated with teaching in higher education, are providing new opportunities for exploring the constructions of community and knowledge, teaching and learning. In the generation of documentary evidence of interactions with content and with others, the structure of the online environment lends itself new kinds of exploration. Eventually, the goal of such inquiries should be to point to ways to improve the teaching and learning environment. Ultimately, the online programs should be able to mobilize recent theory and research into how people learn, and to enhance learning by “enabling the identified characteristics of effective learning environments and ensuring that they are present and accessible” (Herrington et al., 2001, p. 266) From that perspective, the pursuit of quality online teaching and learning environments may become as much an exercise in scholarship as it has been in market or state control.

References

top

About.com, Inc. (N.d.). AboutEducation. Retrieved May 5, 2004, from http://www.about.com/education

Banta, T., & Associates (2002). Building a scholarship of assessment. San Francisco: Jossey-Bass.

Barker, K. (2002a, January). Canadian recommended e-learning guidelines (CanRegs). Retrieved May 20, 2004, from the LICEF Research Center Web site: http://www.licef.teluq.uquebec.ca/fr/
pdf/CanREGs%20Eng.pdf

Barker, K. (2002b. January). Consumers guide to e-learning. Retrieved May 20, 2004, from the LICEF Research Center Web site: http://www
.licef.teluq.uquebec.ca/eng/pdf/ConGuide%20Eng%20CD.pdf

Benjamin Franklin Institute of Global Education. (2001). Distance education: Quality assurance initiative. Retrieved May 5, 2004, from http://www.academyWeb.com/DEQA

Brennan, J., & Shah, T. (2000). Managing quality in higher education: An international perspective on institutional assessment and change. Buckingham: Open University Press.

British Association for Open Learning (BAOL). (2002, April). The development, implementation and use of the BAOL Quality Mark: A report to the Department for Education and Skills. Retrieved May 5, 2004, from http://www.baol.co.uk/PDF/qmrepdfes.pdf

Brown, R. (2000, October). The new UK quality framework. Higher Education Quarterly, 54(4), 323-342.

Canadian Association of University Teachers. (2001, February 8). Online learning report earns “F.” CAUTNow!, 3(2). Retrieved May 5, 2004, from http://www.caut.ca/english/publications/now/20010208_online
.asp

Carnevale, D. (2000, February 18). Assessing the quality of online courses remains and challenge, educators agree. Chronicle of Higher Education, p. A59.

Carnevale, D. (2002, May 24). Accreditors offer views on distance programs. Chronicle of Higher Education, p. A36.

Caudron, S, (2001, February). Evaluating e-degrees. Workforce, 80, 44-48.

Cleary, T. S. (2001). Indicators of quality. Planning for Higher Education, 29(3), 19-28.

Committee of Vice Chancellors and Principals (CVCP) and Higher Education Funding Council for England (HEFCE). (2000, March). The Business of borderless education: UK perspectives, summary report. Retrieved May 5, 2004, from http://www.universitiesuk.ac.uk/book
shop/downloads/BorderlessSummary.pdf

Crow, S. (2000, January/February). Accreditation of online institutions. Technology Source. Retrieved May 5, 2004, from the Michigan Virtual University Web site: http://ts.mivu.org/default.asp?show=article&id=
663

Department of Education, Training and Youth Affairs, Higher Education Division (Australia). (2000). The higher education quality assurance framework. Retrieved May 5, 2004, from http://www.dest.gov.au/
highered/occpaper/00g/00g.pdf

Dolence, M. G., & Norris, D. M. (1995). Transforming higher education: A vision for learning in the 21st century. Ann Arbor, MI: Society for College and University Planning.

Eaton, J. S. (2002). Maintaining the delicate balance: Distance learning, higher education accreditation, and the politics of self-regulation. Retrieved May 5, 2004, from http://www.acenet.edu/bookstore/pdf/
distributed-learning/distributed-learning-02.pdf

Ehrmann, S. C. (N.d.). Technology changes quickly but education changes slowly: A counter-intuitive strategy for using IT to improve the outcomes of higher education. Retrieved May 5, 2004, from the TLT Group Web site: http://www.tltgroup.org/resources/Visions/Outcomes
.html

European Ministers of Education. (1999). The European Higher Education Area: Joint declaration of the European Ministers of Education, convened in Bologna on the 19th of June 1999. Retrieved May 5, 2004, from http://www.bologna-berlin2003.de/pdf/ bologna_declaration.pdf

Flexible Learning Advisory Group, Australian Flexible Learning Framework. (2002, October). Quality auditing of online learning: A guide for AQTF auditors. Retrieved May 5, 2004, from http://www.flexiblelearning.net
.au/qualityaudit/qa_guidelines_31october.doc

Greatrix, P. (2001, November). Quality assurance into the 21st century: Command and control or enlightened accountability? Perspectives, 5(1), 12-16.

Herrington, A., Herrington, J., Oliver, R., Stoney, S., & Willis, J. (2001). Quality guidelines for online courses: The development of an instrument to audit online units. In G. Kennedy, M. Keppell, C. McNaught, & T. Petrovic (Eds.), Meeting at the crossroads: Proceeding of ASCILITE 2001 (pp. 263-270). Melbourne: University of Melbourne. Retrieved May 5, 2004, from http://elrond.scam.ecu.edu
.au/oliver/2001/qowg.pdf

Hooker, M. (1997). The transformation of higher education. In Diana Oblinger & Sean C. Rush (Eds.), The learning revolution. Boston, MA: Anker Publishing Company, Inc. Retrieved May 5, 2004, from http://
horizon.unc.edu/projects/seminars/Hooker.asp

Kompf, M. (2001, September). ICT could be the death knell of professoriate as we know it. CAUT-ACPPU Bulletin Online. Retrieved May 5, 2004, from http://www.caut.ca/english/bulletin/2001_sep/
commentary.asp

Lairson, T. D. (1999). Rethinking the “course” in and online world. Campus-wide Information Systems, 16(5). Retrieved May 5, 2004, from http://fox.rollins.edu/~tlairson/online.html

Milliron, M. D., & Miles, C. L. (2000, November/December). Education in a digital democracy: Leading the charge for learning about, with and beyond technology. EDUCAUSE Review, pp. 50-62. Retrieved May 5, 2004, from http://www.educause.edu/pub/er/erm00/articles006/
erm0064.pdf

Noble, D. F. (2001). Digital diploma mills: The automation of higher education. New York: Monthly Review Press.

Pascarella, E. T. (2001, May-June). Identifying excellence in undergraduate education: Are we even close? Change, 19-23.

Phipps, R. A., & Merisotis, J. P. (2000). Quality on the line: Benchmarks for success in Internet-based education. Retrieved May 5, 2004, from the Institute for Higher Education Policy Web site: http://www.ihep.com/
Pubs/PDF/Quality.pdf

Pond, W. K. (2002, Summer). Distributed education in the 21st century: Implications for quality assurance. Online Journal of Distance Learning Administration, 5(2). Retrieved May 5, 2004, from http://www.westga
.edu/%7Edistance/ojdla/summer52/pond52.html

Quality Assurance Agency for Higher Education (UK). (2000). Handbook for academic review. Retrieved May 5, 2004, from http://qaa.ac.uk/public/
acrevhbook/acrevhbook.pdf

Quality Assurance Agency for Higher Education (UK). (N.d.a). Code of practice for the assurance of academic quality and standards in higher education. Retrieved May 5, 2004, from http://www.qaa.ac.uk/public/
COP/codesofpractice.htm

Quality Assurance Agency for Higher Education (UK). (N.d.b). Guidelines on the quality assurance of distance learning. Retrieved May 20, 2004, from http://www.qaa.ac.uk/public/dlg/contents.htm

Quinn, J. B. (2001, July-August) Services and technology: Revolutionizing higher education. EDUCAUSE, 28-37. Retrieved May 5, 2004, from http://www.educause.edu/ir/library/pdf/erm0141.pdf

Sims, R., Dobbs, G., & Hand, T. (2002, October). Enhancing quality in online learning: Scaffolding planning and design through proactive evaluation. Distance Education, 23(2), 135-148.

Spahn, K. (2000, May). Baldridge and institutional research: Quality measures for the next millennium. Paper presented at the 40th Annual Association of Institutional Research Forum, Cincinnati, OH.

Vidovich, L. (2001). That chameleon “quality”: The multiple and contradictory discourses of “quality” policy in Australian higher education. Discourse: Studies in the Cultural Politics of Education, 22(2), 249-261.

Widrick, S. M., Mergen, E., & Grant, D. (2002). Measuring the dimensions of quality in higher education. Total Quality Management, 13(1), 123-131.

Appendix 16A

top

The Quality Assurance Agency for Higher Education in the United Kingdom has distinct distance learning guidelines that “offer advice on assuring the quality and academic standards” of programs offered at a distance, including those making use of new technologies (QAA, N.d.b). The elements in these standard have been paraphrased in Table 16A-1.