November - 2004
Technical Evaluation Report
40. The International Learning Object Metadata Survey
Abstract
A wide range of projects and organizations is currently making digital learning resources (learning objects) available to instructors, students, and designers via systematic, standards-based infrastructures. One standard that is central to many of these efforts and infrastructures is known as Learning Object Metadata (IEEE 1484.12.1-2002, or LOM). This report builds on Report #11 in this series, and discusses the findings of the author's recent study of ways in which the LOM standard is being used internationally.
Characteristics of Learning Object Metadata Surveyed
“Metadata” refers to systematically created and formatted descriptions of resources, intended for learning, informational, or other purposes. The LOM standard has become the most widely used solution for classifying and describing digital resources intended specifically for learning and education. It is only one way of describing digital and online resources, however. Other metadata standards and methods have been developed for the same purpose, including Dublin Core and the Rich Site Summary (RSS): see Report #11 in this series. A common feature of these standards and methods is the fact that each defines the function and structure of a number of data elements. Examples of these include the title, author, and location of the resource. RSS, for example, focuses on three of these data elements - title, link, and description; while Dublin Core specifies only 16 metadata elements. The LOM standard, on the other hand, includes 76 data elements, covering wide-ranging characteristics attributable to LOs, including their size, level and type of interactivity, and the educational context to which they are best suited.
The LOM defines all of its data elements in interrelationships that are both hierarchical and iterative. At the top of the hierarchy of LOM elements are nine broad category elements: General, Lifecycle, Meta-metadata, Technical, Educational, Rights, Relation, Annotation and Classification. The category elements each contain sub-elements, which, in turn, often contain further sub-elements. Many of the category elements, sub-elements, and subordinate elements can be repeated. This results in complex hierarchical and iterative structures, allowing for a total of over 16,000 possible, concatenated element repetitions. Some of the sub-elements in the LOM (e.g., the title element) can be assigned an alphanumeric value. Other elements are associated with a limited set of pre-defined values (e.g., describing educational context such as school, higher education or training). In this last case, the set of values is often referred to as a “vocabulary” or “controlled vocabulary.” Still other elements in the LOM contain descriptions of persons (authors, editors, etc.) that are specially formulated and formatted using a specification known as vCard.
Given its relative size and complexity, as well as the fact that it is the first technical e-learning standard to be widely adopted, the implementation of the LOM presents an excellent opportunity for study and research. By looking at how it has been implemented in projects and in specific metadata records, it is possible to learn valuable lessons about e-learning standards implementation, and about how to develop and refine further standards to meet implementers' and educators' needs.
The current report presents the basic findings of an international survey of the implementation of the LOM standard. This survey was undertaken as a part of ongoing Canadian work in an international e-learning standardization forum: the ISO/IEC (International Standards Organization/International Electrotechnical Commission) subcommittee on Information Technology for Learning, Education and Training. The survey was conducted in two phases. The first involved the manual analysis of very small sets of randomly selected metadata records from a variety of collections and projects. The second stage involved the statistical, aggregate analysis of much larger sets of sample records, taken from five large collections from widely varying regions, including the European Union, Canada, and China. The findings of both stages of the survey were consistent and mutually reinforcing (see below). Only general findings and conclusions are reported in this paper. More detailed survey data and analysis are available in the original survey reports, submitted to the ISO/IEC committee (Friesen and Nirhamo, 2003; Friesen, 2004).
Survey Questions
The survey of LOM implementation was guided by three specific questions. Each question relates to the data elements of the LOM, and to the way in which each element is understood and used (or alternatively, not used). These questions, and their contextualizing explanations, are provided here:
- Which elements are being designated for use in LOM implementations?
As a first step in implementing the LOM, organizations, projects, consortia,
and national entities will frequently designate a particular set of LOM elements
for use in their respective d'mains. Such localized sets of elements are called
application profiles, and are often created in a process separate from technical
implementation, as a matter of policy. Elements are explicitly recommended,
required, or excluded from use. These policies are often applied in both e-learning
content development and the creation of infrastructures to support the exchange
of this content. Such an element set can include custom elements (element
extensions), adding new elements to the 76 already in the LOM. More often,
however, a subset of LOM elements is chosen, reducing the number of LOM elements,
often by as much as a half.
- Which elements are actually used in metadata records? Regardless
of the elements required, recommended or excluded in application profiles
and policy documents, the elements that are actually used provide additional
information about element utility and metadata requirement. Of those elements
actually populated, some may be utilized consistently, and repeated in order
to have a range of appropriate values assigned to them. Others may appear
only once, with an apparently arbitrary value assigned to them.
- What values are assigned to these elements? Finally, when elements are used, it is important to determine how they are actually applied to the needs of individual projects and resources. Quantifying the kinds of values assigned to elements can be difficult in some cases; but those elements with controlled vocabularies and value sets that are otherwise constrained (e.g., through the use of vCard) can be analyzed quite readily.
Findings
The findings of the current survey are presented as responses to each of the three questions raised above.
- Which elements are being designated for use in LOM implementations?
The survey has shown that, in many cases, the elements designated for use
in application profiles overlap with those already designated in the smaller,
simpler metadata element sets represented by Dublin Core and RSS. In addition,
educational elements in the LOM (those aspects of the data model that add
obvious special value for educational applications) are frequently not
designated for mandatory use in application profiles. Given some of the findings
discussed below, this raises the question of whether the challenges and costs
presented by LOM implementation are readily offset by its benefits –
especially in comparison with alternative metadata solutions such as Dublin
Core.
- Which elements are actually used in metadata records? The answer
is essentially the same as the first one (above), with some qualifying details
identified in the survey data. The elements actually populated in the metadata
records studied can be characterized as focusing on the intellectual
content of the resource. Many of these elements have rough or exact equivalents
in the Dublin Core Metadata element set. The same can be said for those elements
which describe the resource in terms of its characteristics as a media and
Internet file: they are well-utilized and also correspond to elements in the
Dublin Core element set. Those elements which attempt to describe the resource
as a software object, or to associate with it an educational context or level,
are much less frequently utilized. This is reinforced by vocabulary values
which are used to identify contributions to the creation of the resource -
i.e., the roles of author and publisher were well-utilized (together constituting
over 95 percent of the roles or values chosen), but roles associated with
software, instructional design, and media development (e.g., initiator, terminator,
graphical designer, instructional designer) were ignored.
- What values are assigned to these elements? Again, the finding is in keeping with the answers to the first two questions. In many cases, elements with controlled vocabularies were assigned values that reflected traditional, even print-oriented understandings of the resource as a 'ublished asset, rather than as a modular software object. These elements include not only the roles of contributors to the object (as above), but also the many values which can be assigned to indicate the resource's technical format (45 percent of which were indicated as “text/html”).
A number of other findings pointed to issues additional to those raised in the questions above.
- A surprising result was observed regarding the process whereby metadata
records are combined from a wide variety of collections into a single collection
for aggregate analysis. It was found that it is very difficult or, given limited
resources, actually impossible to import the various records into a single
database, and then to perform database queries to discover divergent and common
characteristics. This seems to have been the case in other, more limited survey
efforts (e.g., Najjar, Ternier, and Duval, 2003). LOM structures, with their
hierarchical and iterative interrelationships, make data portability difficult
to realize using conventional, low-cost technologies. Data portability and
reuse is presumably the raison d'tre of the LOM. The difficulties
the LOM presents to educational implementations in this regard are not positive
indicators of the prospect of increased sharing and reuse between implementations
and across jurisdictions.
- Very little of the complexity and detail that vCard information can
supply about contributors is actually exploited (almost 90 percent of the
vCard fields were unused in all instances studied). Any advantage that
the inclusion of vCard might present in LOM records, is far outweighed
by the difficulties of its implementation, and the under-utilization of vCard
fields.
- Only a small number of the potential element iterations and vocabulary values were used overall. This is unfortunate. Given the difficulties that these nested iterations and vocabulary choices can present to systems developers and record creators, the fact that few are used is cause for concern.
Conclusions
What do these findings mean for learning object implementation, and for the many projects and initiatives where learning object metadata are being used? On a positive note, the survey has revealed considerable convergence among implementations in element choice and utilization. Implementers have consistently opted to use roughly the same subset of elements, focusing on the description of the intellectual content of the resource. The fact that these same elements are also included in other, simpler metadata solutions, however, raises an important question: “What is the value added by the multiplicity and complexity of elements and element structures in the LOM?” The fact that a range of elements, and many of the possible element iterations in the LOM, remain unused means that their value is not being realized. At the same time, the price paid for this complexity and multiplicity, in terms of implementation work and data portability issues, is appreciable. These conclusions suggest that a very considerable return on learning object investment will be required for profit ultimately to accrue to learners and end-users.
———————————————————————
The next report in the series examines recent developments in the WebCT course management system.
N.B. Owing to the speed with which Web addresses are changed, the online references cited in this report may be outdated. They can be checked at the Athabasca University software evaluation website: http://cde.athabascau.ca/softeval/. Italicised product names in this report can be assumed to be registered trademarks.
JPB Series Editor, Technical Evaluation Reports
References
Friesen, N. (2004). Final Report on the International LOM Survey. Retrieved 20 October, 2004, from: http://dlist.sir.arizona.edu/archive/00000403/
Friesen, N., Nirhamo, L., and Knoppers, J. (2003). International Survey of Learning Object Metadata Implementation for ISO/IEC. Retrieved 20 October, 2004, from: http://mdlet.jtc1sc36.org/doc/SC36_WG4_N0029.pdf
Najjar, J., Ternier, S., and Duval, E. (2003). The Actual Use of Metadata in ARIADNE: an empirical analysis. Proceedings of the 3rd Annual Ariadne Conference. Retrieved 20 October, 2004, from: http://www.cs.kuleuven.ac.be/~najjar/papers/EmpiricalAnalysis_ARIADNE2003.pdf
PID: http://hdl.handle.net/10515/sy5pv6bm2