Computers in Biology and Medicine 63 (2015) 74–82

Contents lists available at ScienceDirect

Computers in Biology and Medicine

journal homepage: www.elsevier.com/locate/cbm

Practical use of medical terminology in curriculum mapping

Martin Komenda a, Daniel Schwarz a,n, Jan Švancara a, Christos Vaitsis b, Nabil Zary b,
Ladislav Dušek a
a
Institute of Biostatistics and Analyses, Faculty of Medicine, Masaryk University, Czech Republic
b
Department of Learning, Informatics Management and Ethics, Karolinska Institutet, Sweden

art ic l e i nf o a b s t r a c t

Article history: Background: Various information systems for medical curriculum mapping and harmonization have
Received 5 March 2015 been developed and successfully applied to date. However, the methods for exploiting the datasets
Accepted 7 May 2015 captured inside the systems are rather lacking.
Method: We reviewed the existing medical terminologies, nomenclatures, coding and classification
Keywords: systems in order to select the most suitable one and apply it in delivering visual analytic tools and
Medical informatics reports for the benefit of medical curriculum designers and innovators.
Medical coding Results: A formal description of a particular curriculum of general medicine is based on 1347 learning
MeSH units covering 7075 learning outcomes. Two data-analytical reports have been developed and discussed,
Medical education
showing how the curriculum is consistent with the MeSH thesaurus and how the MeSH thesaurus can be
Curriculum management
used to demonstrate interconnectivity of the curriculum through association analysis.
Visual analytics
Association analysis Conclusion: Although the MeSH thesaurus is designed mainly to index medical literature and support
searching through bibliographic databases, we have proved its use in medical curriculum mapping as
being beneficial for curriculum designers and innovators. The presented approach can be followed
wherever needed to identify all the mandatory components used for transparent and comprehensive
overview of medical curriculum data.
& 2015 Elsevier Ltd. All rights reserved.

1. Introduction informatics, the topic covering the broad use and global accep-
tance of medical coding standards still involves two main unre-
Medical coding and classification systems are used in a variety solved challenges: (i) the number of coding standards developed
of applications in medicine, public health, education and medical for medicine and health care is high, whereas their widespread
informatics, including their use in statistical analyses, data mining, adoption has been rather slow; (ii) the developed standards and
and expert and decision support systems or knowledge engineer- vocabularies vary in their coverage, often being contradictory and
ing. Creation and maintenance of the coding systems, nomencla- competing. The basic principles of medical informatics and dis-
tures and standardized vocabularies involves the processes of cussions on medical coding can be found in books and guides, such
transforming descriptions of diagnoses and procedures into a as [4] or [5]. One working group inside the European Federation
universal medical standardized scheme [1]. When talking about for Medical Informatics (EFMI) is focusing on codes, classifications,
standardization, it is necessary to mention also ontologies, which terminologies and nomenclatures in their overview article [6].
formalize the meaning of terms used in practice, expected to play a They provide definitions and history of the International Classifi-
major role in the automated integration of medical data with cation of Primary Care (ICPC), and of the Read code and the
relevant information to support basic discovery and clinical Systematized Nomenclature of Medicine Clinical Terms (SNOMED
research, drug formulation, and drug evaluation through clinical CT). The review [7] analyses the use of SNOMED CT over time from
trials [2]. If we want information systems in health care to process 1966 until June 2006. A more recent review [8] then reports on the
data automatically, to sort them and produce correct statistics and increasing use of SNOMED CT in literature, showing that the
if the principal data should be understood globally, it is necessary adoption of the nomenclature needs additional efforts toward
to use international coding systems and nomenclatures in the harmonization with other standardized terminologies.
process of data acquisition [3]. From the perspective of medical Medical education is an active area of education research, the
importance and value of which has been demonstrated by Harden
[9,10] and later by Wartman [11] already in the late eighties and the
n
Corresponding author. Tel.: þ 420 549492854. early nineties of the previous century. Medical curriculum develop-
E-mail address: [email protected] (D. Schwarz). ment, mapping and harmonization are the natural components of

http://dx.doi.org/10.1016/j.compbiomed.2015.05.006
0010-4825/& 2015 Elsevier Ltd. All rights reserved.

Descargado para Maria Fernanda Prada Reyes ([email protected]) en University Foundation Juan N Corpas de ClinicalKey.es por Elsevier en septiembre 27, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.
 M. Komenda et al. / Computers in Biology and Medicine 63 (2015) 74–82 75

medical education, regardless of whether the underlying model is web-based system for medical curriculum management; (iii) back-
systems-based [12], outcome-based [13] or different (competency- ground and statistical methodology used to develop analytical reports
based, socially accountable, distributed, etc.). Increasing variety in the over curricular datasets. In the following section, two data-analytical
form and resulting proficiency of different curriculum models has reports are shown, depicting the results obtained from frequency and
recently prompted the medical education community to introduce association analyses of the particular medical curriculum while employ-
curriculum mapping guides [14] and interoperability standards [15]. ing the MeSH thesaurus. The importance of these achievements
The MedBiquitous Curriculum Inventory Standard [16], approved by together with their current limitations and future prospects are
the American National Standards Institute (ANSI), provides the tech- discussed and concluded in the final sections.
nical syntax through which a wide range of different curricula can be
expressed and subsequently analysed.
While there are a plethora of references, surveys and books on 2. Materials and methods
medical coding and classification for healthcare informatics appli-
cations, much less can be found on standardized vocabularies The main contribution of this paper lies in the use of a
employed in medical education. The existing cataloguing schemas standardized medical thesaurus in the analyses of a dataset
and thesauri relating, at least partly, to medical education are extracted from a curriculum mapping system. We believe that
reviewed in [17] by the Medical Education Taxonomy Research our approach can be generalized, although we present results
Organization (METRO). Following the results of their research obtained from data stored in a particular curriculum mapping
among stakeholders, METRO uses two existing thesauri: (i) the system, OPTIMED, and a specific thesaurus, MeSH. The selection of
Medical Subject Headings (MeSH) thesaurus, being generic for this specific thesaurus is the result of our scoping review carried
medicine, and the British Education Thesaurus (BET), being generic out according to the principles laid out in the framework for
for education. The thesaurus of descriptors for topics in medical scoping studies by Arksey and O’Malley [31] and recently extended
education resulting from METRO's efforts is not available today, and clarified elsewhere [32,33].
but their set of descriptors for assessment in medical education
[18] using as much as possible from the MeSH and BET thesauri is 2.1. Overview of the existing medical terminologies
still a living project with results available on-line. The need for
identifying a suitable ontology or taxonomy for annotating the In this subsection, we refer to the summary overview relating
content of medical educational resources is emphasized in the to the up-to-date use of various specialized terminologies in
systematic review [19], and especially from the perspective of information sciences and education. Our close attention is primar-
sematic interoperability of information systems used in medical ily focused on medically oriented fields where these taxonomies
education and the potential of the semantic web (Web 3.0) to promote an efficient way of organizing and understanding data. It
support medical curriculum mapping. The review provides a is clear that specialized vocabularies matter, not only in the
tabular overview of all existing standardized vocabularies, includ- traditional library and information sciences world and but also
ing their features, such as language, scope and the underlying for many different digital information stakeholders [21]. For
structure, dividing them according to their context into biomedi- instance, the use of the vocabulary of a particular domain is an
cal, educational and administrative. Education resources shared important initial step in creating formalized knowledge represen-
through social media platforms such as YouTube have undergone tations as an essential part of the education process. These
exponential growth in recent years. Annotating education videos vocabularies follow the ratchet principle: it moves from basic
from YouTube and enriching them with other resources through understanding to thorough understanding, from simple to com-
Linked Data approach [20,21] is the main focus of research plex education [34]. When a virtual learning environment (VLE)
presented in [22]. SNOMED CT nomenclature is used to bridge turns to the task of consolidated education data collection (for
YouTube tagging data and the metadata of enriching education instance content management, curriculum mapping and planning,
resources. The resulting web-based architecture includes also the student engagement and administration, communication and
application interface (API) of BioPortal providing services from the collaboration domain), these vocabularies will present consider-
National Center for Biomedical Ontology [23]. able challenges to standardizing medical education. A prerequisite
In this research, we focus on exploiting the well-known to more comprehensive categorization of the education content is
polyhierarchical MeSH thesaurus while helping the medical curri- the implementation of standardized terminology directly in the
culum designers and innovators to navigate through multidimen- VLE systems. One of the primary aims is to overcome two
sional and extensive datasets embedded in the curriculum significant barriers to effective retrieval of machine-readable and
mapping systems. We are building on our previous experience processable information: (i) the variety of names used to express
with various nomenclatures and standards used in the develop- the same concept, (ii) the absence of a standard format for
ment of the education web portal platform [24] and its extensions distributing terminologies [35]. The purpose of medical vocabul-
[25] adopted by all medical faculties in the Czech Republic and aries is to embody what has been known in the past about every
Slovakia under the umbrella of the MEFANET network [26]. In phase of medicine [36]. These vocabularies continue to increase
terms of methods and tools for curriculum mapping and reengi- and grow, not only in its technological aspects, but also from the
neering, we are using our original web-based OPTIMED platform perspective of medical education quality, which is logically
[27] and the recently adopted visual analytics tools [28–30] with reflected at the global level of health care. Below, the most
the aim to develop new insights into curricular datasets and to widespread, suitable and commonly used standardized methodol-
demonstrate that the use of standardized thesaurus can facilitate ogies are introduced.
curriculum reengineering processes as well as inter-institutional
curriculum comparisons. In short, this paper shows an innovative 2.1.1. UMLS
way how to easily identify all essential components of a medical Unified Medical Language System (UMLS) brings together many
curriculum by adopting the MeSH biomedical nomenclature. health and biomedical vocabularies, ontologies and standards to
The remainder of this paper is as follows. In the methods enable interoperability between computer systems. It was devel-
section, detailed information is provided regarding: (i) existing and oped by the National Library of Medicine and covers the entire
widely-accepted medical terminologies; (ii) our implementation of a terminology domain by integrating more than 60 families of

Descargado para Maria Fernanda Prada Reyes ([email protected]) en University Foundation Juan N Corpas de ClinicalKey.es por Elsevier en septiembre 27, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.
76 M. Komenda et al. / Computers in Biology and Medicine 63 (2015) 74–82

use of SNOMED will demand new types of tools to search and

navigate intuitively in the term “collection”. It is probably the most
comprehensive clinical healthcare terminology in the world
because it systematically supports the development of compre-
hensive high-quality clinical content for health records and pro-
vides a standardized way of representing clinical phrases captured
by the clinician, enabling automatic interpretation. SNOMED CT is
a clinically validated, semantically rich, controlled vocabulary that
facilitates evolutionary growth in expressivity to meet the emer-
ging requirements [45,46].

2.1.5. Emtree
Emtree is a hierarchically structured and controlled terminol-
ogy for biomedicine and related life sciences, developed by Else-
vier as a biomedical and pharmacological online database. It
Fig. 1. The various subdomains integrated in UMLS [37]. includes a whole range of terms for drugs, diseases, medical
devices and essential life sciences concepts. Emtree thesaurus
biomedical vocabularies. The three major components of UMLS are contains about 48 thousand preferred terms and 200 thousand
the Metathesaurus (repository of inter-related biomedical con- synonyms in 15 main domains of drugs and diseases, organized in
cepts covering extensive list of terms and codes from various a multilevel hierarchy. All terms, strictly synonymous, are orga-
vocabularies), the Semantic network (high-level categorization of nized in a concept-based structure [47–49].
Metathesaurus concepts) and the Specialist Lexicon (generating
the lexical variants of biomedical terms) [37–39]. For illustration, 2.1.6. RadLex
Fig. 1 shows UMLS Metathesaurus integrated sub‐domains. RadLex [50] is a standardized vocabulary of radiological terms,
which includes highly detailed terms for anatomy, pathology, and
2.1.2. MeSH radiological observations. It already contains over 8000 anatomic
Although geared specifically for information retrieval, MeSH and pathology terms, many of which are not currently available in
can almost be seen as a general purpose vocabulary with concepts other controlled medical terminology systems. Generally, RadLex
from all segments of the biomedical domain. This classification is a was designed to fill the gaps in other medical terminology
rich and controlled vocabulary generated through an intense systems, thereby creating a single source for medical imaging
indexing process performed by examiners. Terms (namely descrip- terminology. Another key distinguishing feature of RadLex is that
tors) are assigned to documents to delineate their content at 16 it is designed to be continuously supplemented and updated with
different branches of specificity. The 2014 MeSH vocabulary is the incorporation of new concepts, including harmonization with
specifically composed by more than 27 thousand descriptors, other popular medical vocabularies and term sets. The goal of this
which are organized in a tree-like structure. Descriptors may be method is to establish a uniform, consistent terminology to
also complemented by one or more qualifiers. These terms further improve communication of results and to better integrate clinical
contextualize the meaning of the descriptors to which they are practice with education and scientific literature [51].
assigned in relation to the content of the considered document.
The MeSH vocabulary is often used to delineate samples of
documents in a number of medical areas and, as discussed, at 2.1.7. Summary
various levels of specificity [40–42]. The integration of various specialized nomenclatures, vocabularies,
and terminologies allows more precise analysis and improvement of
the education data content. Furthermore, automated systems can
2.1.3. ICD-10 apply the knowledge encoded with the use of mentioned taxonomies
International Classification of Diseases (ICD) is used to classify and human users can easily search and browse the available data in a
and code mortality information worldwide. The tenth revision simple and straightforward manner. It will have both academic and
(ICD-10) is more complex than the previous one (ICD-9). Specifi- clinical implications by enhancing the retrieval as well as the indexing
cally, ICD-10 contains more codes as well as re-structured chapters of information. Besides the introduced approaches, there exist many
and changes in rules for coding. ICD-10 as a whole is designed to other vocabularies that are specifically focused on a particular
be the core classification for a family of disease- and health-related biomedical discipline. Examples include Gene Ontology (GO), Gene
classifications. Some members of the family of classifications are expression data (eVOC), Online Mendelian Inheritance in Man
derived by using a fifth or even sixth character to specify more (OMIM), Merged Disease vocabulary (MEDIC), London Dysmorphology
detail. In others, the categories are condensed to render the broad Database (LDDB), Foundational Model of Anatomy (FMA), Logical
groups suitable for use, for instance, in primary health care or in Observation Identifier Names and Codes (LOINC) and the NCI the-
general medical practice [43,44]. saurus for cancer research. Contrary to these approaches, other
concepts are strongly related to administrative data, i.e. when was
2.1.4. SNOMED-CT what created by whom and who is the intended audience - examples:
SNOMED CT as a coding system for clinical problems was Dublin Core [52] as a pioneering standard for metadata about
created by the merger, expansion and restructuring of two large- electronic resources; Learning Object Meta-data (LOM) ontology
scale terminologies: SNOMED RT (Systematized Nomenclature of developed by IEEE association for describing the learning resources
Medicine Reference Terminology) and CTV3 (Clinical Terms Ver- [53]; Shareable Content Object Reference Model (SCORM) represent-
sion 3). Since it contains more than 380 thousand concepts, with a ing a widely adopted collection of specifications for web-based
total of about 800 thousand descriptions or terms, the practical e-learning systems [54]. On the grounds of our scoping review and

Descargado para Maria Fernanda Prada Reyes ([email protected]) en University Foundation Juan N Corpas de ClinicalKey.es por Elsevier en septiembre 27, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.
 M. Komenda et al. / Computers in Biology and Medicine 63 (2015) 74–82 77

composed requirements, we chose the most suitable thesaurus for the 3. Results
complex process of medical curriculum harmonization.
Since the coding systems, nomenclatures and standardized
vocabularies can be used for formal content description of any
2.2. OPTIMED: web-based curriculum management system
object, they may be also utilized together with traditional educa-
tion resources such as books, media, and programmes – to
For the purposes of enhancing the long-term process of
annotate courses, learning units and outcomes, exam questions,
medical curriculum mapping and harmonization, we decided to
or any other form of online resource. Thus, they allow searching
design, develop, implement and run our original web-based
for thematically related objects [19]. From the perspective of the
OPTIMED platform (http://opti.med.muni.cz/en). This system
education process, implementation of controlled vocabularies
describes and categorizes all learning activities (lectures, seminars,
significantly influences normalization of the achieved results,
clinical practices and self-study) in the theoretical and clinical
because students and teachers very often use different theoretical
sections of medical curriculum. We have aimed to capture the
and clinical terms that in fact mean the same. For example, the
systematic transmission of medical/clinical knowledge to students
terms heart attack, myocardial infarction, and MI abbreviation may
during their courses in General Medicine at the Faculty of
represent the same meaning to academics, but, to a computer,
Medicine of the Masaryk University. By development we refer to
they are different. The main purpose of the thesaurus concept
the Extreme programming methodology [55], which is a popular
integration is allowing users to search one term and to find and
agility-oriented framework. Based on the gathered requirements, a
retrieve learning activities that use synonymous terms. It also
modular structure of the OPTIMED system was proposed. It
enhances the indexing and aggregation process of data across
consists of three fundamental modules, which provide easy
various medical fields of study. In our particular case, we introduce
management as well as fast and transparent browsing through
the practical use of medical terminologies, namely the MeSH
the extensive domain of curriculum metadata: (i) Learning out-
thesaurus, in medical curriculum harmonization. We have devel-
come register, (ii) Learning unit register, (iii) Curriculum browser.
oped a web-based curriculum management system supporting
Overall, OPTIMED gives students, teachers, guarantors, curriculum
correctly compiled and balanced structure of obligatory and
designers and the faculty management a detailed look at where
optional courses across medical fields. We have also adopted the
specific topics and learning outcomes are addressed and how
MeSH nomenclature for standardizing metadata stored in the
education objectives are being met.
system database, which provides a set of descriptive attributes of
the whole curriculum with respect to international framework.
2.3. Statistical and computational methodologies
3.1. Case study: MeSH classification in medical curriculum mapping
We have processed in-depth analysis on medical curriculum
data supported by standardized vocabulary. We came up with a
The outputs of this case study represent systematic identifica-
systematic approach that allows formally describing and effec-
tion of content related items across curriculum boundaries and
tively optimizing the education structure through appropriate
bring an effective and proven approach enabling us to better
metadata attributes, and provides direct interconnection to essen-
understand the domain of continual improvement of medical
tial terms in the thesaurus. The mentioned analytical outputs can
education using modern information and communication tech-
significantly assist in detecting potentially problematic areas and
nologies. As stated above, the implementation of standardized
subsequently, compile comprehensive reports or instructions for
vocabulary can significantly improve the quality of further analy-
onwards in-depth global inspection, which is necessary for main-
tical processing in order to understand stored data. We adopted
taining continuity of the process of first-rate and guaranteed
the MeSH biomedical dictionary, where the main objective is to
studies at higher education institutions. The derived aggregated
classify learning activities appearing in the curriculum. In the past,
and comprehensible information, such as clear tables, transparent
keywords were defined and structured in many forms and there
graphs and other interactive visualizations, serve as supporting
was a growing need for their unification with respect to interna-
material for the long-term evaluation process under the super-
tional framework. The main requirement for standardized dic-
vision of senior guarantors and the faculty management. More-
tionary integration was regular updates of the Czech version,
over, the integration of proper vocabulary also provides a unified
which MeSH fulfils as the only available solution. No other
terminology for instituting curricular change and innovation.
language mutations are foreseen at the moment, but a possible
We based all reports on a set of proven analytical and statistical
change should not bring too many complications in terms of the
techniques encompassing robust computational and visualization
proposed structure. The National Library of Medicine produces
software. The occurrence of MeSH terms and linked categories in
annual editions, including the development and implementation
the description of learning units were described by absolute and
of a concept-centred vocabulary maintenance system for MeSH.
relative frequencies (percentage representation). Associations
This system has been extended to create a multilingual database of
among all MeSH keywords were computed using the Jaccard
translations and allows continual updating of the translations, as
similarity coefficient, which represents a statistic method for
well as facilitating tracking of the changes within MeSH from one
comparing the similarity and diversity of data sets. It provides
year to another. In this way, the vocabulary is useful for the non-
robust measurement of the association widely used in ecology
English user and enables potential connection between various
(measuring biodiversity) or market research (market basket ana-
languages (e.g. Czech and English).
lysis). This coefficient is based on positive similarity (subject to co-
occurrence) and not negative similarity (based on double absence)
and we used it in order to avoid the double-zero problem. All data 3.1.1. The MeSH structure
merging, manipulation, transformation and computations were MeSH is a controlled vocabulary thesaurus comprising over
conducted by IBM SPSS Statistics version 22.0.0. For interactive 27,000 primary terms called descriptors, which are arranged in
visualization of association maps, we used the yEd Graph Editor hierarchical structure and permit searching at various levels of
version 3.10.2, a freely available desktop application developed by specificity. More specific headings are found at narrow levels of
yWorks GmbH typically used for diagramming large adjacency the twelve-level hierarchy. Currently, the MeSH hierarchy is
matrixes with automated layout. divided into 16 categories (A for anatomic terms, B for organisms,

Descargado para Maria Fernanda Prada Reyes ([email protected]) en University Foundation Juan N Corpas de ClinicalKey.es por Elsevier en septiembre 27, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.
78 M. Komenda et al. / Computers in Biology and Medicine 63 (2015) 74–82

Fig. 2. Complete data model of MeSH entities in the OPTIMED database.

Table 1 3.1.2. Advanced data-analytical reports

Attributes describing the medical curriculum used for case study analysis. Implementing MeSH vocabulary into the OPTIMED system
(Source: the OPTIMED PostgreSQL database). requires specific validation tasks based on standardized terms
that we tried to resolve by curriculum data analyses. There are
Name of attribute PostgeSQL data type Sample value
several interesting domains that the selected statistic and analy-
unitName character varying(255) Skin anatomy tical methods may answer: coverage and relations between
unitSection character varying(255) Internal medicine learning activities and the MeSH classification, objective evalua-
unitClass character varying(100) Dermatovenerology tion of the MeSH keywords occurrence, model term integration
meshName character varying(255) Epidermis
into education, etc. Two particular reports based on MeSH classi-
meshPreferred Integer Yes
meshSemanticType character varying(50) Tissue fication, introduced below, assist in finding potentially proble-
meshTreeId character varying(100) A10.272.497 matic areas in the curriculum and provide a comprehensive
meshTreeLevel Integer 3 overview for subsequent in-depth global medical curriculum
meshCategory character varying(50) Anatomy
inspection. The extracted information from the OPTIMED curricu-
meshSubcategory character varying(50) Tissues
lum management system serves as supporting material for the
evaluation process under the supervision of the expert committee
and the faculty management.
C for diseases, D for drugs and chemicals, etc.). Each category is
further divided into subcategories (117 in total). For instance,
category B consists of the following subcategories: eukaryota, 3.1.3. Medical curriculum data set
archaea, bacteria, viruses, organism, and forms. These hierarchies We have already designed and implemented formal specifica-
are extremely valuable as they allow associated data to be viewed tion of the database scheme as a basic building element of the
at various levels of granularity, with data annotated to children of OPTIMED platform, which covers all elements pertaining to global
a branch to be aggregated at each higher level of the hierarchy curriculum harmonization including detailed metadata arrange-
[42]. Generally, the entire MeSH vocabulary is available for down- ment down to the level of learning units, interconnections to the
load in XML, ASCII or MARC formats. In order to better understand learning outcomes in accordance with Bloom's taxonomy and
the thesaurus construction, we decided to model a complicated direct interconnection to specialized MeSH nomenclature. The
structure of the descriptors, concepts and terms. Using the Bloom's taxonomy is often used when designing educational and
diagramming framework Visual paradigm, we carefully identified learning processes, where educational objectives are divided into
all important entities and attributes of spreading the MeSH three domains: cognitive (mental skills known as knowledge),
structure and transformed the XML form into a set of entity- affective (growth in feelings or emotional areas known as attitude
relationship models, thus describing the data in systematic and or self), and psychomotor (manual or physical skills known as
more transparent manner. Fig. 2 represents a conceptual data skills). We proposed a special script for transforming the MeSH
model covering all essential MeSH parameters for curriculum tree structure to linear representation of individual terms. The
description. main reason was to simplify the links between the keywords and

Descargado para Maria Fernanda Prada Reyes ([email protected]) en University Foundation Juan N Corpas de ClinicalKey.es por Elsevier en septiembre 27, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.
 M. Komenda et al. / Computers in Biology and Medicine 63 (2015) 74–82 79

the particular learning units. For the purpose of further analysis, the thematic categorisation of all learning units belonging to his/
we merged the data (see Table 1) about learning units (unitName, her expertise; the guarantor should, therefore, be able to define
unitSection, unitClass) with data describing MeSH keywords potential discrepancies very quickly, on relatively general level of
(meshName, meshPreferred, meshSemanticType, meshTreeId, curriculum description, without the necessity of performing in-
meshTreeLevel, meshCategory, meshSubcategory). The final data- depth analysis. A view across sections, such as the comparison of
set contains information about 1347 learning units, which were categorizations between selected sections, can be classified as
described by 3224 varied MeSH terms (from one to five terms per outputs primarily intended for the faculty management. Like in
learning unit) and classified into 133 categories and subcategories. the first case, potential overlaps—whether more or less desirable—
can be identified at first glance.
The second analysis consists in identifying links based on the
3.1.4. Results assigned MeSH keywords in order to reveal the structure of
Two reports relating to MeSH categories in accordance with the individual sections. At the present time, the OPTIMED portal
created learning units are described here. The primary motivation contains a very comprehensive description of the entire
of our effort was to allow medical curriculum designers, senior
guarantors and the faculty management answer questions based
on aggregated data from the OPTIMED portal platform. Several
analytical dashboards were built and prepared for further review
and evaluation. The implications of these findings can help
academics in their curriculum redesign activities, and provide a
clear overview of the curriculum structure.
The first analytical output is a view of the representation of
categories and subcategories according to the MeSH biomedical
dictionary based on learning units, which have been created with
the aim of describing in detail the education in General Medicine
field of study at the Faculty of Medicine of the Masaryk University.
In the OPTIMED project, the study of medicine was systematically
divided into four sections (theoretical sciences, surgical sciences,
diagnostics and neurosciences, internal medicine). With respect to
the fact that suitable MeSH classifiers must be obligatorily
assigned to each learning unit, the available links to a tree
structure and to multilevel categorisation make it possible to draw
summarising graphs. Fig. 3 shows the comparison of two sections
of education (surgical sciences and theoretical sciences) according
to the most frequent subcategories.
The main MeSH categories are very general (only 16 areas); in
this case, therefore, we decided to work with subcategories (a total
of 117 subcategories), which are more detailed and thus make it
possible to define clearer and more understandable reports. Fig. 4. Global association map of all MeSH terms occurring in one particular section
Similar to other outputs of this analysis, a fully standardised of the medical curriculum. Individual clusters represent various subsets of the
dictionary is employed containing set phrases and specialised MeSH keywords describing the same learning units. Less often used keywords are
located in smaller clumps out of the main cluster including the majority of MeSH
terminology commonly used in medicine. These features provide terms. (For interpretation of the references to colour in this figure, the reader is
a simple overview to the guarantor of a given section, describing referred to the web version of this article.)

Fig. 3. The MeSH categories according to number of learning units (only 20 most frequent categories are visualized).

Descargado para Maria Fernanda Prada Reyes ([email protected]) en University Foundation Juan N Corpas de ClinicalKey.es por Elsevier en septiembre 27, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.
80 M. Komenda et al. / Computers in Biology and Medicine 63 (2015) 74–82

curriculum. It is, thus, not humanly possible to go through the specific case, the term “inflammation” is directly linked to 10 other
entire contents or to imagine how the individual learning units, MeSH terms. Generally, these association maps visualize inter-
courses, or branches of medicine are interrelated. We have, there- connections among the standardized MeSH classification used for
fore, again used the MeSH classification to suggest a report describing the medical curriculum. Each node represents one
displaying the so-called association maps, i.e. links among MeSH keyword and its size is calculated from an absolute sum of
keywords according to their use in individual learning units. For associations (measured by the Jaccard coefficient) for the given
the sake of clarity, each of the resulting visualizations is divided keyword. Since an association between two terms is always
according to the four above-mentioned sections. The graphs are reversible, also each edge is doubled. At the bottom of this
interactive, making it possible to zoom in on a selected segment, visualization are located the MeSH keywords, which are used only
up to the level of a specific MeSH keyword. This feature makes it for describing a single learning unit and are not, in our data set,
very simple for an expert in a given field to critically evaluate associated with any other MeSH keyword.
whether the coverage of a given section makes sense, and whether
all relevant areas are represented. Figs. 4 and 5 demonstrate the
association map of the Theoretical Sciences section. The first view 3.1.5. Discussion and future visions
is mostly general, showing all connections. As an example, the For the purposes of optimizing medical education encompass-
term “inflammation” is marked in red. The second view corre- ing a correctly compiled and balanced curriculum, we proposed
sponds to a several-fold zoom that, however, is still illegible, and it the formal database metadata arrangement including standar-
is not obvious how the terms are interconnected. By contrast, Fig. 6 dized MeSH vocabulary, which was chosen in accordance with
represents the most detailed view of a given element of the graph. the proven scoping review. In our particular case, the database of
At maximum zoom, the user can easily reveal all terms that have medical curriculum contains more than 140 courses, which are
been used to describe the same or related learning units. In this described by 1347 learning units and 7075 learning outcomes, i.e.

Fig. 5. A more detailed association map of selected MeSH terms occurring in one particular section of the medical curriculum. (For interpretation of the references to colour
in this figure, the reader is referred to the web version of this article.)

Descargado para Maria Fernanda Prada Reyes ([email protected]) en University Foundation Juan N Corpas de ClinicalKey.es por Elsevier en septiembre 27, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.
 M. Komenda et al. / Computers in Biology and Medicine 63 (2015) 74–82 81

Fig. 6. The deepest level of association map representing all connections between MeSH keywords occurring in theoretical sciences from the perspective of the term
“Inflammation”.

more than 2500 pages of text. From the perspective of human this system provides search assistance through a rich set of lexical
cognition abilities, it is not possible to carefully read and verify all look-up facilities, which are based on MeSH's complex structure
of the medical curriculum content. Classification by standardized including terms with tree structured contexts. Moreover, two
medical vocabulary can significantly influence effectiveness and data-analytical reports depicting the results obtained from fre-
shorten the total time requirements placed on curriculum inspec- quency and association analyses of the particular medical curri-
tion. Before we integrated the MeSH thesaurus, we had to take culum while employing the MeSH thesaurus are described. They
into consideration the fundamental properties and characteristics represent the aggregated tabular and graphic outputs including
of potential candidates such as linear/branched structure (how the interactive visualizations, which play an essential role during
terms are organized and interconnected), annual updates and evaluation of the whole curriculum. The main aim is to identify
multilingual translation at regular intervals, scope (specific focus information rich data relations and offer clear and transparent
or general purpose), data format and licensing. MeSH fully met our overview for simpler and easier understanding of the curriculum
visions and fulfilled the requirements. Two analytical reports show structure. We believe that our approach can be easily generalized
different unique approaches on how to clearly visualize a com- and used by any higher education institution without any meth-
pleted structure and the relations between the vast seas of data. By odological or technological restrictions.
applying the described dashboards, we are able to determine the
similarity between individual sections of medical education
explored at the level of selected attributes, such as MeSH key- Conflict of interest
words or subcategories. We have also classified the set of learning
units into one of several predefined classes according to Bloom's None.
taxonomy, which represents intellectual behaviour into three
various levels: cognitive, affective, and psychomotor.
In the future, the idea of Automatic Term Mapping (ATM) of Acknowledgement
medical curriculum data is planned. The ATM technique [56] was
developed by the US National Center of Biotechnology Information The authors were supported from the grants project OPTIMED -
for mapping end-user queries to the MeSH thesaurus and other OPTImized MEDical education: horizontal and vertical connec-
search fields. The basic aim will be to improve information tions, innovations and efficiency in practice reg. no: CZ.1.07/2.2.00/
retrieval as structured information: searching indexes instead of 28.0042, which is funded by the European Social Fund and the
only the free text. Furthermore, with proper use of structured state budget of the Czech Republic. Special thanks are dedicated to
vocabularies and medical curriculum standards, which define the the management of Faculty of Medicine at Masaryk University
core curriculum data for education programmes in health profes- represented by a vice-dean for education in clinical branches prof.
sions, we will be able to automatically compare and exchange data Jaroslav Štérba and dean of the faculty prof. Jiří Mayer.
between various educational systems. For example, a particular
comparative analysis of a medical curriculum against topics in the References
MeSH (or other) lexicon, where the number of articles is rapidly
increasing, could help to identify areas within the already crowded [1] A. Holzinger, Biomedical Informatics: Discovering Knowledge in Big Data, Springer,
curriculum that may need more (or less) emphasis, or perhaps just 2014. 〈http://www.google.com/books?hl=cs&lr=&id=-FMlBAAAQBAJ&oi=fnd&pg=-
PR7&dq=Biomedicalþ Informatics:þDiscoveringþ Knowledge
a mention to students as an area to watch. þinþ BigþDa&ots=POzGZuK9Bn&sig=w5mDOXcVJskGqgnzFH-G4L6g_Sc〉 (accessed
01.22.2015).
[2] J.S. Luciano, B. Andersson, C.R. Batchelor, O. Bodenreider, T. Clark, C.K. Denney,
et al., The translational medicine ontology and knowledge base: driving
4. Conclusions personalized medicine by bridging the gap between bench and bedside., J
Biomed. Semant. 2 (2011) S1.
In this paper, the global summary of specialized medical [3] M. Dvořák, Progress of information technology in healthcare, current state,
outlook toward future, Mefanet J. 1 (2013) 23–28.
terminologies, nomenclatures, coding and classification systems [4] Handbook of Medical Informatics, in: J.H. van Bemmel, M.A. Musen, J.C. Helder
was introduced. We identified the significant problem of selecting (Eds.), Bohn Stafleu Van Loghum ; Springer Verlag, AW Houten, Netherlands :
and integrating proper vocabulary for computer-processable Heidelberg, Germany, 1997.
[5] E. Coiera, Guide to Health Informatics, 2nd ed., Arnold ; Distributed in the USA
description in compliance with particular requirements. Based
by Oxford University Press, London : New York, NY, 2003.
on the scoping review, we adopted the MeSH biomedical diction- [6] S. de Lusignan, Codes, classifications, terminologies and nomenclatures:
ary, where the main objective was to classify learning activities definition, development and application in practice, Inform. Prim. Care 13
appearing in the medical curriculum of General Medicine. Thus, (2005) 65–70.
[7] R. Cornet, N. de Keizer, Forty years of SNOMED: a literature review, BMC Med.
we have integrated MeSH into our original web-based system for Inform. Decis. Mak. 8 (Suppl 1) (2008) S2. http://dx.doi.org/10.1186/1472-
innovating and harmonizing the medical curriculum. Currently, 6947-8-S1-S2.

Descargado para Maria Fernanda Prada Reyes ([email protected]) en University Foundation Juan N Corpas de ClinicalKey.es por Elsevier en septiembre 27, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.
82 M. Komenda et al. / Computers in Biology and Medicine 63 (2015) 74–82

[8] D. Lee, N. de Keizer, F. Lau, R. Cornet, Literature review of SNOMED CT use, J. [33] H.M. Daudt, C. van Mossel, S.J. Scott, Enhancing the scoping study methodol-
Am. Med. Inform. Assoc. JAMIA 21 (2014) e11–e19. http://dx.doi.org/10.1136/ ogy: a large, inter-professional team’s experience with Arksey and O’Malley’s
amiajnl-2013-001636. framework, BMC Med. Res. Methodol. 13 (2013) 48. http://dx.doi.org/10.1186/
[9] R.M. Harden, Approaches to research in medical education, Med. Educ. 20 1471-2288-13-48.
(1986) 521–531. http://dx.doi.org/10.1111/j.1365-2923.1986.tb01394.x. [34] C. Adelman, The Bologna Process for US Eyes: Re-Learning Higher Education in
[10] R.M. Harden, Ten questions to ask when planning a course or curriculum, Med. the Age of Convergence, Institute for Higher Education Policy, Washington,
Educ. 20 (1986) 356–365. DC, 2009, accessed 22.01.2015.
[11] S.A. Wartman, Research in medical education: the challenge for the next [35] D.A. Lindberg, B.L. Humphreys, A.T. McCray, The unified medical language
decade, Acad. Med. 69 (1994) 608–614. http://dx.doi.org/10.1097/00001888- system, Methods Inf. Med. 32 (1993) 281–291.
199408000-00003. [36] J. Anderson, The computer: medical vocabulary and information, Br. Med. Bull.
[12] M.H. Davis, R.M. Harden, Planning and implementing an undergraduate 24 (1968) 194–198.
medical curriculum: the lessons learned, Med. Teach. 25 (2003) 596–608. [37] O. Bodenreider, The unified medical language system (UMLS): integrating
http://dx.doi.org/10.1080/0142159032000144383. biomedical terminology, Nucleic Acids Res. 32 (2004) D267–D270.
[13] R.M. Harden, Learning outcomes as a tool to assess progression, Med. Teach. [38] Unified Medical Language System (UMLS), (n.d.), 〈http://www.nlm.nih.gov/
29 (2007) 678–682. http://dx.doi.org/10.1080/01421590701729955. research/umls/〉 (accessed 22.01.2015).
[14] R.M. Harden, AMEE Guide No. 21: curriculum mapping: a tool for transparent [39] Performance evaluation of unified medical language system’s synonyms
and authentic teaching and learning, Med. Teach. 23 (2001) 123–137. expansion to query PubMed, BMC Med. Inform. Decis. Mak. 12, 2012 12–17.
[15] R.H. Ellaway, S. Albright, V. Smothers, T. Cameron, T. Willett, Curriculum inventory: http://dx.doi.org/10.1186/1472-6947-12-12.
modeling, sharing and comparing medical education programs, Med. Teach. 36 [40] Fact SheetMedical Subject Headings (MeSHs), (n.d.). 〈http://www.nlm.nih.
(2014) 208–215. http://dx.doi.org/10.3109/0142159X.2014.874552. gov/pubs/factsheets/mesh.html〉 (accessed 12.07.2012).
[16] R. Ellaway, V. Smothers, ANSI/MEDBIQ CI.10.1-2013 Curriculum Inventory [41] D. Rotolo, L. Leydesdorff, Matching MEDLINE/PubMed data with web of
Specifications, 2013, 〈http://www.medbiq.org/sites/default/files/files/Curriculu science (WoS): a routine in R language, J. Assoc. Inf. Sci. Technol. (2014),
mInventorySpecification.pdf〉. http://dx.doi.org/10.1002/asi.23385.
[17] A. Haig, R. Ellaway, M. Dozier, D. Liu, J. McKendree, METRO-the creation of a [42] A.P. Davis, T.C. Wiegers, M.C. Rosenstein, C.J. Mattingly, MEDIC: a practical
taxonomy for medical education, Health Inf. Libr. J. 21 (2004) 211–219. http: disease vocabulary used at the Comparative Toxicogenomics Database, Data-
//dx.doi.org/10.1111/j.1471-1842.2004.00534.x. base: The Journal of Biological Databases and Curation 2012 (2012) bar065,
[18] A. Haig, M. Dozier, D. Liu, J. Mckendree, T. Roper, C. Selai, METRO taxonomy – http://dx.doi.org/10.1093/database/bar065.
progress report on assessment, Med. Teach. 27 (2005) 155–157. http://dx.doi. [43] W. ICD, Classification of mental and behavioural disorders, Edinb. Lond. Melb.
org/10.1080/01421590500091102. N. Y. Tokyo Churchill Livingstone, 1991.
[19] W.E. Blaum, A. Jarczweski, F. Balzer, P. Stötzner, O. Ahlers, Towards Web 3.0: [44] F. Gjertsen, S. Bruzzone, M.E. Vollrath, M. Pace, Ø. Ekeberg, Comparing ICD-9
taxonomies and ontologies for medical education - a systematic review, GMS Z.
and ICD-10: the impact on intentional and unintentional injury mortality
Med. Ausbild. 30 (1) (2013) Doc13: 9–16. http://dx.doi.org/10.3205/zma000856.
statistics in Italy and Norway, Injury 44 (2013) 132–138. http://dx.doi.org/
[20] C. Bizer, T. Heath, T. Berners-Lee, Linked data - the story so far, Int. J. Semant.
10.1016/j.injury.2012.01.010.
Web Inf. Syst. 5 (2009) 1–22. http://dx.doi.org/10.4018/jswis.2009081901.
[45] P. Ruch, J. Gobeill, C. Lovis, A. Geissbühler, Automatic medical encoding with
[21] E. Méndez, J. Greenberg, Linked data for open vocabularies and HIVE’s global
SNOMED categories, BMC Med. Inform. Decis. Mak. 8 (2008) S6. http://dx.doi.
framework, El Prof. Inf. 21 (2012) 236–244.
org/10.1186/1472-6947-8-S1-S6.
[22] S. Konstantinidis, L. Fernandez-Luque, P. Bamidis, R. Karlsen, The role of
[46] P.L. Elkin, S.H. Brown, C.S. Husser, B.A. Bauer, et al., Evaluation of the content
taxonomies in social media and the semantic web for health education. A
coverage of SNOMED CT: ability of SNOMED clinical terms to represent clinical
study of SNOMED CT terms in YouTube health video tags, Methods Inf. Med.
problem lists, Mayo Clin. Proc. 81 (2006) 741–748.
52 (2013) 168–179. http://dx.doi.org/10.3414/ME12-02-0005.
[47] C. Fluit, M. Sabou, F. van Harmelen, Ontology-Based Information Visualization:
[23] P.L. Whetzel, N.F. Noy, N.H. Shah, P.R. Alexander, C. Nyulas, T. Tudorache, et al.,
Toward Semantic Web Applications, in: V.G.Ds. MSc, C.C. BSc MSc (Eds.), Vis.
BioPortal: enhanced functionality via new Web services from the National
Semantic Web, Springer London, 2006, pp. 45–58. 〈http://link.springer.com/
Center for Biomedical Ontology to access and use ontologies in software
chapter/10.1007/1-84628-290-X_3〉 (accessed 22.01.2015).
applications, Nucleic Acids Res. 39 (2011) W541–W545.
[48] T. Bekhuis, D. Demner-Fushman, R. Crowley, Comparative effectiveness
[24] M. Komenda, D. Schwarz, J. Feberová, S. Štípek, V. Mihál, L. Dušek, Medial
faculties educational network: Multidimensional quality assessment, Comput. research designs: an analysis of terms and coverage in Medical Subject
Methods Progr. Biomed. 108 (2012) 900–909. Headings (MeSH) and Emtree, J. Med. Libr. Assoc. 101 (2013) 92–100. http:
[25] D. Schwarz, P. Štourač, M. Komenda, H. Harazim, M. Kosinová, J. Gregor, et al., //dx.doi.org/10.3163/1536-5050.101.2.004.
Interactive algorithms for teaching and learning acute medicine in the [49] M. Taboada, R. Lalin, D. Martinez, An automated approach to mapping external
network of medical faculties MEFANET, J. Med. Internet Res. 15 (2013) e135. terminologies to the UMLS, IEEE Trans. Biomed. Eng. 56 (2009) 1598–1605.
http://dx.doi.org/10.2196/jmir.2590. http://dx.doi.org/10.1109/TBME.2009.2015651.
[26] D. Schwarz, L. Dušek, M. Bareš, Computer applications, Systems and Networks [50] C.P. Langlotz, RadLex: a new method for indexing online educational materials
for Medical Education, Masaryk University, Institute of Biostatistics and 1, Radiographics 26 (2006) 1595–1597.
Analyses, Facta Medica, Brno, 2014. [51] M.W. Shore, D.L. Rubin, C.E. Kahn Jr, Integration of imaging signs into RadLex,
[27] M. Komenda, D. Schwarz, C. Vaitsis, N. Zary, J. Štěrba, L. Dušek, OPTIMED J. Digit. Imaging 25 (2012) 50–55.
Platform: Curriculum Harmonisation System for Medical and Healthcare [52] Dublin Core Metadata Element Set, Version 1.1: Reference Description, (n.d.).
Education, in: Madrid, 2015. 〈http://dublincore.org/documents/1999/07/02/dces/〉.
[28] C. Vaitsis, G. Nilsson, N. Zary, Visual analytics in healthcare education: [53] X. Ochoa, J. Klerkx, B. Vandeputte, E. Duval, On the use of learning object
exploring novel ways to analyze and represent big data in undergraduate metadata: the GLOBE experience, in: Proceedings of the Ubiquitous Learn.,
medical education, PeerJ 2 (2014) e683. http://dx.doi.org/10.7717/peerj.683. Springer, 2011, pp. 271–284. http://link.springer.com/chapter/10.1007/
[29] C. Vaitsis, G. Nilsson, N. Zary, Big data in medical informatics: improving 978-3-642-23985-4_22 (accessed 18.02.2015).
education through visual analytics, Stud. Health Technol. Inf. 205 (2014) [54] A. Maratea, A. Petrosino, M. Manzo, Automatic generation of SCORM com-
1163–1167. pliant metadata for portable document format files, in: Proceedings of the
[30] M. Komenda, D. Schwarz, Visual analytics in environmental research: a survey 13th International Conference on Computer Systems and Technologies, ACM,
on challenges, methods and available tools, in: J. Hřebíček, G. Schimak, 2012, pp. 360–367. 〈http://dl.acm.org/citation.cfm?id=2383328〉 (accessed
M. Kubásek, A.E. Rizzoli (Eds.), Environmental Software Systems Fostering 18.02.2015).
Information Sharing, Springer Berlin Heidelberg, Berlin, Heidelberg, 2013, [55] K. Beck, C. Andres, Extreme Programming Explained: Embrace Change, Addison-
pp. 618–629, accessed 25.02.2015. Wesley Professional, 2004 〈http://dl.acm.org/citation.cfm?id=1076267〉, accessed
[31] H. Arksey, L. O’Malley, Scoping studies: towards a methodological framework, 21.03.2014.
Int. J. Soc. Res. Methodol. 8 (2005) 19–32. [56] N.C. for B. Information, U.S.N.L. of M. 8600R. Pike, B. MD, 20894 USA, PubMed
[32] D. Levac, H. Colquhoun, K.K. O’Brien, Scoping studies: advancing the methodology, Help, (2014). 〈http://www.ncbi.nlm.nih.gov/books/NBK3827/〉 (accessed
Implement. Sci. 5 (2010) 69. http://dx.doi.org/10.1186/1748-5908-5-69. 22.01.2015).

Descargado para Maria Fernanda Prada Reyes ([email protected]) en University Foundation Juan N Corpas de ClinicalKey.es por Elsevier en septiembre 27, 2019.
Para uso personal exclusivamente. No se permiten otros usos sin autorización. Copyright ©2019. Elsevier Inc. Todos los derechos reservados.