Objective: This paper presents the evaluation of two MeSH? indexing systems for French and English on a parallel corpus. Material and methods: We describe two automatic MeSH indexing systems- MTI for English, and MAIF for French. The French version of the
Evaluation of French and English MeSH Indexing Systems with a Parallel Corpus
Aurélie Névéola, b , James G. Morkc, Alan R. Aronsonc, and Stefan J. Darmonia, b
Laboratoire PSI - FRE 2645 CNRS - INSA de Rouen, France
aneveol@insa-rouen.fr
b
CISMeF - CHU de Rouen, 1, rue de Germont, 76031 Rouen, France
stefan.darmoni@chu-rouen.fr
c
National Library of Medicine, 8600 Rockville Pike, Bethesda, MD 20894, USA
{mork,alan}@nlm.nih.gov
tains 15 million MeSH-indexed resources in English. Since 1995, CISMeF2 (French acronym of Catalogue and Index of Abstract
Medical On-Line Resources) has been carrying on similar work on the most important resources of institutional health
information in French [1]. It currently contains more than
indexing systems for French and English on a parallel corpus.
14,000 resources selected for health professionals (e.g. evi-dence-based resources - practice guidelines & consensus con-ferences- and technical reports), medical students (e.g. lecture dexing systems - MTI for English, and MAIF for French. The
notes), and patients (e.g. patient education handouts). An av-French version of the evaluation resources has been manually
erage of 55 new resources are added each week. Indexing is a indexed with MeSH keyword/qualifier pairs. This professional
decisive step for the efficiency of information retrieval within indexing is used as our gold standard in the evaluation of
both the MEDLINE database and CISMeF catalogue, and it is both systems on keyword retrieval.
also one of the most time consuming tasks for the librarians.
This observation shows that it is necessary to develop auto-precision and recall (78% precision and 21% recall at rank 1,
matic tools to assist the human indexers in their work. Such
vs. 37%. precision and 6% recall for MAIF ). Moreover, the
systems have been developed for MeSH indexing in English
performance of both systems can be optimised by the break-as early as the 1980s [2]. More recently, MeSH indexing tools
age function used by the French system (MAIF), which selects
have also been available for French. This paper presents the
an adaptive number of descriptors for each resource indexed.
results of a comparative evaluation of two MeSH indexing systems, the Medical Text Indexer (MTI), which is used at the NLM to help with the indexing of English resources [9], and systems have features that can benefit each other.
the MesH Automatic Indexer for French (MAIF) which is currently developed within the CISMeF team. The evaluation
Keywords: aims at assessing both the systems performance and the com-plementarity of the methods implemented. Automatic Indexing, Internet, Medical Subject Headings, Par-allel Corpus. Materials and Methods
a
Introduction
The Internet has become a very ubiquitous source of informa-tion in numerous fields, including health. Several tools have been developed in order to meet the users' need to find pre-cisely what they are looking for in terms of health information among the numerous documents available online. With Pub-Med®1 and the MEDLINE® database, the U.S. National Li-brary of Medicine (NLM) was among the pioneers in medical information retrieval. Today, the MEDLINE database con-1
This section introduces the different elements involved in the evaluation, viz. the MeSH indexing systems developed both in the United States and France, the evaluation (parallel) corpus and the evaluation methods. NLM - Medical Text Indexer (MTI)
MTI results from the combination of two MeSH Indexing methods. These methods are a Natural Language Processing (NLP) approach based on MetaMap Indexing (MMI) and a
2
doc.guandang.net? doc.guandang.net
Text indexing and search took place using Xapian ...abstract, (ii) co-occurrence in the same ...(2008) Evaluation of text-mining systems for ...
Text indexing and search took place using Xapian ...abstract, (ii) co-occurrence in the same ...(2008) Evaluation of text-mining systems for ...
热门文档