29
COMPARING CORPORA
Adam Kilgarriff
Source: International Journal of Corpus Linguistics, 6:1 (2001), 97–133.
Abstract
Corpus linguistics lacks strategies for describing and compar-
ing corpora. Currently most descriptions of corpora are textual,
and questions such as ‘what sort of a corpus is this?’, or ‘how
does this corpus compare to that?’ can only be answered
impressionistically. This paper considers various ways in which
different corpora can be compared more objectively. First we
address the issue, ‘which words are particularly characteristic
of a corpus?’, reviewing and critiquing the statistical methods
which have been applied to the question and proposing the use
of the Mann-Whitney ranks test. Results of two corpus com-
parisons using the ranks test are presented. Then, we consider
measures for corpus similarity. After discussing limitations of
the idea of corpus similarity, we present a method for evaluat-
ing corpus similarity measures. We consider several measures
and establish that a χ2-based one performs best. All methods
considered in this paper are based on word and ngram fre-
quencies; the strategy is defended.
1Introduction
There is a void at the heart of corpus linguistics. The name puts ‘corpus’ atthe centre of the discipline.1 In any science, one expects to nd a usefulaccountof how its central constructs are taxonomised and measured, and how thesubspecies compare. But to date, corpus linguistics has measured corporaonly in the most rudimentary ways, ways which provide no leverage on thedifferent kinds of corpora there are. The terms it has used for taxonomisingcorpora have been unrelated to any measurement: a corpus is describedas “Wall Street Journal” or “transcripts of business meetings” or “foreignlearners’ essays (intermediate grade)”, but if a new corpus is to be com-pared with existing ones, there are no methods for quantifying how itstands


in relation to them.
Comparing 暂无评价 6页 免费 Comparing top k 25页 免费 Comparing and Contrastin... 6页 1财富值 Comparing Media System 59页 免费 Comparing corpora 29页...
Comparing 暂无评价 6页 免费 Comparing top k 25页 免费 Comparing and Contrastin... 6页 1财富值 Comparing Media System 59页 免费 Comparing corpora 29页...
Comparing corpora with W... 7页 免费 A new approach for cost ... 暂无评价 9页 免费 奔驰On-Site Evaluation t... 14页 5财富值 The ECORP ...
We welcomed contributions concerned with measuring and comparing corpora from any field. References Biber, Douglas. 1988. Variation across speech and writing....
In this paper, we present a method for comparing Lexicalized Tree Ad-joining Grammars extracted from annotated corpora for three lan-guages: English, ...
When comparing corpora of different sizes, we need to normalize the frequencies to a common base (e.g. per million tokens) – Normalised freq = raw ...
Monitor corpora: gigantic, ever moving stores of text. It has the capacity... 1.comparing different usages of the same word; 2.analysing keywords; 3....
churen@gate.sinica.edu.tw Abstract This paper proposes a corpora-based approach in comparing the Mapping Principles for economy metaphors in English and ...
deals with the principles and practice of using corpora in language study. ...(in SLA) a way of comparing two languages to identify their similarities ...
1998. Tag sequences in learner corpora: A key to interlanguage grammar and...Comparing native and learner perspectives on English grammar: A study of ...
热门文档