We study the impact of richer syntactic dependencies on the performance of the structured language model (SLM) along three dimensions: parsing accuracy (LP/LR), perplexity (PPL) and worderror-rate (WER, N-best re-scoring). We show that our models achieve a
Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), Philadelphia, July 2002, pp. 191-198.
AStudyonRicherSyntacticDependenciesforStructuredLanguage
Modeling
PengXu
CenterforLanguageandSpeechProcessing
JohnsHopkinsUniversityBaltimore,MD21218xp@clsp.jhu.edu
CiprianChelbaMicrosoftResearchOneMicrosoftWayRedmond,WA98052chelba@microsoft.com
FrederickJelinek
CenterforLanguageandSpeechProcessing
JohnsHopkinsUniversityBaltimore,MD21218jelinek@clsp.jhu.edu
Abstract
Westudytheimpactofrichersyntac-ticdependenciesontheperformanceofthestructuredlanguagemodel(SLM)alongthreedimensions:parsingaccu-racy(LP/LR),perplexity(PPL)andword-error-rate(WER,N-bestre-scoring).Weshowthatourmodelsachieveanim-provementinLP/LR,PPLand/orWERoverthereportedbaselineresultsus-ingtheSLMontheUPennTreebankandWallStreetJournal(WSJ)corpora,respectively.Analysisofparsingper-formanceshowscorrelationbetweenthequalityoftheparser(asmeasuredbypre-cision/recall)andthelanguagemodelper-formance(PPLandWER).AremarkablefactisthattheenrichedSLMoutperformsthebaseline3-grammodelintermsofWERby10%whenusedinisolationasasecondpass(N-bestre-scoring)languagemodel.
about40%.Thekeytoachievingthisreductionisagoodguessofthe nalbestparseforagivensen-tenceasitisbeingtraversedleft-to-right,whichismuchharderthan ndingthe nalbestparsefortheentiresentence,asitissoughtbyaregularstatisticalparser.Nevertheless,itisexpectedthattechniquesdevelopedinthestatisticalparsingcommunitythataimatrecoveringthebestparseforanentiresen-tence,i.e.asjudgedbyahumanannotator,shouldalsobeproductiveinenhancingtheperformanceofalanguagemodelthatusessyntacticstructure.
Thestatisticalparsingcommunityhasusedvar-iouswaysofenrichingthedependencystructureunderlyingtheparametrizationoftheprobabilisticmodelusedforscoringagivenparsetree(Charniak,2000)(Collins,1999).Recently,suchmodels(Char-niak,2001)(Roark,2001)havebeenshowntoout-performtheSLMintermsofbothPPLandWERontheUPennTreebankandWSJcorpora,respectively.In(ChelbaandXu,2001),asimplewayofenrichingtheprobabilisticdependenciesintheCONSTRUC-TORcomponentoftheSLMalsoshowedbetterPPLandWERperformance;thesimplemodi ca-tiontothetrainingprocedurebroughttheWERper-formanceoftheSLMtothesamelevelwiththebestasreportedin(Roark,2001).
Inthispaper,wepresentthreesimplewaysofenrichingthesyntacticdependencystructureintheSLM,extendingtheworkin(ChelbaandXu,2001).Theresultsshowthatanimprovedparser(asmea-suredbyLP/LR)isindeedhelpfulinreducingthePPLandWER.Anotherremarkablefactisthatforthe rsttimealanguagemodelexploitingelemen-
1Introduction
Thestructuredlanguagemodeluseshiddenparsetreestoassignconditionalword-levellanguagemodelprobabilities.Asexplainedin(ChelbaandJelinek,2000),Section4.4.1,ifthe nalbestparseisusedtobetheonlyparse,thereductioninPPL—relativetoa3-grambaseline—usingtheSLM’sheadwordparametrizationforwordpredictionis
A study on richer syntactic dependencies for structured language modeling The paper investigates the use of richer syntactic dependencies in the structured ...
Discriminative syntactic... 暂无评价 8页 免费 A study on richer syntac.....1. INTRODUCTION The structured language model uses hidden parse trees to ...
Discriminative syntactic... 暂无评价 8页 免费 A study on richer syntac.....[7] Ciprian Chelba and Frederick Jelinek, “Structured language modeling,” ...
A study on richer syntac... 暂无评价 8页 ... Language Modeling 暂无评价 63页 免费 Structured ... enabling the use of long distance dependencies. ...
and syntactic structure for language modeling...更多>>Structured language models for speech ...We show that non-headword dependencies contribute ...
A study on richer syntac... 暂无评价 8页 ... Language Modeling 暂无评价 63页 免费 Structured ... enabling the use of long distance dependencies. ...
Discriminative syntactic... 暂无评价 8页 免费 Morphology-based languag... ...Structured language modeling for speech recognition. In Proceedings of NLDB99....
Richer Syntactic Dependencies for Structured Language Modeling Ciprian Chelba STG Microsoft Research chelba@microsoft.com Peng Xu CLSP Johns Hopkins University xp...
Structured Language Modeling for Speech Recognition A new language model for speech recognition is presented. The model develops hidden hierarchical syntactic-...
热门文档