文档下载 文档下载导航
设为首页 | 加入收藏
搜索 请输入内容:  
 导航当前位置: 文档下载 > 所有分类 > A study on richer syntactic dependencies for structured language modeling
侵权投诉

A study on richer syntactic dependencies for structured language modeling

We study the impact of richer syntactic dependencies on the performance of the structured language model (SLM) along three dimensions: parsing accuracy (LP/LR), perplexity (PPL) and worderror-rate (WER, N-best re-scoring). We show that our models achieve a

Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL), Philadelphia, July 2002, pp. 191-198.

AStudyonRicherSyntacticDependenciesforStructuredLanguage

Modeling

PengXu

CenterforLanguageandSpeechProcessing

JohnsHopkinsUniversityBaltimore,MD21218xp@clsp.jhu.edu

CiprianChelbaMicrosoftResearchOneMicrosoftWayRedmond,WA98052chelba@microsoft.com

FrederickJelinek

CenterforLanguageandSpeechProcessing

JohnsHopkinsUniversityBaltimore,MD21218jelinek@clsp.jhu.edu

Abstract

Westudytheimpactofrichersyntac-ticdependenciesontheperformanceofthestructuredlanguagemodel(SLM)alongthreedimensions:parsingaccu-racy(LP/LR),perplexity(PPL)andword-error-rate(WER,N-bestre-scoring).Weshowthatourmodelsachieveanim-provementinLP/LR,PPLand/orWERoverthereportedbaselineresultsus-ingtheSLMontheUPennTreebankandWallStreetJournal(WSJ)corpora,respectively.Analysisofparsingper-formanceshowscorrelationbetweenthequalityoftheparser(asmeasuredbypre-cision/recall)andthelanguagemodelper-formance(PPLandWER).AremarkablefactisthattheenrichedSLMoutperformsthebaseline3-grammodelintermsofWERby10%whenusedinisolationasasecondpass(N-bestre-scoring)languagemodel.

about40%.Thekeytoachievingthisreductionisagoodguessofthe nalbestparseforagivensen-tenceasitisbeingtraversedleft-to-right,whichismuchharderthan ndingthe nalbestparsefortheentiresentence,asitissoughtbyaregularstatisticalparser.Nevertheless,itisexpectedthattechniquesdevelopedinthestatisticalparsingcommunitythataimatrecoveringthebestparseforanentiresen-tence,i.e.asjudgedbyahumanannotator,shouldalsobeproductiveinenhancingtheperformanceofalanguagemodelthatusessyntacticstructure.

Thestatisticalparsingcommunityhasusedvar-iouswaysofenrichingthedependencystructureunderlyingtheparametrizationoftheprobabilisticmodelusedforscoringagivenparsetree(Charniak,2000)(Collins,1999).Recently,suchmodels(Char-niak,2001)(Roark,2001)havebeenshowntoout-performtheSLMintermsofbothPPLandWERontheUPennTreebankandWSJcorpora,respectively.In(ChelbaandXu,2001),asimplewayofenrichingtheprobabilisticdependenciesintheCONSTRUC-TORcomponentoftheSLMalsoshowedbetterPPLandWERperformance;thesimplemodi ca-tiontothetrainingprocedurebroughttheWERper-formanceoftheSLMtothesamelevelwiththebestasreportedin(Roark,2001).

Inthispaper,wepresentthreesimplewaysofenrichingthesyntacticdependencystructureintheSLM,extendingtheworkin(ChelbaandXu,2001).Theresultsshowthatanimprovedparser(asmea-suredbyLP/LR)isindeedhelpfulinreducingthePPLandWER.Anotherremarkablefactisthatforthe rsttimealanguagemodelexploitingelemen-

1Introduction

Thestructuredlanguagemodeluseshiddenparsetreestoassignconditionalword-levellanguagemodelprobabilities.Asexplainedin(ChelbaandJelinek,2000),Section4.4.1,ifthe nalbestparseisusedtobetheonlyparse,thereductioninPPL—relativetoa3-grambaseline—usingtheSLM’sheadwordparametrizationforwordpredictionis

第1页

你可能喜欢

热门文档

相关文档

站点地图 | 文档上传 | 侵权投诉 | 手机版
本站所有资源均来自互联网,本站只负责收集和整理,均不承担任何法律责任,如有侵权等其它行为请联系我们.
文档下载 Copyright 2013 doc.guandang.net All Rights Reserved.  -  email