In this paper, we propose a new statistical Japanese dependency parser using a cascaded chunking model. Conventional Japanese statistical dependency parsers are mainly based on a probabilistic model, which is not always efficient or scalable. We propose a
JapaneseDependencyAnalysisusingCascadedChunking
TakuKudoandYujiMatsumotoGraduateSchoolofInformationScience,NaraInstituteofScienceandTechnology
taku-ku,matsu@is.aist-nara.ac.jp
Abstract
Inthispaper,weproposeanewstatisticalJapanesedependencyparserusingacascadedchunkingmodel.ConventionalJapanesestatisticaldepen-dencyparsersaremainlybasedonaprobabilisticmodel,whichisnotalwaysef cientorscalable.Weproposeanewmethodthatissimpleandef cient,sinceitparsesasentencedeterministicallyonlyde-cidingwhetherthecurrentsegmentmodi esthesegmentonitsimmediaterighthandside.Experi-mentsusingtheKyotoUniversityCorpusshowthatthemethodoutperformsprevioussystemsaswellasimprovestheparsingandtrainingef ciency.
1Introduction
DependencyanalysishasbeenrecognizedasabasicprocessinJapanesesentenceanalysis,andanum-berofstudieshavebeenproposed.Japanesedepen-dencystructureisusuallyde nedintermsoftherelationshipbetweenphrasalunitscalledbunsetsusegments(hereafter“segments”).
MostofthepreviousstatisticalapproachesforJapanesedependencyanalysis(FujioandMat-sumoto,1998;Harunoetal.,1999;Uchimotoetal.,1999;Kanayamaetal.,2000;Uchimotoetal.,2000;KudoandMatsumoto,2000)arebasedonaprobabilisticmodelconsistingofthefollowingtwosteps.First,theyestimatemodi cationprob-abilities,inotherwords,howprobableonesegmenttendstomodifyanother.Secondtheoptimalcombi-nationofdependenciesissearchedfromtheallcan-didatesdependencies.Suchaprobabilisticmodelisnotalwaysef cientsinceitneedstocalculatetheprobabilitiesforallpossibledependenciesandcre-ates
inasentence)training(whereexamplesisthenumberpersentence.ofsegmentsInaddition,theprobabilisticmodelassumesthateachpairsofdependencystructureisindependent.
Inthispaper,weproposeanewJapanesedepen-dencyparserwhichismoreef cientandsimplerthantheprobabilisticmodel,yetperformsbetterintrainingandtestingontheKyotoUniversityCorpus.Themethodparsesasentencedeterministicallyonlydecidingwhetherthecurrentsegmentmodi esseg-mentonitsimmediaterighthandside.Moreover,itdoesnotassumetheindependenceconstraintbe-tweendependencies
2AProbabilisticModel
Thissectiondescribesthegeneralformulationoftheprobabilisticmodelforparsingwhichhasbeenap-pliedtoJapanesestatisticaldependencyanalysis.Firstofall,wede neasentenceasasequence
ofsegments
ticstructureasasequenceofdependencyanditspatterns
syntac-meansthatthethissegmentdepends,where
on
(modi es)segment.Inframework,weas-sumethatthedependencysequencesatis esthefollowingtwoconstraints.
1.Japaneseisahead- nallanguage.Thus,ex-ceptfortherightmostone,eachsegmentmod-i esexactlyonesegmentamongthesegments
appearingtoitsright.2.Dependenciesdonotcrossoneanother.Statisticaldependencyanalysisisde nedasasearchingproblemforthedependencypatternthatmaximizestheconditionalprobability
oftheinputsequenceundertheabove-mentionedconstraints.Ifweassumethatthedependencyprob-abilitiesaremutuallyindependent,
canberewrittenas:
Japanese Dependency Structure Analysis Based on Support Vector Machines Taku Kudo and Yuji This paper presents a method of Japanese dependency structure ...
Japanese Dependency Structure Analysis Based on Support Vector Machines Taku Kudo and Yuji This paper presents a method of Japanese dependency structure ...
热门文档