Ngram-based Statistical Machine Translation relies on a standard Ngram language model of tuples to estimate the translation process. In training, this translation model requires a segmentation of each parallel sentence, which involves taking a hard decisio
sent
TrainsetArabicEnglishDevsetArabicTestsetArabic
20k5061006
words180.5k189.2k3.63k7.22k
vocab16.0k7.2k1.18k1.9k
avglen9.09.5OOVs196OOVs356
refs11616
wordsinsourcesideandNULLintargettobelongtothemodeldictionary.
However,resultsshowthatnoneofthesetechniquesisbene- cial.Clearly,andincontrasttoNULLsinthetuplesourceside,NULLsinthetargetsideareausefulmechanismfortheNgrammodelto ndgoodcontextsandsigni cantlyincreaseperformanceregardlessofthetranslationdirection.Thisholdsevenwhenweal-lowtupleswithcontentwordsinsourcesideandNULLintarget.3.3.TranslationNgramsstudy
Tobetterunderstandtheseresults,Table4showsthetuplevocab-ularyobtainedintrainingforeachsegmentation(tupvcb),andrel-evantstatisticsoftranslatedoutput,namelythepercentageoftu-plesusedas1-grams,2-gramsand3-gramsduringtranslation,theaveragetuplelengthobtained(measuringsourceandtargetsidesseparately)andthenumberoftupleswithNULLintarget(inthetranslatedoutput).
Regardingtuplevocabularysize,thealwaysNEXTcriterionproducesthebiggestvocabularyintrainingwhencomparedtoPOSentropyandIBM1weight,whichproducesthesmallest.WhenremovingthetargetNULLs,thevocabularysizeissigni cantlyimproved.
InEng→Spa,weobservethattranslationwithalwaysNEXTandPOSentropysegmentationcriteriatendtousemore3gramsthanIBM1weight,whichcanbeexplainedbytheirconsistencyintakingsegmentationdecisions(theyinvariablytakethesamedecisiongiventhetargetwordsinvolved),whereasIBM1weightdependsonsourceandtargetwordsandismorevariable.
However,usingmore3gramsisnotdirectlycorrelatedwithtranslationscores,andthenumberoftuplestoNULLneedstobetakenintoaccount.ThehighnumberoftuplestoNULLfortheal-waysNEXTcriterionisoutstanding,andtellsusthattranslationisindeedachievingmany3gramsbycatenatingsequencestoNULL,whichdonotnecessarilyachievebetterperformance.InthecaseofIBM1weightandespeciallyofPOSentropy,thenumberoftu-pleswithNULLintargetisstronglyreduced.Whereasthisap-pearstobepositivefortranslationperformance,whencompletelyorpartiallyremovingNULLsintarget(trgNULLandtrgNULL-pos),averagetuplelengthincreases,notonlyinthesourcesidebutalsointhetargetside,andthemodelloosestuplecontextandfallsmuchmoreoftento1gram.Apparently,thishasanegativeeffectintranslationquality.
Therefore,wecanconcludethatthebestrelationshipbetweenhigh-ordertuplecontextandsmallamountoftuplestoNULLisachievedbytheproposedPOSentropysegmentationcriterion.Dif-ferencesaremuchsmallerintheSpa→Engdirection,althoughthesametendencyinnumberoftupleswithtargetNULListobefound,andconclusionsareanalogous.3.4.Translationmodel+featuresresults
Tofurtherevaluatetheimpactofthesesegmentationdifferences,wehavelog-linearlycombinedtheNgramtranslationmodelwithadditionalfeatures,namelytwolexiconfeaturesbasedonIBMmodel1probabilities,astandardtarget3-gramLanguageModelandaconstantwordbonus,andoptimizedaccordingtoBLEUscoreindevelopmentset,assimilarlydonein[3].Table5showsthetranslationresultsforthetwobestsegmentationsforeachtask.Asitcanbeseen,theimprovementofbettersegmentingtupleswithNULLsinsourceispracticallycompensatedbythecontribu-tionofadditionalfeatures,especiallyintheSpa→Engtask.
Table2:TravelExpressionsArabic-Englishcorpusstatistics.
BLEU0.42150.42210.43250.42490.43130.46610.46980.47560.47280.47330.36840.36560.3691
mWER43.9843.6043.4844.4743.7539.3738.7338.6439.2338.7841.8041.9441.91
NIST9.229.199.309.219.299.869.919.959.919.937.167.147.17
E→S
S→E
A→E
alwaysNEXTIBM1weightPOSentropytrgNULLtrgNULLposalwaysNEXTIBM1weightPOSentropytrgNULLtrgNULLposalwaysNEXTIBM1weightPOSentropy
Table3:Translationmodelperformanceforeachsegmentationstrategy.’E’standsforEnglish,’S’forSpanishand’A’forArabic.’POSentropy’inTable3,referingtoeachsegmentationstrategydiscussedinsections2.1to2.3,respectively.
Regardingthelarge-vocabularytasks,theproposedlinguistically-guidedsegmentationoutperformsallotherstrategiessigni cantly,especiallyintheEng→Spadirection.ThisresultisconsistentwiththefactthatSpanishisamoreword-generativelanguagethanEnglish,andtherefore,moreNULLsarefoundintheEnglishsideofextractedtuples.
Eventhoughtheimpactofchangingthesegmentationcrite-rionwhentranslatingintoEnglishissmaller,theimprovementofthePOSentropyapproachissigni cant.Inthesmall-vocabularyAra→Engtaskdifferencesarelesssigni cant,incorrelationwiththefactthatonly7%oftuplescontainNULLsinArabicside,com-paredtothe14%ofEng→Spatask.
Remarkably,whereasIBM1weightprovidesbetterresultsinlarge-vocabularytasksthanthealwaysNEXTcriterion,theresultisoppositeinthesmall-vocabularyAra→Engtask.Ontheotherhand,thePOSentropyapproachprovestobemoregeneralandrobusttoataskchange,achievingbestperformanceinalltasks.3.2.RemovingNULLsintarget
Followingtheideasuggestedinsection2.4,Table3alsopresentsresultswhenapplyingthebestsegmentationcriterion(POSen-tropy)toavoidNULLsinthetuplestargetside,asshowninrowsnamed’trgNULL’and’trgNULLpos’forlarge-vocabularySpanish–Englishtask.The rstreferstoapplyingthecriteriontoalltuples,whereasthelattertoonlyapplyingitwhenthetuplecontainsaPOSofaNoun,AdjectiveorVerb.Theobjectiveofthisistominimiseomissionerrorsbypreventingtupleswithcontent
热门文档