This paper compares the retrieval performance of seven public search engines on Thai queries by using a blind evaluation. Two of the engines used in this study are Thaifocused search services, while the remaining engines are large, commercial search engine
MEASURINGTHEEFFECTIVENESSOFPUBLICSEARCHENGINESON
THAIQUERIES
ShisanuTongchimandVirachSornlertlamvanichandHitoshiIsahara
ThaiComputationalLinguisticsLaboratory
112PhahonYothinRd.Pathumthani12120,Thailand
email:{shisanu,virach}@tcllab.org,isahara@nict.go.jp
ABSTRACT
ThispapercomparestheretrievalperformanceofsevenpublicsearchenginesonThaiqueriesbyusingablindeval-uation.TwooftheenginesusedinthisstudyareThai-focusedsearchservices,whiletheremainingenginesarelarge,commercialsearchenginesthathavewidercollec-tionsofwebdataandsupportseverallanguages.There-sultsarecomparedbyMeanaverageprecision(MAP)andMeanreciprocalrankofthe rstcorrectanswer(MRR).Theseresultsarecalculatedfrombinaryrelevancejudg-mentsofthe rst20returnedresults,using56topics.Sta-tisticaltestingshowsthattherearesigni cantdifferencesamongengines.
KEYWORDS
Websearchengine,Blindevaluation,Thaiqueries
1Introduction
Anumberofstudiesexaminingtheperformanceofwebsearchenginesregularlyappearinliterature.Mostofthestudiesinsearchengineevaluationhavebeendonebyus-ingEnglishqueries.AstudybasedonlanguagesotherthanEnglish(i.e.Chinese)wasshownin[1].Toourknowledge,however,nosystematicevaluationofpublicsearchenginesonThaiquerieshasbeenreported.Inthispaper,wecom-paretheperformanceofsevensearchenginesbyusingThaiqueries.Theevaluationiscompletelyblind.Thatis,there-sultsfromdifferentenginesarepooledandpresentedtothejudgesinrandomorder.Noinformationaboutthesearchenginesisshown.Besidescomparingtheperformanceofsearchenginesthroughablindevaluation,wealsodiscussaboutsearchenginepopularitybasedonreferrerdatafromThaiwebsites.Theresultsbasedonananalysisofreferrerdatahavebeenprovidedbythelargestwebstatisticscol-lectorinThailand,Truehits1.Theresultsfromreferrerdatawouldsuggestthepopularityandsuccessofsearchenginesinrealusage.Theseresultswillbecomparedwiththe nd-ingsfromtheblindevaluation.Thiswillsuggestwhetherthereareanycorrelationsbetweensearchenginepopularityandtheresultsofourblindevaluationornot.
ByusingThaitestqueries,achallengingissueen-2doc.guandang.net
Estimating the relative size and overlap of public web search engines ...way of measuring search engine coverage and overlap through random queries. ...
MEASURING THE EFFECTI... 暂无评价 6页 免费 Public perception of pri...... researchers have either manually submitted queries to the search engine’s ...
Search Engines Information Retrieval in Practice ...complex queries and results of query transformation... – Measuring and tuning ranking effectiveness ? ...
The effectiveness of twenty public search engines is evaluated using TREC-inspired methods and a set of 54 queries taken from real Web search logs. The ...
measuring Incorporating new data -Coverage and ...effectiveness 5/N 08: Evaluating Search Engines 8...Queries are the title fields from TREC topics ...
Typicallyusetestcollectionofdocuments,queries, andre...effectiveness measures BigIssuesinIR UsersandInformation...Measuringandimprovingtheefficiencyofsearch e.g.,...
Search Engines:Information Retrieval in Practice...queriesandresultsofquerytransformation e.g.,Boole...Measuringandtuningrankingeffectiveness Performanceanalysis...
Simply, KSA is a process by which search engines place advertising in the...(CTR) CTR is used for measuring an advertisement’s effectiveness because ...
MEASURING THE EFFECTI... 暂无评价 6页 免费 ... how do search engines handle Arabic queries...1992. The effectiveness of stemming for natural-...
search engines are based on retrieval models – ...Performance – Measuring and improving the ... more seriously, the effectiveness of the results...
热门文档