文档下载 文档下载导航
设为首页 | 加入收藏
搜索 请输入内容:  
 导航当前位置: 文档下载 > 所有分类 > Agreeing to disagree Search engines and their public interfaces
侵权投诉

Agreeing to disagree Search engines and their public interfaces

Google, Yahoo and MSN all provide both web user interfaces (WUIs) and application programming interfaces (APIs) to their collections. Whether building collections of resources or studying the search engines themselves, the search engines request that resea

AgreeingtoDisagree:SearchEnginesandTheir

PublicInterfaces

OldFrankDominionMcCown

UniversityComputerScienceDepartmentNorfolk,Virginia,USA23529

fmccown@cs.odu.edu

ABSTRACT

Google,YahooandMSNallprovidebothwebuserinterfaces(WUIs)andapplicationprogramminginterfaces(APIs)totheircollections.Whetherbuildingcollectionsofresourcesorstudyingthesearchenginesthemselves,thesearchenginesrequestthatresearchersusetheirAPIsandnot“scrape”theWUIs.However,anecdotalevidencesuggeststheinterfacesproducedi erentresults.Weprovidethe rstindepthquan-titativeanalysisoftheresultsproducedbytheGoogle,MSNandYahooAPIandWUIinterfaces.Wehavequeriedbothinterfacesfor vemonthsandfoundsigni cantdiscrepanciesbetweentheinterfacesinseveralcategories.Ingeneral,wefoundMSNtoproducethemostconsistentresultsbetweentheirtwointerfaces.Our ndingssuggestthattheAPIin-dexesarenotolder,buttheyareprobablysmallerforGoogleandYahoo.Wealsoexaminedhowsearchresultsdecayovertimeandbuiltpredictivemodelsbasedontheobservedde-cayrates.Basedonour ndings,itcantakeoverayearforhalfofthetop10resultstoapopularquerytobereplacedinGoogleandYahoo;forMSNitmaytakeonly2-3months.CategoriesandSubjectDescriptors:H.3.5[Informa-tionStorageandRetrieval]OnlineInformationServices:Web-basedservices

GeneralTerms:Experimentation,Measurement

Keywords:API,distancemeasurement,searchenginere-sults,searchengineinterfaces

1.INTRODUCTION

Commercialsearchengineshavelongbeenusedinacad-emicstudies.Sometimesthesearchenginesthemselvesarebeingstudied,andsometimestheyareusedtostudytheWeborwebphenomena.Inthepast,researchershaveei-thermanuallysubmittedqueriestothesearchengine’swebuserinterface(WUI),ortheyhavecreatedprogramsthatautomatethetaskofsubmittingqueries.Thereturnedre-

Thisistheauthor’sversionofthework.ItispostedherebypermissionofACMforyourpersonaluse.Notforredistribution.

JCDL’07,June17–22,2007,Vancouver,BritishColumbia,Canada.Copyright2007ACM978-1-59593-644-8/07/0006...$5.00.

OldMichaelDominionL.UniversityNelson

ComputerScienceDepartmentNorfolk,Virginia,USA23529

mln@cs.odu.edu

sultshavebeenprocessedmanuallyorbyprogramsthatrelyonbrittlescreen-scrapingtechniques.

Butdatacollectionmechanismsforsearchengineshavechangedinthepastfewyears.Threeofthemostpopularcommercialsearchengines(Google,MSNandYahoo)havedevelopedfreelyavailableAPIsforaccessingtheirindex,andresearcherscannowusetheseAPIsintheirautomateddatacollectionprocesses.Unfortunately,theAPIsdonotalwaysgivethesameresultsastheWUI.Thelistservesandnews-groupsthatcatertotheAPIcommunitiesarefullofques-tionsregardingtheperceiveddi erencesinresultsbetweenthetwo.Noneofthesearchenginespubliclydisclosethein-nerworkingsoftheirAPIs,sousersareleftwonderingiftheAPIsaregivingsecond-ratedata.Thisanecdotalevidencehasledsomeresearcherstoquestionthevalidityoftheir ndings.Forexample,Bar-YossefandGurevich[8]statethat“duetolegallimitationsonautomaticqueries,weusedtheGoogle,MSN,andYahoo!websearchAPIs,whichare,reportedly,servedfromolderandsmallerindicesthantheindicesusedtoservehumanusers.”Otherresearchersmaybetotallyunawareofthedi erences.Whenwritingabouta2004experimentusingtheGooglesearchengine,Thelwall[47]statedthat“theGoogleAPI...couldhavebeenusedtoautomatetheinitialcollectionoftheresultspages,whichshouldhavegiventhesameoutcome”asusingtheWUI.Themainpurposeofthisstudyistoexaminethedi er-encesbetweenwhatisreportedbetweentheWUIandAPIwhenqueriedwithavarietyofqueriesthatresearchersandusersfrequentlyuse.Our ndingsallowustoaddressthequestionofwhethertheAPIsarepullingfromolderandsmallerindexes.Asecondarypurposeistoexaminehowsearchresultsdecayovertimeandprovidepredictivemod-elsfordeterminingthehalf-livesofsearchresults.

2.BACKGROUNDANDRELATEDWORK2.1

SearchEngineAPIs

Inearly2002,Googlegeneratedagreatdealoffanfarewhenitbecamethe rstsearchenginetoreleaseafreeSOAP-basedAPIforaccessingitsindex.Googlerequireduserstoregisterforalicensekeythatallowed1000requeststobeissuedperday.YahooandMSNreleasedpublicAPIsinearlyandlate2005,respectively,butinane orttoout-doGoogle,theygreatlyreducedtheusagerestrictionsoftheirAPIs.Yahoo’sREST-basedAPIallows5000dailyrequestsperIPaddress,perapplicationID[51].MSN’sSOAPAPIallows10,000dailyrequestsperIPaddress,perapplication

第1页

你可能喜欢

热门文档

相关文档

站点地图 | 文档上传 | 侵权投诉 | 手机版
本站所有资源均来自互联网,本站只负责收集和整理,均不承担任何法律责任,如有侵权等其它行为请联系我们.
文档下载 Copyright 2013 doc.guandang.net All Rights Reserved.  -  email