文档下载 文档下载导航
设为首页 | 加入收藏
搜索 请输入内容:  
 导航当前位置: 文档下载 > 所有分类 > IT/计算机 > 互联网 > An Empirical Evaluation on Semantic Search Performance of Keyword-Based and Semantic Search Engines
侵权投诉

An Empirical Evaluation on Semantic Search Performance of Keyword-Based and Semantic Search Engines

2009 Fourth International Conference on Internet Monitoring and Protection

An Empirical Evaluation on Semantic Search Performance of Keyword-Based

and Semantic Search Engines: Google, Yahoo, Msn and Hakia

Duygu Tümer1, Mohammad Ahmed Shah2, Y ltan Bitirim1

1

Department of Computer Engineering, Eastern Mediterranean University,

Famagusta / T.R.N.C.

027860@students.emu.edu.tr, yiltan.bitirim@emu.edu.tr

2

Department of Computer Engineering, Middle East Technical University

Northern Cyprus Campus, Güzelyurt / T.R.N.C.

shah@metu.edu.tr

Abstract

This paper investigates the semantic search performance of search engines. Initially, three keyword-based search engines (Google, Yahoo and Msn) and a semantic search engine (Hakia) were selected. Then, ten queries, from various topics, and four phrases, having different syntax but similar meanings, were determined. After each query was run on each search engine; and each phrase containing a query was run on the semantic search engine, the first twenty documents on each retrieval output was classified as being “relevant” or “non-relevant”. Afterwards, precision and normalized recall ratios were calculated at various cut-off points to evaluate keyword-based search engines and the semantic search engine. Overall, Yahoo showed the best performance in terms of precision ratio, whereas Google turned-out to be the best search engine in terms of normalized recall ratio. However, it was found that semantic search performance of search engines was low for both keyword-based search engines and the semantic search engine.

Keywords: Information retrieval, Semantic search performance, Semantic search engine, Keyword-based search engine, Evaluation.

1. Introduction

Web search engine is a computer program that allows users to search and retrieve web documents with queries for their information needs. The most popular search engines are Google [1], Yahoo [2] and Msn [3] with 71.9%, 17.7% and 4.2% volume of search ratios (based on US Internet usage), respectively [4]. Google, Yahoo and Msn are keyword-based search engines. However, semantic

search engines are an alternative to these search engines. The difference of semantic search engines from conventional search engines is that the semantic search engines are meaning-based. Hakia [5] is one of the emerging publicly available semantic search engines on the web [6].

Researchers have conducted several studies to evaluate retrieval effectiveness of keyword-based search engines and some have proposed search engines, methods and technologies for semantic search. Atsaros et al. compared the performance of keyword-based search engines (site-specific and general purpose) in terms of precision and relative recall [7]. Kasneci et al. proposed NAGA semantic search engine which uses knowledge base, introducing a graph-based query language and a novel scoring mechanism [8]. However, NAGA is not a publicly available search engine. Lee et al. proposed a semantic web services search system to provide better retrieval effectiveness using clustering and ontology approaches [9].

Although the research trend on semantic search engines is increasing, publicly available search engines on the web are, generally, keyword-based search engines. Keyword-based search engines, such as Google and Yahoo, have a large user-base [4]. Therefore, evaluation of semantic search performance of popular keyword-based and semantic search engines is a valuable work intended to motivate researchers and search engine providers to improve current systems further.

Perfect search engine model might be the one that always finds the precise document(s) on the web for the user. The result of a perfect search engine would, ideally, satisfy the expectations of its users, whenever a query is searched. The inspiration, for this study, is to motivate researchers and search engine providers towards reaching this perfect search engine model.

第1页

你可能喜欢

热门文档

相关文档

站点地图 | 文档上传 | 侵权投诉 | 手机版
本站所有资源均来自互联网,本站只负责收集和整理,均不承担任何法律责任,如有侵权等其它行为请联系我们.
文档下载 Copyright 2013 doc.guandang.net All Rights Reserved.  -  email