With increase in cut-off point value, Google’s precision ratios decreased for all cut-off points. Hakia’s precision ratio slightly increased at cut-off point 10, then decreased slightly at cut-off points 15 and 20. Furthermore, Google retrieved more relevant documents than Msn for all cut-off points with approximately 3% difference. Although Yahoo retrieved approximately 4% more relevant documents than Hakia at cut-off points 10, 15 and 20, it retrieved the highest number of relevant documents at all cut-off points and its best precision ratio was 48% at cut-off point 5. However, Msn retrieved the least number of relevant documents at all cut-off points. Generally, precision ratios of search engines decreased when the cut-off point values were increased.

Figure 2. Mean normalized recall ratios of search
engines
Figure 2 shows mean normalized recall ratios of keyword-based search engines and those of the semantic search engine at various cut-off points. Yahoo’s normalized recall ratio was approximately the same as that of Msn at cut-off point 5. However, for cut-off points 10, 15 and 20, while the normalized recall ratio for either of these search engines increased, the other search engine’s normalized recall ratio decreased. Normalized recall ratios of Google and Msn were approximately the same at cut-off point 15 and the difference between Google, Yahoo and Msn was about 1% at cut-off point 20. Google had the highest performance at cut-off point 5 (88%) but when the cut-off point increased to 10 and 15, Google’s normalized recall ratio decreased. However, Google had the same normalized recall ratio at cut-off points 15 and 20. At all cut-off points, Google had the highest performance for displaying relevant documents retrieved in the top ranks of the retrieval output. Although Hakia’s normalized recall ratio increased gradually at all cut-off points, Hakia had the least performance for displaying relevant documents retrieved in the top ranks of the retrieval output.
4. Conclusion and Future Work
In this paper, an investigative evaluation on search performance of keyword-based and semantic search engines is detailed. It was found that Google, Yahoo and Msn retrieved at least one relevant document for all queries, whereas Hakia3 failed to retrieve any relevant document for one of the queries. Although Hakia4 retrieved more relevant documents compared to other search engines, Hakia3 retrieved least relevant documents.
In terms of overall performance, Hakia retrieved more relevant documents compared to Google and Msn at all cut-off points. However, Yahoo retrieved the highest number of relevant documents at all cut-off points with its best precision ratio being 48% at cut-off point 5. Google showed the highest performance for displaying relevant documents in the top ranks of the retrieval output at all cut-off points. Yahoo and Msn come next, while Hakia showed the least performance for displaying relevant documents retrieved in the top ranks of the retrieval output.
Generally, precision ratios of search engines decreased with increased cut-off point values. However, it was seen from the results that the performance of search engines, when displaying relevant documents in the top ranks, is better than their relevant document retrieval.
Finally, it was seen that semantic search performance of search engines was low regardless of the type of the search engine used. Therefore, search engines need to improve their systems, taking into consideration the importance of the role semantic search can play in helping users getting precise information from the web with minimal effort.
As a future work, the most frequently used queries and phrases could be run on search engines. Furthermore, the number of search engines, queries and phrases could be increased. In addition, elaborate statistical analysis could be provided.
5. References
[1] Google, doc.guandang.net (Accessed on 09 February 2009)
[2] Yahoo, doc.guandang.net (Accessed on 09 February 2009)
[3] Msn, doc.guandang.net (Accessed on 09 February 2009)
[4] Hitwise, “Hitwise US – Leading Search Engines”, October 2008. doc.guandang.net (Accessed on 12 November 2008)
Ball and Brown_1968_An empirical evaluation of accounting income numbers_金融/投资_经管营销_专业资料。 今日推荐 160份文档 四级养生 中医养生与保健 中医...
An empirical evaluation of models of text ...Introduction Modeling the semantic similarity between...ts, improving the performance of search engines, ...
An Empirical Evaluation of Co-ordination Strategies for an AUV and UAV翻译 隐藏>> 对无人驾驶飞行机(无人机)和水下自航行器的一个基于经 验评估的协调...
Empirical analysis on a keyword-based semantic system arXiv:0801.4163v2 [physics.data-an] 9 May 2008 Zi-Ke Zhang a, Linyuan L¨ b , Jian-Guo ...
An empirical evaluation ... 暂无评价 27页 免费...performance of Bod & Kaplan's probability model ..."A Corpus-Based Approach to Semantic Interpretation...
An Empirical Evaluation ... 暂无评价 10页 免费...cation performance, feature set parsimony, and ... local and stochastic search wrappers (Guyon and ...
Web Service Discovery with UDDI Based on Semantic Similarity of Service_...services and will largely enhance search performance of recall and precision....
Empirical study on evaluation of regional agri-food logistics capability in e-commerce environme_专业资料。Dcmbr00Vlme,.2(eil。)eee 1,ou No1Sr 9 29aNo...
Empirical evaluation of confidence and prediction intervals for spatial models of forest structu_专业资料。Junlfoet eerh(0 2()1916oraoFrsyRsac21)22:5-6 ...
Evaluation and Improvement of a SVD-Based Empirical Atmospheric Model_专业资料。An empirical atmospheric model(EAM) based on the singular value decomposition...
热门文档