文档下载 文档下载导航
设为首页 | 加入收藏
搜索 请输入内容:  
 导航当前位置: 文档下载 > 所有分类 > 工程科技 > 电子/电路 > A Content-Based Image Retrieval System Based on Hadoop and Lucene
侵权投诉

A Content-Based Image Retrieval System Based on Hadoop and Lucene

key of the record and the columns of the record without tokenizing. And then an IndexWriter will make inverted index for the Document.

Indexing is a process that is done by a MapReduce job. The job’s inputs should be key-value pairs each of which consists of a key that is the row key of a record of

FeatureBase and a value that is a list of column families of the record. The IdentityTableMap class of HBase library can convert records of a HBase table into this kind of key-pairs. So we use it directly and don’t customize Map process. And then, we implement a custom Reduce process, IndexTableReduce, and a custom output format, IndexOutputFormat, to accomplish the creation of inverted indexing as the previous paragraph said. The process is showed in Fig. 3.

The middle layer includes two servlets respectively

called SearchServlet which is responsible for searching and RelFeedbackServlet which is responsible for relevance feedback.

The bottom layer is the intermediate between the middle layer and IndexBase. It includes common classes that have implemented Euclidean distance for similarity measure when searching and a method of dynamic adjustment of weights of feature vectors for relevance feedback. This kind of relevance feedback is a re-searching based on the original searching results, users’ marks and a scoring rule, see [15].

IV. SYSTEM DEMO

For the demo, we established a Hadoop cluster with 4 nodes and ImageBase with over 1000 images. It took the system less than 10 minutes to update FeatureBase and IndexBase.

After the system was ready to serve for users, we landed onto the main page and used file selector to select a querying image about pandas.

After the Search button pressed, the system returned the results instantly as Fig 5.showed. The results were sorted by the descending similarities calculated by the system. But it seemed not so satisfied for us, so we marked all images about pandas and pressed the FeedBack button.

Figure 3. The process of the indexing.

For the parallelization of the indexing, we have adopted incremental indexing of Lucene. For all index files are stored in IndexBase on the HDFS, we have designed HDFSDirectory class which extends from Lucene

Directory and encapsulates HDFS. So it is easy for other modules to access IndexBase.

E. Searching

In Hadoop, MapReduce jobs must be firstly submitted in a waiting queue for running. If searching is done by a

MapReduce job, the instantaneity is preponderant when the number of users visiting the system at the same time is small but unsatisfied when the number is large. So we design the searching module uncoupled with Hadoop and use load balancing method which can make tasks allocated to different servers in the Hadoop cluster.

The design of the searching module is based on B/S architecture and JSP/Servlet technique. It consists of three layers. The top layer is the user interface implemented by HTML and JSP. The compact main page as Fig. 4 shows consists of a textbox, a file selector, a Search button and a logo.

Figure 5. The results of searching.

After the FeedBack button pressed, the system returned the results instantly as Fig. 6 showed. Obviously, all images about pandas ranked top.

Figure 6. The results of relevance feedbacking.

Figure 4. The design of the main page. More experiments showed the system could always rank first the image in ImageBase whose low-level features were

上一页第3页

你可能喜欢

热门文档

相关文档

  • lire-lucene image retrieval

    a Java library for content based image retrieval....system – is often relying on fast, global ...image mosaics (see Figure 3) based on LIRe is...

  • 关系型与非关系型数据仓库

    content-based image retrieval, and MapReduce, among...le system (DFS), being Hadoop [4, 5] the ...ciently evaluate analytic queries based on di?...

  • 图像搜索引擎的存储与索引技术研究与实现

    基于Lucene的面向主题搜索... 暂无评价 3页 2.00元 图像搜索引擎案例评价及检...CIRES: A System for Content-based Retrieval in Digital Image Libraries. ...

  • 基于Mahout的推荐系统的分析与设计_朱倩

    content-based and so on, than introduced system ...Apache Mahout起源于2008年, 是Apache Lucene的 子...(IR, Information Retrieval 的知识将文档拆分为词项...

  • Hadoop Overview-Tutorial-20081128

    lucene indexes fetch URL list pages (segments) ...Hadoop for SW Amazon S3 TIFF Image (4TB) PDF..., Information Retrieval, ???, ??? ? Hadoop...

  • 2011#A Real-Time Search Engine with Lucene and S4

    2011#A Real-Time Search Engine with Lucene and ...S4 applied to Information Retrieval 2/5/2011 ...Let’s move it to Hadoop PDF Text Extractor ...

  • 科学文献

    Hadoop集群性能优化技术... 3页 免费 Lucene_3.0...System’, Proceedings of the ACM-SIGMOD Conference...elimination of inconsistencies in data retrieval. ?...

  • MapReduce应用

    Only few retrieval results are needed Not ideal ...A scale-out system, e.g., Nutch/Lucene, ...Hadoop based on SQL [Thusoo, Hive ApacheCon 2008...

  • MapReduce应用

    Only few retrieval results are needed Not ideal ...A scale-out system, e.g., Nutch/Lucene, ...Hadoop based on SQL [Thusoo, Hive ApacheCon 2008...

站点地图 | 文档上传 | 侵权投诉 | 手机版
本站所有资源均来自互联网,本站只负责收集和整理,均不承担任何法律责任,如有侵权等其它行为请联系我们.
文档下载 Copyright 2013 doc.guandang.net All Rights Reserved.  -  email