文档下载 文档下载导航
设为首页 | 加入收藏
搜索 请输入内容:  
 导航当前位置: 文档下载 > 所有分类 > 工程科技 > 电子/电路 > A Content-Based Image Retrieval System Based on Hadoop and Lucene
侵权投诉

A Content-Based Image Retrieval System Based on Hadoop and Lucene

programmer can access a table easily by APIs provided by HBase.

C. Lucene

Lucene is a popular open source search library and often used to build up full-text search engine [14]. Its powerful component of inverted indexing can also be used for constructing a CBIR system.

In Lucene’s indexing, Directory class stands for the directory storing index files; Document class stands for an indexing document, such as a text or a record of a table; Field class stands for different parts of the document and IndexWriter class is responsible for creating index for the document.

III. SYSTEM DESIGN

A. Architecture

A typical architecture of CBIR systems consists of four components of image collection, image feature extraction, indexing and searching. We design the architecture of our CBIR system as Fig. 1 shows referring to it and the architecture of Hadoop.

information such as web URLs, sizes, etc in the column families.

The image feature database is defined as a HBase table named FeatureBase. It uses the absolute path of the image in ImageBase as the row key and stores feature vectors extracted by various algorithms respectively in every column family. The pattern design of the FeatureBase table is showed in TABLE I.

TABLE I. Row Key <AbsolutePath>

THE PATTEN DESIGN OF FEATUREBASE TABLE

Column Family “ColorHistogram”

Column Family “Tamura”

ByteArray

A Content-Based Image Retrieval System Based on Hadoop and Lucene

…

Tamra: ByteArray

The index database is defined as a directory on the HDFS named IndexBase. All index files created by Lucene are stored in it.

C. Image Collection and Extraction

1) Image Collection

Image collection module is responsible to collect images from image sources by crawlers and then store them in ImageBase. Meanwhile, a program is to run to draw basic information of images out and store them in ImageInfoBase. The process is implemented by load balancing method which can make tasks allocated to different servers in the Hadoop cluster and without MapReduce. 2) Feature Extraction

Feature extraction module is responsible to extract

feature vectors from images into ImageBase. The process is implemented by a MapReduce job with

ImageInputFormat, a custom data input format, and

ExtractMap, a custom MapReduce process without Reduce. The process above is showed in Fig. 2.

Figure 1. The architecture of the CBIR system.

The system is mainly divided into four modules: storage, image collection and extraction, indexing and searching. The four modules depend on each other closely and all except searching are based on Hadoop. Next, we’ll simply explain their design.

B. Storage

In the system, the storage module is the data center for other modules. It is not only necessary for a MapReduce Job to run but also includes system’s databases which are the image database, the image information database, the image feature database and the index database.

The image database is defined as a directory on the HDFS named ImageBase, in which all images of the system are stored.

The image information database is defined as a HBase table named ImageInfoBase. It uses the absolute path of the image in ImageBase as the row key and stores images’

Figure 2. The process of the feature extraction.

3) Extraction Algorithms

Every image is as a split inputted into an ExtractMap process to run on a single machine, so extraction algorithms are hard and unnecessary to parallelize.

For expandability of extraction algorithms, we define an interface called ImageFeature for every algorithm to

implement. So far, we have adopted color histogram, color layout, color correlogram, Gabor filter and Tamura texture in the system.

D. Indexing

In the system, we have adopted inverted indexing and Lucene. Specifically, we encapsulate a record of

FeatureBase in a Document whose Fields include the row

上一页第2页

你可能喜欢

热门文档

相关文档

  • lire-lucene image retrieval

    a Java library for content based image retrieval....system – is often relying on fast, global ...image mosaics (see Figure 3) based on LIRe is...

  • 关系型与非关系型数据仓库

    content-based image retrieval, and MapReduce, among...le system (DFS), being Hadoop [4, 5] the ...ciently evaluate analytic queries based on di?...

  • 图像搜索引擎的存储与索引技术研究与实现

    基于Lucene的面向主题搜索... 暂无评价 3页 2.00元 图像搜索引擎案例评价及检...CIRES: A System for Content-based Retrieval in Digital Image Libraries. ...

  • 基于Mahout的推荐系统的分析与设计_朱倩

    content-based and so on, than introduced system ...Apache Mahout起源于2008年, 是Apache Lucene的 子...(IR, Information Retrieval 的知识将文档拆分为词项...

  • Hadoop Overview-Tutorial-20081128

    lucene indexes fetch URL list pages (segments) ...Hadoop for SW Amazon S3 TIFF Image (4TB) PDF..., Information Retrieval, ???, ??? ? Hadoop...

  • 2011#A Real-Time Search Engine with Lucene and S4

    2011#A Real-Time Search Engine with Lucene and ...S4 applied to Information Retrieval 2/5/2011 ...Let’s move it to Hadoop PDF Text Extractor ...

  • 科学文献

    Hadoop集群性能优化技术... 3页 免费 Lucene_3.0...System’, Proceedings of the ACM-SIGMOD Conference...elimination of inconsistencies in data retrieval. ?...

  • MapReduce应用

    Only few retrieval results are needed Not ideal ...A scale-out system, e.g., Nutch/Lucene, ...Hadoop based on SQL [Thusoo, Hive ApacheCon 2008...

  • MapReduce应用

    Only few retrieval results are needed Not ideal ...A scale-out system, e.g., Nutch/Lucene, ...Hadoop based on SQL [Thusoo, Hive ApacheCon 2008...

站点地图 | 文档上传 | 侵权投诉 | 手机版
本站所有资源均来自互联网,本站只负责收集和整理,均不承担任何法律责任,如有侵权等其它行为请联系我们.
文档下载 Copyright 2013 doc.guandang.net All Rights Reserved.  -  email