programmer can access a table easily by APIs provided by HBase.
C. Lucene
Lucene is a popular open source search library and often used to build up full-text search engine [14]. Its powerful component of inverted indexing can also be used for constructing a CBIR system.
In Lucene’s indexing, Directory class stands for the directory storing index files; Document class stands for an indexing document, such as a text or a record of a table; Field class stands for different parts of the document and IndexWriter class is responsible for creating index for the document.
III. SYSTEM DESIGN
A. Architecture
A typical architecture of CBIR systems consists of four components of image collection, image feature extraction, indexing and searching. We design the architecture of our CBIR system as Fig. 1 shows referring to it and the architecture of Hadoop.
information such as web URLs, sizes, etc in the column families.
The image feature database is defined as a HBase table named FeatureBase. It uses the absolute path of the image in ImageBase as the row key and stores feature vectors extracted by various algorithms respectively in every column family. The pattern design of the FeatureBase table is showed in TABLE I.
TABLE I. Row Key <AbsolutePath>
THE PATTEN DESIGN OF FEATUREBASE TABLE
Column Family “ColorHistogram”
Column Family “Tamura”
ByteArray

…
Tamra: ByteArray
The index database is defined as a directory on the HDFS named IndexBase. All index files created by Lucene are stored in it.
C. Image Collection and Extraction
1) Image Collection
Image collection module is responsible to collect images from image sources by crawlers and then store them in ImageBase. Meanwhile, a program is to run to draw basic information of images out and store them in ImageInfoBase. The process is implemented by load balancing method which can make tasks allocated to different servers in the Hadoop cluster and without MapReduce. 2) Feature Extraction
Feature extraction module is responsible to extract
feature vectors from images into ImageBase. The process is implemented by a MapReduce job with
ImageInputFormat, a custom data input format, and
ExtractMap, a custom MapReduce process without Reduce. The process above is showed in Fig. 2.
Figure 1. The architecture of the CBIR system.
The system is mainly divided into four modules: storage, image collection and extraction, indexing and searching. The four modules depend on each other closely and all except searching are based on Hadoop. Next, we’ll simply explain their design.
B. Storage
In the system, the storage module is the data center for other modules. It is not only necessary for a MapReduce Job to run but also includes system’s databases which are the image database, the image information database, the image feature database and the index database.
The image database is defined as a directory on the HDFS named ImageBase, in which all images of the system are stored.
The image information database is defined as a HBase table named ImageInfoBase. It uses the absolute path of the image in ImageBase as the row key and stores images’
Figure 2. The process of the feature extraction.
3) Extraction Algorithms
Every image is as a split inputted into an ExtractMap process to run on a single machine, so extraction algorithms are hard and unnecessary to parallelize.
For expandability of extraction algorithms, we define an interface called ImageFeature for every algorithm to
implement. So far, we have adopted color histogram, color layout, color correlogram, Gabor filter and Tamura texture in the system.
D. Indexing
In the system, we have adopted inverted indexing and Lucene. Specifically, we encapsulate a record of
FeatureBase in a Document whose Fields include the row
a Java library for content based image retrieval....system – is often relying on fast, global ...image mosaics (see Figure 3) based on LIRe is...
content-based image retrieval, and MapReduce, among...le system (DFS), being Hadoop [4, 5] the ...ciently evaluate analytic queries based on di?...
基于Lucene的面向主题搜索... 暂无评价 3页 2.00元 图像搜索引擎案例评价及检...CIRES: A System for Content-based Retrieval in Digital Image Libraries. ...
content-based and so on, than introduced system ...Apache Mahout起源于2008年, 是Apache Lucene的 子...(IR, Information Retrieval 的知识将文档拆分为词项...
lucene indexes fetch URL list pages (segments) ...Hadoop for SW Amazon S3 TIFF Image (4TB) PDF..., Information Retrieval, ???, ??? ? Hadoop...
2011#A Real-Time Search Engine with Lucene and ...S4 applied to Information Retrieval 2/5/2011 ...Let’s move it to Hadoop PDF Text Extractor ...
Hadoop集群性能优化技术... 3页 免费 Lucene_3.0...System’, Proceedings of the ACM-SIGMOD Conference...elimination of inconsistencies in data retrieval. ?...
Only few retrieval results are needed Not ideal ...A scale-out system, e.g., Nutch/Lucene, ...Hadoop based on SQL [Thusoo, Hive ApacheCon 2008...
Only few retrieval results are needed Not ideal ...A scale-out system, e.g., Nutch/Lucene, ...Hadoop based on SQL [Thusoo, Hive ApacheCon 2008...
热门文档