文档下载 文档下载导航
设为首页 | 加入收藏
搜索 请输入内容:  
 导航当前位置: 文档下载 > 所有分类 > 工程科技 > 电子/电路 > A Content-Based Image Retrieval System Based on Hadoop and Lucene
侵权投诉

A Content-Based Image Retrieval System Based on Hadoop and Lucene

2012 Second International Conference on Cloud and Green Computing

A Content-based Image Retrieval System Based on Hadoop and Lucene

Chunhao GU

Suzhou Branch Office

Oracle (China) Software Systems Co., Ltd.

Suzhou, China guch2010@gmail.com

Abstract—This paper introduces a content-based image retrieval (CBIR) system based on Hadoop and Lucene. Hadoop is a kind of open source software with powerful parallelization and scalability and gradually becomes a popular technique for processing and storage of big data in recent years. We design and implement the system to overcome performance bottlenecks brought by computing complexity and big amount of data when constructing a CBIR system. We will present our ideas, designs and fruits of the system in this paper.

Keywords: CBIR; Hadoop; HBase; Lucene

Yang GAO

Department of Computer Science & Technology

Nanjing University Nanjing, China gaoy@nju.edu.cn

[4], Virage [1], MARS [15], TinEye [10], etc. For all CBIR systems, algorithms for image feature extraction, methods of similarity measure between images and methods of relevance feedback are most basic issues.

1) Feature Extraction Algorithms

Low-level feature extraction algorithms are very important for research of CBIR techniques. These algorithms extract color, texture, shape and space features from an image into feature vectors. Commonly used algorithms for color features are color histogram, color layout, color correlogram, etc [13] and texture features are Tamura texture, Gabor filter, etc [17]. 2) Similarity Measure

The similarity between two images is measured by the distance of feature vectors of the images. Commonly used methods are Euclidean distance, histograms’ cross, Mahalanobis distance, etc. 3) Reference Feedback

Relevance feedback technique is a kind of human-computer interaction method to improve performance of a system. It was introduced firstly in MARS [15]. Commonly used methods are naïve Bayes, SVM, and a method of dynamic adjustment of weights of feature vectors, etc. B. Hadoop

Hadoop consists of two components of HDFS and MapReduce, which are respectively the implement of Google’s GFS and MapReduce [18].

HDFS is a distributed file system for big data storage with outstanding scalability and fault tolerance [16].

MapReduce is a distributed framework for data processing, especially big data [3]. The MapReduce process consists of two steps, Map and Reduce. Splits of data are inputted into the Map process which will output intermediate key-value pairs. And then lists of these key-value pairs each of which has a common key are inputted into the Reduce process to output final key-value pairs. For Hadoop, a programmer needs to customize his MapReduce program and submit it to the Hadoop cluster as a job to run parallelly.

HBase is a kind of NoSQL database software based on Hadoop [5, 7]. A record of a HBase table consists of a sole row key and some column families. Every column family has one or more columns defined by qualifiers. A

I. INTRODUCTION

Recently, with the development of the Internet and multimedia techniques, global digital images are increasing at an unprecedented speed. Therefore, to retrieve and manage these distributed images has become an important research issue.

For the difference between text data and image data, e.g. an image’s texture is hard to describe in a text, traditional text-based image retrieval systems have not be able to satisfy all requests. A lot of companies and organizations have begun to develop content-based image retrieval systems, known as CBIR systems.

Hadoop is a kind of open source software under the Apache Foundation [6, 18]. For its powerful parallelization and scalability, Hadoop has gradually become a popular technique for processing and storage of big data. Yahoo!, eBay, Facebook and Baidu have applied Hadoop in many products, such as data mining, searching, recommendation, etc.

For a long time, high computation tasks caused by computing complexity and big amount of data during storing, indexing, etc have been bottlenecks for constructing a CBIR system. Therefore, we study, design and implement a system based on Hadoop to explore the solution to the problem.

This paper is organized as follows: we discuss previous related works in Section II; we introduce our design of the system’s architecture and modules in Section III; in Section IV, we show the demo of the system’s prototype.

II.

PREVIOUS RELATED WORKS

A. CBIR

CBIR technique has begun from 1990s [2]. After that, there appeared a lot of famous CBIR systems, such as QBIC

第1页

你可能喜欢

热门文档

相关文档

  • lire-lucene image retrieval

    a Java library for content based image retrieval....system – is often relying on fast, global ...image mosaics (see Figure 3) based on LIRe is...

  • 关系型与非关系型数据仓库

    content-based image retrieval, and MapReduce, among...le system (DFS), being Hadoop [4, 5] the ...ciently evaluate analytic queries based on di?...

  • 图像搜索引擎的存储与索引技术研究与实现

    基于Lucene的面向主题搜索... 暂无评价 3页 2.00元 图像搜索引擎案例评价及检...CIRES: A System for Content-based Retrieval in Digital Image Libraries. ...

  • 基于Mahout的推荐系统的分析与设计_朱倩

    content-based and so on, than introduced system ...Apache Mahout起源于2008年, 是Apache Lucene的 子...(IR, Information Retrieval 的知识将文档拆分为词项...

  • Hadoop Overview-Tutorial-20081128

    lucene indexes fetch URL list pages (segments) ...Hadoop for SW Amazon S3 TIFF Image (4TB) PDF..., Information Retrieval, ???, ??? ? Hadoop...

  • 2011#A Real-Time Search Engine with Lucene and S4

    2011#A Real-Time Search Engine with Lucene and ...S4 applied to Information Retrieval 2/5/2011 ...Let’s move it to Hadoop PDF Text Extractor ...

  • 科学文献

    Hadoop集群性能优化技术... 3页 免费 Lucene_3.0...System’, Proceedings of the ACM-SIGMOD Conference...elimination of inconsistencies in data retrieval. ?...

  • MapReduce应用

    Only few retrieval results are needed Not ideal ...A scale-out system, e.g., Nutch/Lucene, ...Hadoop based on SQL [Thusoo, Hive ApacheCon 2008...

  • MapReduce应用

    Only few retrieval results are needed Not ideal ...A scale-out system, e.g., Nutch/Lucene, ...Hadoop based on SQL [Thusoo, Hive ApacheCon 2008...

站点地图 | 文档上传 | 侵权投诉 | 手机版
本站所有资源均来自互联网,本站只负责收集和整理,均不承担任何法律责任,如有侵权等其它行为请联系我们.
文档下载 Copyright 2013 doc.guandang.net All Rights Reserved.  -  email