Golden Retriever-A Java Bad Open Source Image
Retrieval Engine
Lazaros T.Tsochatzidis,Chryssanthi Iakovidou,Savvas A.Chatzichristofis and
Yiannis S.Boutalis
Democtitus University of Thrace
Department of Electrical and Computer Engineering
Xanthi,Greece
{lazatsoc,ciakovid,schatzic,ybout}@
——
SUBMITTED TO ACM MULTIMEDIA2013OPEN SOURCE SOFTWARE COMPETITION
ABSTRACT
Golden Retriever Image Retrieval Engine(GRire)is an open source light weight Java library developed for Content Bad Image Retrieval(CBIR)tasks,employing the Bag of Visual Words(BOVW)model.It provides a complete framework for creating CBIR system including image analysis tools, classifiers,weighting schemes etc.,for efficient indexing and retrieval procedures.Its eminent feature is its extensibility, achieved through the open source nature of the library as well as a ur-friendly embedded plug-in system.
GRire is available on-line along with install and develop-ment documentation ire and on its Google Code le.com/p/grire.It is distributed either as a Java library or as a standalone Java application,both GPL licend.
Categories and Subject Descriptors
H.3.3[Information Storage and Retrieval]:Information Search and Retrieval
Keywords
Image Retrieval,Visual Words,Bag-of-Visual-Words,Open Source,Image Search,Image Indexing
1.INTRODUCTION
Content bad image retrieval approaches can be mainly classified into two groups bad on the types of low level visual features that they employ.Thefirst group consist of methods that u global features while the cond one is formed by approaches that employ local features.Addition-ally,CBIR systems can be parated into two general class bad on the type of the queries they handle.In applied rearch,CBIR systems are retrieving images with similar visual characteristics to the query if the query Permission to make digital or hard copies of all or part of this work for personal or classroom u is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on thefirst page.Copyrights for components of this work owned by others than ACM must be honored.Abstracting with credit is permitted.To copy otherwi,or republish,to post on rvers or to redistribute to lists,requires prior specific permission and/or a fee.Request permissions from permissions@acm.
MM’13,October21–25,2013,Barcelona,Spain.
Copyright2013ACM978-1-4503-2404-5/$15.00.is a red image,the retrieved images are also red)[4].To sup-port the queries,mainly global features are employed.At the same time,many advanced systems are able to recognize the context of the query and to retrieve mantically similar
if the query is a red luxurious car,the retrieved images are depicting a Ferrari).In this ca,the queries are mainly supported by local features.
Recent CBIR approaches have strongly focud on the combination of global and local features.The approaches are employing the local features in order to produce a holis-tic,global reprentation of the image.An example of such approaches is the bag-of-visual-words(BOVW)approach. This BOVW reprentation is analogous to the bag-of-words reprentation,a well-known and widely ud method in text retrieval,where a document is reprented by a t of dis-tinct keywords.
BOVW methods are fast becoming a widely ud repre-ntation for CBIR,mainly for three reasons:their better re-trieval effectiveness over global feature reprentations,the much better efficiency than local feature reprentations and the fact that they can be ud both,by systems that attempt to identify the content of the query to retrieve images with mantically similar content and by systems designed to re-trieve images with similar visual concept to the
query.
天使的翅膀电影Figure1:Golden Retriever Logo.
This paper prents a new open source and extensible Java image retrieval library named Golder Retriever.The scope of the library is to provide solutions on how to integrate CBIR techniques in an easy way,on a wide range of appli-cations.Additionally,GRire provides a testing platform for rearchers that could be ud to evaluate existing or new CBIR approaches.
Current version of the library is mainly focud on the BOVW approach,integrating a variety of local features, classifiers as well as methods from thefield of information retrieval adjusted to meet the ima
ge retrieval perspective.
2.THE GRIRE PROJECT
The main objective of the project is to help developers create and evaluate their methods in any image databa with minimum effort and without needing to concern about the details of the model itlf.Furthermore,developers are welcome to integrate custom components,such as feature extractors and descriptors,into the GRire library.Thus,a whole BOVW system can be created and tested adopting the provided implemented weighting schemes and similarity models.Towards this direction,GRire combines an adapt-able and easy-to-u plug-in system together with a powerful and efficient indexing and retrieving mechanism.
2.1Project Structure
The GRire project can be operated in two ways;either as a Java library to be integrated into other applications or through an easily ud graphical ur interface called GRireFX.The two versions are distributed in the same file available for download,under the ire irefx respectively and they both provide the same functionality and extensibility.Esntially,the GRire library consists of two main parts:the core and the compo-nents.
2.2The Core
The core of GRire implements the main workflow for the indexing and the retrieval procedures using a BOVW archi-tecture.It consists of the class required by the model and it is considered to be the‘fixed’part of the library.The class of the core are organized into two different types hereby referred to as Structures and Functions.Structures are objects responsible for handling databa’s operations such as storing and fetching.On the other hand,Func-tions implement the procedures for generating the model, like indexing and retrieving.Furthermore,Structures are usually an‘extra layer’above the GeneralStorer which pro-vides more basic structures described in ctions2.3and3.2.
2.2.1Structures
ImagePool An image pool is a collection of images.It pairs a unique id with the absolute path of an image and stores it in a databa.It provides all the required methods for adding and deleting an image from the pool.
PoolFeatures This structure is the collection of features extracted from an ImagePool.
暴利行业Codebook The main component of the BOVW model.It is
a lexicon of key-features lected from the PoolFeatures
structure using various ways(clustering etc.).
Index This structure stores the reprentation(descriptor) of each image.
2.2.2Functions
Importer This is a simple class that allows batch importing of image from multiple folders. PoolFeatureExtractor It extracts features from all the images part of an ImagePool to generate a PoolFea-ture structure for storing them.The features are ex-tracted using a FeatureExctractor component(kindly refer to ction2.3).ClusteringCodebookFactory This class creates a new lex-icon(codebook)by performing a ClusteringAlgorithm
on the features provided.By default,the Clustering-
CodebookFactory will create clusters using the30%of
the data but urs have the ability to modify the per-
centage.The training t is stored temporary in the
file system.
IndexFactory This function produces the index of the im-age reprentations(descriptors)using the PoolFea-
tures extracted earlier.The reprentation for each
image is created according to the supplied VisualWord-
Descriptor component(details in ction2.3). QueryPerformer This class performs the arching proce-dure for one or more queries,given as input,generating
a TREC formattedfile with the results for easy and
accurate evaluation.
2.3The Components
The BOVW model can be asmbled by veral different components that execute discrete steps of the architecture, thus making it highly customizable.The multiple possible component combinations p
rovide a great variety of exper-imental tups of the model.Each component allows for independent enhancement attracting rearchers/developers from varying scientificfields to explore and extend.
Type Component Name
GeneralStorer MapDB,See ction3.2
FeatureExtractor
SURF[3]
SIFT[6]
ORB[8]
MSER[7]+FREAK[1]
CEDD[5](Bag of CEDDs1) ClusteringAlgorithm
K-Means
SGONG[2]
SimilarityMeasure
Euclidean Distance
Cosine Similarity
WeightingScheme SMART[9]
Stemmer Euclidean Stemmer
Table1:A list of the currently implemented Com-ponents
The components are interfaces who implementations are given as arguments to the core class.A list of the components is shown below while Table1briefly prents the implemented components:
1.GeneralStorer.It manages the way data are written
in thefile system,providing simple structures such as
maps and sorted maps.
2.FeatureExtractor.It is employed from the core for
the extraction of local features from an image.
1The propod library has the ability to adjust the informa-tion and the advantages derived from the Global low level Features(GF),into the BOVW model.GRire parates the images into a pret number of image blocks and calculates a global CEDD)from each one.Bad on this approach,every image is reprented with multiple GFs. In the quel,the global features are considered as visual words.
3.ClusteringAlgorithm.This component implements a
clustering algorithm ud for the creation of the code-book.
4.SimilarityMeasure.It is ud for the comparison of
the reprentation(descriptor).
5.VisualWordDescriptor.This component’s task is to
form the reprentation(descriptor)of an image using all the previously created structures.
6.WeightingScheme.The weighting scheme ud during
the retrieval process defines how thefinal reprenta-tion(descriptor)of an image will be formed,just before it is compared with other reprentations.
7.Stemmer.Stemmers are the objects that define how a
visual word derived from an image will be assigned to
a word from the codebook.
2.4Work-flow
Initially,the images are imported into an ImagePool ob-ject.Then,the PoolFeatureExtractor extracts the local features from the imported images,using a FeatureExtrac-tor component and stores them in a PoolFeatures struc-ture.In the quel,the IndexFactory creates the repre-ntation(descriptor)for each image using a VisualWord-Descriptor component,according to a Codebook structure. The results are stored it in the Index.The codebook was created by the CluteringCodebookFactory class in advance, using a ClusteringAlgorithm component on PoolFeatures. The system is now ready to perform any retrieval task us-ing the QueryPerformer together with the WeightingScheme and Simila
rityMeasure components.
3.TECHNICAL DETAILS
3.1Extending GRire
GRire has been designed with particular attention to be highly customizable giving the opportunity to developers/ rearchers to implement their own methods and integrate them in the framework.This is achieved through the plug-in system which adopts the open source jspf2.Developers can easily create a plug-in without any constraints on the number of the needed parameters so as to implement one of the components described earlier.For the urs’conve-nience a t of common parameters has been predefined to be recognizable by the framework.The plug-in interacts with GRire notifying it about the number and the type of the parameters and the program dynamically prompts the ur for the right input method.Every plug-in has the following functionality:
•It implements only one component of GRire.
•It may require input parameters.
•It may require a tup process manually called by the ur before it can be executed.
巴山夜雨涨秋池打一字•Additional parameters for the aforementioned tup process may be also required.
•Default initialization values may be available for all or some of the parameters.
The GRire library employs the very powerful and promis-ing MapDB3that supports fast and efficient storage and -rialization of the data.This embedded databa provides maps(Hash,Sorted and Multi-maps)as basic structures that are ud to form all other more complex needed struc-tures(Codebook etc.).MapDB offers its own rialization, compression and cache memory system while it is portable and allows processing of large amount of data,with mini-mum overhead in memory.
3.3Compiling and Installation
It is decided that the components of GRire will be dis-tributed parately from its core,grouped in plug-in packs. Every component may have its own dependencies,which may be external libraries or even programs.The git reposi-tory of the project includes the source code of the core along with the source code of every component pack that will be available as plug-in.A universal ant configurationfile ac-companies every module(core and packs)as well as every plug-in pack.竹类植物
After obtaining the core jarfile,the ur can execute it to launch the GRireFX or import it as an external library to another Java application.The components can be ud integrated with the following ways:
•If GRire operates as a Java library,then the plug-ins can be also imported to the application to be ud like any other class.
•If GRire operates through its GUI then a textfile named‘plugins’is required to be prent in the same folder as the jarfile.This textfile contains the paths to the jarfiles of the plug-in packs.
4.GRIREFX
For urs that do not wish to create their own applica-tion and just want to test their component of the BOVW model,the ideal solution is the graphical ur interface of GRire.It is designed not to inrt any limitation compared to the GRire library version,retaining its customizability by allowing enrichments in the form plug-in.Infigure2there is a screen shot of GRireFX while the ur is tting up a multi-query retrieval task.The results of the procedure is a TREC formatedfile.
As mentioned before,GRireFX automatically recognizes the number and the types of the parameters,
as well as their default values(if they exist),and dynamically requests from the ur the corresponding input.Figure3illustrates the dynamic tree of parameters that grows according to every component’s needs while the ur makes interacts with the interface.
借景抒情的诗句
5.PERFORMANCE
Indicative experiments have been performed to evaluate the performance of the GRire library.It is worth noting that,the propod project is a framework for implement-ing and integrating various features with different compu-tational costs and complexity,so the time needed for the indexing as well as for the retrieval procedure greatly varies and depends on the components that will be ud(features, /jankotek/MapDB
Figure 2:GRireFX performing a multi-query exper-
iment.
Figure 3:GRireFX dynamically adds leaves to the tree of parameters.
招商classifiers,codebook size,weighting scheme,databa size,etc).Table 2prents the time (in minutes)elapd for two different experiments that have been performed.
SURF SIFT Feature Extraction
109163Codebook Generation (K-Means)73158Descriptor Calculation
2026Retrieval Procedure (100queries)
<2
<2
Table 2:Results of the time needed for each process measured in minutes.
For the first experiment,the SURF(64)features were ud as the local features,while for the cond one,the SIFT(128)features were employed.In both cas,experiments con-ducted on the UK-BENCH [7]image datat (10200images),using a codebook of 1024visual words.For the retrieval pro-cedure,we executed a multi-query experiment (100queries linear arch)using the tf-idf weighting scheme and cosine similarity measure.Experiments carried out on a Core 2Quad Q6600CPU with 4GB of RAM system.The storage needs for the created databas reached a total of 2.89GB and 5.48GB respectively.
6.CONCLUSIONS
This paper prents a new framework for CBIR bad on the BOVW approach,that can be ud in a wide range of applications.The open source nature of the propod
library allows and encourages rearches to extend GRire by implementing more components,methods and features.It’s worth noting that apart from a powerful and fast in-dexing and retrieving mechanism,GRire additionally us an extremely easy to u plug-in system.The GRire li-brary is provided under GPL licen and is available on-line along with install and development documentation ire and on its Google Code le.com/p/grire
7.ACKNOWLEDGMENTS
This rearch has been co-financed by the European Union (European Social Fund-ESF)and Greek national funds through the Operational Program“Education and Lifelong Learning”of the National Strategic Reference Framework (NSRF)-Re-arch Funding Program:Heracleitus II.Investing in knowl-edge society through the European Social Fund.
8.REFERENCES
[1]Alexandre Alahi,Raphael Ortiz,and Pierre
Vandergheynst.Freak:Fast retina keypoint.In CVPR ,pages 510–517,2012.
[2]Antonios Atsalakis and Nikos Papamarkos.Color
reduction and estimation of the number of dominant colors by using a lf-growing and lf-organized neural gas.Eng.Appl.of AI ,19(7):769–786,2006.
[3]Herbert Bay,Andreas Ess,Tinne Tuytelaars,and Luc
J.Van Gool.Speeded-up robust features (surf).Computer Vision and Image Understanding ,110(3):346–359,2008.
[4]S.A.Chatzichristofis,C.Iakovidou,Y.Boutalis,and
O.Marques.Co.vi.wo.:Color visual words bad on non-predefined size codebooks.Cybernetics,IEEE Transactions on ,43(1):192–205,2013.
[5]Savvas A.Chatzichristofis and Yiannis S.Boutalis.三国演义火烧赤壁
Cedd:Color and edge directivity descriptor:A
compact descriptor for image indexing and retrieval.In ICVS ,pages 312–322,2008.
[6]David G.Lowe.Distinctive image features from
scale-invariant keypoints.International Journal of Computer Vision ,60(2):91–110,2004.
重庆招办
[7]D.Nister and H.Stewenius.Scalable recognition with a
vocabulary tree.In CVPR ,volume 5,pages 2161–2168,2006.
[8]Ethan Rublee,Vincent Rabaud,Kurt Konolige,and
Gary Bradski.Orb:an efficient alternative to sift or surf.In Computer Vision (ICCV),2011IEEE
International Conference on ,pages 2564–2571.IEEE,2011.
[9]G.Salton.The SMART Retrieval System -Experiments
in Automatic Document Processing .Prentice-Hall,Inc.,Upper Saddle River,NJ,USA,1971.
