文档下载 文档下载导航
设为首页 | 加入收藏
搜索 请输入内容:  
 导航当前位置: 文档下载 > 所有分类 > Merging Storyboard Strategies and Automatic Retrieval for Improving Interactive Video Searc
侵权投诉

Merging Storyboard Strategies and Automatic Retrieval for Improving Interactive Video Searc

The Carnegie Mellon University Informedia group has enjoyed consistent success with TRECVID interactive search using traditional storyboard interfaces for shot-based retrieval. For TRECVID 2006 the output of automatic search was included for the first time

The topics are defined by NIST to reflect many of the sorts of queries real users pose, based on query logs against video corpora like the BBC Archives and other empirical data [6]. The topics include requests for specific items or people and general instances of locations and events, reflecting the Panofsky-Shatford mode/facet matrix of specific, generic, and abstract subjects of pictures. In video retrieval, a broadcast is commonly decomposed into numerous shots, with each shot represented by a keyframe: a single bitmap image extracted from that shot. The numerous keyframes can then be subjected to image retrieval strategies. This simplified approach to video retrieval is taken here, with the benefit that many lessons learned for such shot-based video retrieval will be applicable as well for digital still image retrieval. Two shortcomings left for separate investigations are determining how to best represent the contents of a shot with a keyframe or set of keyframes, and how to account for and leverage temporal information present in video, such as rate and direction of motion I

and camera pans and zooms. n the nformedia interfaces discussed here, the temporal progression of information in video is presented through the arrangement of shot thumbnail imagery into storyboards, where shots related in video time within the same video segment are displayed sequentially in time order. n each and every TRECVD prior to 2006, the nformedia storyboard-based interface [2, 3] has either produced the top score for interactive search, or produced high scores statistically clustered with the top-scoring MediaMill interface [8, 9]. It has been an interface with consistently proven effectiveness over the years, but was challenged in TRECVID 2005 with a different type of interface, an “extreme” video retrieval (XVR) interface based on rapid serial visual presentation of keyframes [5]. The XVR interface, the Informedia storyboard interface, and the MediaMill interface collectively produced significantly better interactive search runs than all other submissions in TRECVD 2005. MediaMill and Informedia shared 3 common means of querying the news corpus to produce shot sets: query-by-textual-keyword, query-by-image-example, and query-by-concept [9]. The XVR interface provided a fourth strategy: do no queries, but work only against the output of a fully automatic search result for the topic. For TRECVID 2006, we specifically examined the effects of different interfaces presenting the ranked shot output of automatic retrieval systems, to understand better the reason behind the good performance of extreme video retrieval interface. Is it because XVR is an inherently better interface than the dense storyboards used in the traditional Informedia interface, or is it due to the availability of good ranked shot output from automatic retrieval? In addition, we explored whether there would be a benefit toward including another shot set producing strategy into the Informedia storyboard interface: looking at the automatic search run output, in addition to query-by-text, query-by-image, and query-by-concept.

To facilitate easier comparison across TRECVID participants, we adopt the query access descriptions provided by Snoek and colleagues in their work [8, 9]: “query-by-textual-keyword”, “query-by-image-example”, and “query-by-concept.” To still distinguish the primary modality of the query but to save space, we discuss these access methods as “query-by-text”, “query-by-image”, and “query-by-concept”, respectively, with one addition: the available of the ranked shot output of the fully automatic search, which we label “query-by-best-of-topic” since this is a topic-specific shot list. The six TRECVID Informedia search runs

discussed in the remainder of this paper can be summarized as follows:xAutomatic Baseline: automatic search based only on transcript text

xAutomatic: automatic search encompassing all modalities text, visual, and aural

x

Informedia Full: storyboard interface with 4 means of access: query-by-text, query-by-image, query-by-concept, and query-by-best-of-topic (the latter “query-by-best-of-topic” looks through the ranked shot list produced by the Automatic run)

xInformedia Limited: the same interface as Informedia Full, except that only query-by-best-of-topic is available x

XVR-1: the same data as Informedia Limited, i.e., the output of the Automatic run just as XVR was used with success in TRECVD 2005, but an “extreme” interface manual browsing with resizable pages

x

XVR-2: the same data as Informedia Limited, but an “extreme” interface rapid serial visual presentation.

Given that all of the runs make use of automatic search, a discussion of the state of automatic search is given first, followed by the results from the different interactive search runs.

2.TRECVID AUTOMATIC SEARCH

TheAutomatic search run makes use of relevance-based probabilistic retrieval models to determine weights for combining a large set of semantic concepts when addressing a given topic [1, 4, 10]. This model translated the retrieval task into a supervised learning problem with the parameters learned discriminatively. Rather than treating retrieval as a classification problem, we used an algorithm called “ranking logistic regression” [11] by accounting for the order information between training data, so that the optimization is closely associated with the retrieval performance criteria. We also use a direct query expansion approach to automatically map the multimodal query to existing semantic concepts.

2.1Relevance-Based Retrieval Model

Although we considered a relevance-based probabilistic retrieval model for knowledge combination in our implementation, previous work showed that simply adopting a query-independent knowledge combination strategy is not flexible enough to handle variations in users’ information needs. We developed more advanced methods to incorporate the factor of query information into the probabilistic retrieval model, by making the following assumptions on the query space: 1.

The entire query space can be described by a finite number of mixtures, where the queries from each mixture have the similar characteristics and share the same combination function.

2.

Query descriptions can be used as indicators to which mixture the query (i.e., topic) belongs.

Based on these assumptions, the retrieval model can be represented as:

上一页第2页

你可能喜欢

热门文档

相关文档

  • ...Digital Video Retrieval System

    Merging Storyboard Strategies and Automatic Retrieval for Improving Interactive Video Search. In Proc. Image and Video Retrieval (CIVR) (July 2007). [2] ...

  • ...Digital Video Retrieval System

    Merging Storyboard Strategies and Automatic Retrieval for Improving Interactive Video Search. In Proc. Image and Video Retrieval (CIVR) (July 2007). [2] ...

站点地图 | 文档上传 | 侵权投诉 | 手机版
本站所有资源均来自互联网,本站只负责收集和整理,均不承担任何法律责任,如有侵权等其它行为请联系我们.
文档下载 Copyright 2013 doc.guandang.net All Rights Reserved.  -  email