文档下载 文档下载导航
设为首页 | 加入收藏
搜索 请输入内容:  
 导航当前位置: 文档下载 > 所有分类 > 资格考试/认证 > IT认证 > CLARIT Experiments in Batch Filtering Term Selection and Threshold Optimization in IR and S
侵权投诉

CLARIT Experiments in Batch Filtering Term Selection and Threshold Optimization in IR and S

The Clairvoyance team participated in the Filtering Track, submitting two runs in the Batch Filtering category. While we have been exploring the question of both topic modeling and ensemble filter construction (as

CLARIT Experiments in Batch Filtering:

Term Selection and Threshold Optimization in IR and SVM Filters

David A. Evans, James Shanahan, Norbert Roma, Jeffrey Bennett, Victor Sheftel,

Emilia Stoica, Jesse Montgomery, David A. Hull, Waibhav Tembe

Clairvoyance Corporation, Pittsburgh, PA

1. Introduction

The Clairvoyance team participated in the Filtering Track, submitting two runs in the Batch Filtering

category. While we have been exploring the question of both topic modeling and ensemble filter construction (as in our previous TREC filtering experiments [5]), we had one distinct objective this year, to explore the viability of monolithic filters in classification-like tasks. This is appropriate to our work, in part, because monolithic filters are a crucial starting point for ensemble filtering, and it is possible for them to contribute substantially in the ensemble approach. Our primary goal in

experiments this year, thus, was to explore two issues in monolithic filter construction: (1) term count selection and (2) filter threshold optimization.

In fact, our pre-TREC experiments were conducted in a brief period and we were unable to complete all the tests we had planned. Our official submissions reflect essentially our first, baseline results. They are overall poor in comparison to other results reported this year.

However, an additional focus of our work relates to the general problem of exploiting training data, in particular, where there are only a few positive-example documents for a topic. We regard such cases as more realistic (e.g., in commercial settings) than the categorization-oriented tasks we have seen in TREC filtering in the past, e.g., based on the Reuters collection. Thus, in a series of follow-up experiments, we explored the

strengths and limitations of classifier-based approaches (using kernel methods) and CLARIT-IR-based ones on the fifty TREC-2002 “Assessor” Topics.

In our CLARIT-IR-based experiments, we aimed to establish a more accurate baseline than the one

reflected in our official submissions. We also sought to vary the term-extraction techniques we used, to optimize performance on a topic-by-topic basis.

In our kernel-based (SVM) experiments, we used non-mathematical (non-QP) based approaches to learning SVMs. We used both NLP-based features and simple white-space-delimited ones; and we developed a preliminary approach to thresholding the classifier margin.

In the following sections we first describe our official submitted runs and results and then present in greater detail the post-TREC experiments that we conducted.

2. Official Batch Filtering Runs

Our official batch filtering runs reflected a straight-forward extraction of term vectors from positive training documents, the setting of thresholds based on

calibration of the term vectors over the training data, and the use of the term vectors to score (retrieve/rank) documents in the test collection.

2.1. Preparing Filters and Testing

As a general approach to handling the available training data, we divided the training corpus equally into two parts, one half of which we used for

constructing filters (i.e., extracting terms, assigning weights, and determining the optimal term profile cutoff), the other half of which we used for validation (including score-threshold setting). We constructed monolithic filters for each topic automatically, based on the positive examples of each topic. In fact, we used a slightly modified version of the training database: we added two additional, identical positive example “documents” for each topic. These were created by the system from the topic’s title and its short and long descriptions, with all meta-language (“Retrieve documents which…,” “Find documents that…”) automatically removed. We chose to add these

artificial documents to increase the number of training documents and to emphasize terms that we

anticipated would be especially useful in the filter.

Terms for all topics in this combination of positive examples and artificial documents were generated by CLARIT NLP (yielding morphologically normalized single words, phrases, and sub-phrases), then weighted, ranked, and selected for extraction (to represent the term profile in the filter) using our thesaurus extraction method “Prob2” (as given in Figure 1). We used Prob2 as our default and only term-extraction method based, in part, on our

observations of Prob2’s overall robust performance compared to other term-extraction methods in our TREC 2001 experiments. While keeping the method of selecting terms for topics constant, we

第1页

你可能喜欢

热门文档

相关文档

  • clarity

    clarity_英语考试_外语学习_教育专区。Sentence Clarity and Combining Sentence Clarity Why do we need to be concerned with sentence clari...

  • Clarity 中文版安装指南

    岛津Clarity气相中文工作站 安装指南 安装 软件安装 硬件安装 外部连接 软件配置 联机启动 revision 28. Feb.2006 www.dataapex.com 2 软件安装安装硬件(采集卡)...

  • GC-2014C Clarity培训教材

    GC-2014C Clarity培训教材_能源/化工_工程科技_专业资料。岛津 GC-2014C 气相色谱仪(Clarity 工作站) 培训教材 岛津国际贸易(上海)有限公司 分析中心 岛津 GC-...

  • Clarity 中文色谱工作站操作指导

    Clarity 中文色谱工作站操作指导_定量篇_外标法 本资料以含乙醇的样品为例,介绍了如何使用 Clarity 工作站进行外标法定量。 校准样品: 样品 1:含 0.4mg/g 的...

  • Clarity for GC-2014C培训教程

    启动Clarity 双击桌面快捷图标 以启动Clarity 发送方法对话框 选择“否” 可勾选“设置当前操作 为默认并在以后执行时 不再提示”,该对话框 将不再弹出。 提示...

  • CLARITYII浊度分析仪浊度分析仪浊度分析仪浊度分析仪

    CLARITYII浊度分析仪浊度分析仪浊度分析仪浊度分析仪_交通运输_工程科技_专业资料。CLARITYII浊度分析仪浊度分析仪浊度分析仪浊度分析仪...

  • sentence+clarity+and+combining

    sentence+clarity+and+combining_英语学习_外语学习_教育专区。英语写作Sentence Clarity and Combining 1. Review : avoid comma-splices, sentence commafragment and...

  • clarity 高麦色谱 操作软件使用方法

    Clarity的安装与使用 美国葛麦克仪器公司北京代表处 技术服务中心 一. 工作站的安装 将光盘放进光驱,会自动跳出安装 界面,如无自动跳出,则如下: 双击我的电脑-CD...

  • Structure & Summary of Clarity in Writing

    Structure & Summary of Clarity in Writing_英语学习_外语学习_教育专区 暂无评价

  • 【论文】数据采集卡与Clarity色谱工作站接口程序设计

    介绍了利用Clarity色谱工作站的SDK包,开发US...

站点地图 | 文档上传 | 侵权投诉 | 手机版
本站所有资源均来自互联网,本站只负责收集和整理,均不承担任何法律责任,如有侵权等其它行为请联系我们.
文档下载 Copyright 2013 doc.guandang.net All Rights Reserved.  -  email