A large amount of information on the Web is contained in regularly structured objects, which we call data records. Such data records are important because they often present the essential information of their host pages, e.g., lists of products and service
Mining Data Records in Web Pages
Bing Liu
Department of Computer Science University of Illinois at Chicago
851 S. Morgan Street Chicago, IL 60607-7053
Robert Grossman
Dept. of Mathematics, Statistics, and
Computer Science
University of Illinois at Chicago 851 S. Morgan Street, IL 60607
Yanhong Zhai
Department of Computer Science University of Illinois at Chicago
851 S. Morgan Street Chicago, IL 60607-7053
liub@cs.uic.edu
grossman@uic.edu
yzhai@cs.uic.edu
ABSTRACT
A large amount of information on the Web is contained in regularly structured objects, which we call data records. Such data records are important because they often present the essential information of their host pages, e.g., lists of products and services. It is useful to mine such data records in order to extract information from them to provide value-added services. Existing approaches to solving this problem mainly include the manual approach, supervised learning, and automatic techniques. The manual method is not scalable to a large number of pages. Supervised learning needs manually prepared positive and negative training data and thus also require substantial human effort. Current automatic techniques are still unsatisfactory because of their poor performances. In this paper, we propose a much more effective automatic technique to perform the task. This technique is based on two important observations about data records on the Web and a string matching algorithm. The proposed technique is able to mine both contiguous and non-contiguous data records. By non-contiguous data records, we mean that two or more data records intertwine in terms of their HTML codes. When they are displayed on a browser, each of them appears contiguous. Existing techniques are unable to mine such records. Our experimental results show that the proposed technique outperforms existing techniques substantially.
Categories and Subject Descriptors
I.5 [Pattern Recognition]: statistical and structural. H.2.8 [Database Applications]: data mining
Keywords
Web data records, Web mining
1. INTRODUCTION
A large amount of information on the Web is presented in regularly structured objects. A list of such objects in a Web page often describes a list of similar items, e.g., a list of products or services. They can be regarded as database records displayed in Web pages with regular patterns. In this paper, we also call them data records. Mining data records in Web pages is useful because it allows us to extract and integrate information from multiple
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Conference ’00, Month 1-2, 2000, City, State.
Copyright 2000 ACM 1-58113-000-0/00/0000…$5.00.
sources to provide value-added service, e.g., customizable Web information gathering, comparative-shopping, meta-search, etc. Figure 1 gives an example, which is a segment of a Web page that lists two Apple notebooks. The full description of each notebook is a data record. The objective of the proposed technique is to

automatically mine all the data records in a given Web page.
Figure 1. An example: two data records
Several approaches have been reported in the literature for mining data records (or their boundaries) from Web pages. The first approach is the manual approach. By observing a Web page and its source code, the programmer can find some patterns and then writes a program to identify each data record. This approach is not scalable to a large number of pages. Other approaches [2][4] [6][7][8][9][10][12][14][16][17][18][19][21][22][23] all have some degree of automation. They rely on some specific HTML tags and/or machine learning techniques to separate objects. These methods either require prior syntactic knowledge or human labeling of specific regions in the Web page to mark them as interesting. [10] presents an automatic method which uses a set of heuristics and domain ontology to perform the task. Domain ontology is costly to build (about 2-man weeks for a given Web site) [10]. [2] extends this approach by designing some additional heuristics without using any domain knowledge. We will show in the experiment section that the performance of this approach is poor. [4] proposes another automatic method, which uses Patricia tree [11] and approximate sequence alignment to find patterns (which represent a set of data records) in a Web page. Due to the inherent limitation of Patricia tree and inexact sequence matching, it often produces many patterns and most of them are spurious. In many cases, none of the actual data records is found. Again, this method performs poorly. [19] proposes a method using clustering
Mining data records in web pageMining data records in web page隐藏>> Mining Data Records in Web Pages Bing Liu Department of Computer Science University...
Web Conference [WWW7], pages 541--550._专业资料。[S73] H. Small. ... Mining data records in... 10页 免费 喜欢此文档的还喜欢 1 Locality ...
networks of web pages, complex relational ...Propositional data are individual records, each of...Data mining in networks. Papers of the Symposium...
They concern mining patient records, tax returns, telecommunication data, consumer transactions, content of web pages, and software source code. These ...
web pages, click streams, multimedia data, business transactions, or ...In many of the data mining applications mentioned above, the number of ...
Share in Pages 暂无评价 4页 1下载券 Example_CV_two_pages 暂无评价 2页 2下载券 Mining data records in... 6页 免费 Mining data records in... 10...
FiVaTech Page-Level Web ... 6页 免费 ABSTRACT DATA MINING ... 267页 ... ABSTRACT Collecting Hidden Web Pages for Data ExtractionABSTRACT Collecting ...
Zhai. Mining Data Records in Web Pages. In Proc. of the ACM SIGKDD Intl. Conf. on Knowledge Discovery and Data Mining (KDD’03), pp. 601–606. ...
principles of data mining_计算机软件及应用_IT/...Records Table Database Statistics Articial ...web pages using a feature vector whose elements ...
Data Mining with Python_IT/计算机_专业资料。...(Linguistics) Which web pages are visited in the...(20 clusters of 50 easier than 1000 records) ...
热门文档