文档下载 文档下载导航
设为首页 | 加入收藏
搜索 请输入内容:  
 导航当前位置: 文档下载 > 所有分类 > 自然科学 > 生物学 > Research+and+Implement+of+Classification+Algorithm+on+Web+Text+Mining
侵权投诉

Research+and+Implement+of+Classification+Algorithm+on+Web+Text+Mining

characteristic vector. Then it confirms categories of samples that need to be divided by method of computing text similitude degree. When the text is denoted as a model of space vector, the similitude degree of text can be denoted recurring to inner accumulate among the characteristic vector.

In practical application, VSM algorithm commonly establishes category vector space according to the training sample in the database and the sort system in advance. As carrying classification to a sample waiting to sort, it only need to compute the similitude degree what is namely inner accumulate to the sample waiting to sort with each category vector. Then we can select similitude degree the biggest category act as the corresponding category of the sample waiting to sort. As computing space vector of the category in advance in the VSM method is needed, the establishment of the space vector very great degree depends on the characteristic items which are included in the category vector. According to the research discover, there are more excessive nonzero characteristic items which are included in the category, the expression capability to the category in inclusive each characteristic item is more weak. Therefore compared with other classification method, VSM algorithm is fitter for the classification of the professional literature.

Among them, tfik means lemma emergence frequence in Dk, N means classification system number, Nk means lemma web page frequence.

The similitude degree method of literature search technique is adopted in the system to classification mining namely carry on characteristic vector match. Suppose that the sample information is U, needed to be classified information is V, cosine of vector angle can be used to measure both of the similitude degree, it is shown as formula (2).

Sim(V,U) = cos ( V ,U)=

k=1k=1 (2) k=1

Text classification is a kind of typical model directive machine learing problem. It is generally divided into training and categorizing two stages. Its concrete algorithm is described as follows: Training stage:

(1) C={c 1, c 2, ……, cn} // Define the category set (2) S ={s 1, s 2, ……, sm} // Give training text set For i =1 to m

Training text si is marked as the sign cj that is belonged to category Endfor

(3) For i =1 to m

X[si] Å characteristic vector of si

X[cj] Å characteristic vector is representative of each category Cj of corresponding si Endfor

Categorizing stage:

(4) TreeÅinformation classification tree

nÅleaf node total number of the tree

X[1…n]Å characteristic vector of each leaf node

Threshold[1…n] Å threshold of information

similitude degree for each leaf node

InfoÅinformation to wait for classification (5) For i=1 to n Do

DegÅ information similitude degree between

Info and X[i]

If Deg> Threshold[i] Then

Add Info and Deg to classification form of

corresponding X[i] sort

For each node P from X[i] node to Tree root

path Do

Add Info and Deg to classification form of

corresponding P sort

Endfor Endif Endfor

∑(W

n

vk

*Wuk)

∑W∑W

2vk

nn

2

uk

4.2. Text classification algorithm

So, Multilayer information classification strategy is adopted in the paper. First, the whole extraction information set can be divided into some big types. Second, the each big type can be subdivided, until they couldn’t be subdivided again. So, the whole extraction information set is looked as the root of the tree, and each classification as the node of the tree, the whole classification system constitutes a text classification tree.

To classify information at first, the text is compared with all the sorts of leaf nodes. The information belongs to the sort of one leaf node, it also belongs to all the sorts of nodes from the leaf node to the root node. Vice versa, the text can only be compared with the sorts of all leaf nodes to classify a certain text. The web page is regarded as a set of lemma form of ( T1, T2, …, Tn ) in the VSM. Each lemma is endowed with certain authority value Wi. Each web page can be expressed by a web page characteristic vector (T1, W1; T2, W2; …; Tn, Wn) to express. The formula (1) can be used to calculate each lemma authority value Wik for all web pages in each web page train set Dk.

Wik = (1 + ln tfik ) ln( N/ Nk) (1)

5. Examples

上一页第3页

你可能喜欢

热门文档

相关文档

站点地图 | 文档上传 | 侵权投诉 | 手机版
本站所有资源均来自互联网,本站只负责收集和整理,均不承担任何法律责任,如有侵权等其它行为请联系我们.
文档下载 Copyright 2013 doc.guandang.net All Rights Reserved.  -  email