High dimensional directional data is becoming increasingly important in contemporary applications such as analysis of text and gene-expression data. A natural model for multivariate directional data is provided by the von Mises-Fisher (vMF) distribution on
GenerativeModel-basedClusteringofDirectionalData
ArindamBanerjee
InderjitDhillon
UniversityDeptofAustin,TX,ofECETexasUSA
UniversityDeptofAustin,TX,ofCSTexasUSAabanerje@ece.utexas.edu
inderjit@cs.utexas.edu
HighABSTRACT
dimensionaldirectionaldataisbecomingincreasinglyimportantincontemporaryapplicationssuchasanalysisoftextandgene-expressiondata.Anaturalmodelformulti-variatedirectionaldataisprovidedbythevonMises-Fisher(vMF)distributionontheunithyperspherethatisdanal-ogoustothemulti-variateGaussiandistributioninR.Inthispaper,weproposemodelingcomplexdirectionaldataasamixtureofvMFdistributions.WederiveandanalyzetwovariantsoftheExpectationMaximization(EM)frameworkforestimatingtheparametersofthismixture.Wealsopro-posetwoclusteringalgorithmscorrespondingtothesevari-ants.Aninterestingaspectofourmethodologyisthatthesphericalkmeansalgorithm(kmeanswithcosinesimilarity)canbeshowntobeaspecialcaseofbothouralgorithms.Thus,modelingtextdatabyvMFdistributionslendstheo-reticalvaliditytotheuseofcosinesimilaritywhichhasbeenwidelyusedbytheinformationretrievalcommunity.Aspartofexperimentalvalidation,wepresentresultsonmodelinghigh-dimensionaltextandgene-expressiondataasamix-tureofvMFdistributions.Theresultsindicatethatourapproachyieldssuperiorclusteringsespeciallyfordi cultclusteringtasksinhigh-dimensionalspaces.
H.3.3Categories[InformationandSubjectStorageDescriptors
andRetrieval]:Informa-tionSearchandRetrieval;I.5.3[PatternRecognition]:Clustering
Clustering,Keywords
directionaldata,mixtures,vonMises-Fisher,EM
Clustering1.INTRODUCTION
orsegmentationofdataisafundamentaldataanalysisstepthathasbeenwidelystudiedacrossvariousdis-ciplines[19].However,severallargedatasetsthatarebeingacquiredfromscienti cdomains,aswellastheworldwideweb,haveavarietyofcomplexcharacteristicsthatseverelychallengetraditionalmethodsforclustering[15].Thisar-ticleisconcernedwiththeclusteringofhigh-dimensional
Permissionpersonaltomakedigitalorhardcopiesofallornotbearmadeororclassroomdistributeduseforispro tgrantedorcommercialwithoutfeeprovidedpartofthatthiscopiesworkforarerepublish,thisnoticepermissiontopostandontheserversfullcitationoronthe rstpage.advantageTocopyandotherwise,thatcopiesorDC,USA.Copyrightand/orafee.2003SIGKDDtoredistributetolists,requirespriorspeci cACM1-58113-737-0/03/0008...’03,August24-27,2003,$5.00.
Washington,JoydeepGhosh
SuvritSra
UniversityDeptofofECETexasUniversityDeptofCSAustin,TX,USAAustin,TX,ofTexasUSAghosh@ece.utexas.edu
suvrit@cs.utexas.edu
directionaldatathatisbecomingincreasinglycommoninseveralapplicationdomains.
Onecanbroadlycategorizeclusteringapproachesintogener-ative(parametric)[29,18]anddiscriminative(non-parametric)[17,27]ones.Theperformanceofanapproach(andofaspe-ci cmethodwithinthatapproach)isquitedatadependent;thereisnoclusteringmethodthatworksthebestacrossalltypesofdata.Generativemodels,however,oftenpro-videbetterinsightintothenatureoftheclusters.Fromanapplicationpointofview,alotofdomainknowledgecanbeincorporatedintogenerativemodelssothatclusteringofdatauncoversspeci cdesirablepatternsthatoneislook-ingfor.Clusteringalgorithmsusingthegenerativemodelframework,ofteninvolveanappropriateapplicationoftheExpectationMaximization(EM)algorithm[8,5]onaprop-erlychosenstatisticalgenerativemodelforthedataunderconsideration.Forvectordata,therearewellstudiedclus-teringalgorithmsforpopulargenerativemodelssuchasamixtureofGaussians,whosee ectisanalogoustotheuseofEuclideanorMahalanobistypedistancesfromthedis-criminativeperspective.
There1.1areMotivationseveralapplicationandRelateddomainsWork
whereclusteringbasedonminimizingEuclideandistortionsyieldspoorresults[30].Forexample,empiricalstudiesininformationretrievalap-plicationsshowthatcosinesimilarityisafarmoree ectivemeasureofsimilarityforanalyzingandclusteringtextdocu-ments.Suchdomainsrequiretheuseofdirectionaldata[22]—datathatdealsonlywiththedirectionsofunitvectors.Thus,thereisaneedforgenerativemodelsthataremoreap-propriatefortheanalysisandclusteringofdirectionaldata.Inthisarticle,weproposeagenerativemixturemodelfordirectionaldataontheunithypersphereandderivetwoclus-teringalgorithmsusingthismixturemodel.Weshowtheconnectionbetweentheproposedalgorithmsandaclassofexistingalgorithmsforclusteringhigh-dimensional,direc-tionaldataandpresentdetailedexperimentalcomparisonsamongthem.
Asalreadymentioed,oneimportantdomainwheredirec-tionaldataisencounteredistextanalysis,andtextclus-teringinparticular.Ithasbeenexperimentallyshownthatinordertoremovethebiasarisingduetothelengthofadocument,itoftenhelpstonormalizethedatavectors[11].Further,thespkmeansalgorithm[11],thatperformskmeansusingcosinesimilarityinsteadofEuclideandistortion,hasbeenfoundtoempiricallyoutperformseveralotherschemes.
Generative model-based document clustering a comparative study Semi-supervised learning has become an attractive methodology for improving classification models ...
Generative model-based c... 暂无评价 10页 免费 Data clustering based on... 暂无评价 5页 20财富值 Uncertain Centroid based... 暂无评价 12页 免费 ...
clustering, has been shown to be a special case of a generative model ...directional data in the sense that it is the unique distribution of L2 -...
Generative model-based... 暂无评价 10页 免费 Generative model-based... ...Clustering in Relational Data/Community Detection Topic Models, (Generative ...
E cient algorithms for model-based hierarchical clustering for special cases of Gaussian generative models are described in Fraley ( 1]). In this paper ...
12页 1财富值 Generative model-based c... 暂无评价 10页 免费... A Fuzzy Clustering Model of Data with Proportional Membership S. Nascimento...
Hamdan and Govaert developed a theory on mixture model-based clustering for interval data. In this context, they proposed two interval data-based maximum ...
Clustering Images Usin... 暂无评价 18页 免费 Latent Dirichlet Alloc... 6页 免费 Latent Dirichlet Alloc... 30页 免费 Latent Dirichlet Alloc... 30页...
generated from a particular class of generative ...Each entry shows the directional change from an ...rg K?mper. Clustering of gene expression data ...
Bayesian regularization for normal mixture estimation and model-based clustering Normal mixture models are widely used for statistical modeling of data, ...
热门文档