Classification and Clustering via Dictionary Learning with Structured Incoherence and Shared Features
Classi cationandClusteringviaDictionaryLearningwithStructured
IncoherenceandSharedFeatures
IgnacioRamirez,PabloSprechmann,andGuillermoSapiro ElectricalandComputerEngineering,UniversityofMinnesotaAbstract
Aclusteringframeworkwithinthesparsemodelinganddictionarylearningsettingisintroducedinthiswork.In-steadofsearchingforthesetofcentroidthatbest tthedata,asink-meanstypeofapproachesthatmodelthedataasdistributionsarounddiscretepoints,weoptimizeforasetofdictionaries,oneforeachcluster,forwhichthesignalsarebestreconstructedinasparsecodingmanner.Thereby,wearemodelingthedataasaunionoflearnedlowdimen-sionalsubspaces,anddatapointsassociatedtosubspacesspannedbyjustafewatomsofthesamelearneddictionaryareclusteredtogether.Anincoherencepromotingtermen-couragesdictionariesassociatedtodifferentclassestobeasindependentaspossible,whilestillallowingfordiffer-entclassestosharefeatures.Thistermdirectlyactsonthedictionaries,therebybeingapplicablebothinthesu-pervisedandunsupervisedsettings.Usinglearneddictio-nariesforclassi cationandclusteringmakesthismethodrobustandwellsuitedtohandlelargedatasets.Thepro-posedframeworkusesanovelmeasurementforthequal-ityofthesparserepresentation,inspiredbytherobustnessofthe 1regularizationterminsparsecoding.Inthecaseofunsupervisedclassi cationand/orclustering,anewini-tializationbasedoncombiningsparsecodingwithspectralclusteringisproposed.Thisinitializationclustersthedic-tionaryatoms,andthereforeisbasedonsolvingalowdi-mensionaleigen-decompositionproblem,beingapplicabletolargedatasets.We rstillustratetheproposedframeworkwithexamplesonstandardimageandspeechdatasetsinthesupervisedclassi cationsetting,obtainingresultscompa-rabletothestate-of-the-artwiththissimpleapproach.Wethenpresentexperimentsforfullyunsupervisedclusteringonextendedstandarddatasetsandtextureimages,obtain-ingexcellentperformance.
dueinparttothefactthatanimportantvarietyofsignalssuchasaudioandnaturalimagescanbewellapproximatedbyalinearcombinationofafewelements(atoms)ofsome(often)redundantbasis,usuallycalleddictionaries[2].Sparsemodelingaimsatlearningthesenonparametricdictionariesfromthedataitself.Severalalgorithmshavebeendevelopedforthistask,e.g.,theK-SVDandthemethodofoptimaldirections(MOD).Recentpublicationsinawidespectrumofsignalsandapplicationshaveshownthatthisapproachcanbeverysuccessful,leadingtostate-of-theartresults,e.g.,inimagerestorationanddenoising,texturesynthesis,andtextureclassi cation.Inthesupervisedorweaklysupervisedclassi cationsetting,thisclassofalgo-rithmslearndictionariesfromthelabeledtrainingdatasetandusefeaturesofthesparsedecompositionofthetestingsignalforclassi cation(see[14,17,22,34]).
Inthispaperweproposeaframeworkforclusteringdatasetsthatarewellrepresentedinthesparsemodelingframeworkwithasetoflearneddictionaries(see[26]forourearlierworkinthisdirection).GivenKclusters,welearnKdictionariesforrepresentingthedata,andthenas-sociateeachsignaltothedictionaryforwhichthe“best”sparsedecompositionisobtained.Notethatitisnotthateachdatapointbelongstoaunionofsubspacesasforex-amplein[7,10].Comparingwithblock/groupsparsity,hereasingledictionary(block)isselectedperdatapoint,andthepointissparselyrepresented(subspace)withatomsonlyfromthisdictionary.1Alsoincontrastwithmoreclassicalsubspaceclustering,datapointsinthesameclasscanbe-longtomorethanonesubspace,sinceeachdictionaryrep-resentsalargenumberofsubspaces(eachsparsitypatternde nesonesubspace).Themodelisthenveryrichandnon-linear.
The rstbuildingblockoftheproposedclusteringframe-workisbasedonconsidering
min
K i=1xj∈Ci
1.IntroductionandBasicFormulation
Inrecentyears,sparserepresentationshavereceivedalot
ofattentionfromthesignalprocessingcommunity.Thisis
IR
1Sharing
Di,Ci
R(xj,Di),(1)
andPScontributedequally.
atomsbetweentheclasses,andthereforehavingnon-empty
intersectingsubspacesispermittedinourframeworkaswell,seeSec-tion3.1
1
Projective dictionary pair learning for pattern classification_计算机软件及应用...cation and clustering via dictionary learning with structured incoherence and ...
Projective dictionary pair learning for pattern classification_计算机软件及应用...cation and clustering via dictionary learning with structured incoherence and ...
热门文档