where isalearningrate.But,exactmaximumlikelihoodlearninginthismodelisintractablebecauseexactcomputationoftheexpectationmodelisveryexpensive.Hence,inpractice,learningisdonebyfollowinganapproximationtothegradientofadifferentobjectivefunction,calledtheContrastiveDivergence(CD)[11].
2.2DeepBeliefNetworks
Oneofthemainproblemsintrainingdeepnetworksishowtoinitializeweights.Itisdif culttooptimizetheweightsinnonlinearDeepNetworkswithmultiplehiddenlayers.Withgoodinitialweights,gradientdescentworkswell,but ndingsuchinitialweightsrequiresaverydifferenttypeofalgorithmthatlearnsonelayeroffeaturesatatime.Hintonetal.[1]introducedanewalgorithmtosolvetheaboveproblembasedonthetrainingofasequenceofRBMs.ToconstructaDBNwetrainsequentiallyasmanyRBMsasthenumberofhiddenlayersintheDBN,i.e.foraDBN

withhhiddenlayerswehavetotrainhRBMs.TheseRBMsareplacedoneontopoftheother.Figure2givesanoverviewofthebasicconcept.Forthe rstRBM,whichconsistsoftheDBNsinputlayerandthe rsthiddenlayer,theinputofRBMisthetrainingset.ForthesecondRBM,whichconsistsoftheDBNs rstandsecondhiddenlayers,theinputistheoutputofthepreviousRBMandsoforotherRBMs.
Figure2:Left:GreedylearningastackofRBMsinwhichthesamplesfromthelower-levelRBMareusedasthedatafortrainingthenextRBM.Right:ThecorrespondingDeepBeliefNetwork
[13].
Afterperformingthislayer-wisealgorithmwehaveobtainedagoodinitializationforthehiddenweightsandbiasesoftheDBNandthentheDBNis ne-tunedwithrespecttoatypicalsupervisedcriterion(suchasmeansquareerrororcross-entropy).
3Learning
TotrainaDBNforimagedenoising,thenormalizedvaluesofanimagepixelsareused.Usingmin-maxnormalization,grayscalevalueofapixel(anintegerbetween0and255)istransformedtoa oatingpointnumberbetween0and1.Unlike rstandlastlayerofDBN,otherlayershavebinarynodes.ThemainideaistotrainaDBNsuchthatitlearnstomapnoisyimagestoimageswithlowernoiseorevenwithoutnoise.Theideacanbeimplementedbylearningthebehaviorofnoiseandimagecontentsandpresentingthesebehaviorsinsomenodesatthelastlayerofthenetwork.Thenetworkistrainedwithacollectionconsistedofbothnoisyandnoiselessimages.Weusedacriterioncalledrelativeactivitytodetectnoisenodes.Relativeactivityofeachnodeisde nedasthedifferencebetweentwovaluesofaparticularnoderesultedfromfeedingthenetworkusinganoiselessimageanditscorrespondingnoisyimage(Twoimageswithsamecontentsbutoneofthem
Image Denoising by Sparse Code Shrinkage Aapo Hyv rinen, Patrik Hoyer and Erkki Oja Neural Networks Research Centre Helsinki University of Technology P.O....
? ? ? ? Restricted Boltzmann machines Deep Belief Networks Denoising ...image 72 Convolutional RBM (CRBM) [Lee et al, ICML 2009] For “filter...
Deep Belief Network Ba... 暂无评价 11页 1下载券 喜欢此文档的还喜欢 ... Network Engineering for Complex Belief Networks Suzanne M. Mahoney Information...
Deep Belief Networks,作为 DBN 的 breakthrough 3.Deep Learning Algorithm 的...Denoising Auto-Encoders - easy steps into unsupervised pre-training for deep...
卷积深度信念网络ConvolutionalDeepBeliefNetworks_计算机软件及应用_IT/计算机_...For each column: (top) input image. (middle) reconstruction from the ...
Deep belief nets Stacked denoising auto-encoders ...Matching Using Sparse Coding for Image ...Y. Ng. Convolutional deep belief networks for ...
Deep Belief Networks Denoising Autoencoders K-means Spatial pyramids / ...a fast computer (for learning 25 features from 10000 8x8 image patches)....
widely applied to image fusion[4], segmentation[5] and image denoising[6]...for Boundary and Surface Representation, " Neural Networks, vol.8, pp.1005...
(sparse/denoising) Autoencoder Sparse Coding SHALLOW SP Deep Belief Net ... Visualizing and Understanding Convolutional Networks, arXiv 1311.2901, 2013 6...
Loopy belief propagation and probabilistic image processing,” Neural Networks for Signal P_专业资料。Abstract. The hyperparameter estimation in the ...
热门文档