onthevaluesoflastlayernodes.Subsequently,ifweeliminatetheeffectsofnodespresentingthenoise,thereconstructedimagewillbenoiseless.Fromnowon,wecalledthenodespresentingnoiseandthenodespresentingimagecontent“noisenodes”and“imagenodes”,respectively.Therestofthepaperisorganizedasfollows:InSection2webrie ydescribeDeepLearning.InSection3wedescribethelearningprocess.Section4isourexperimentalresultsand nally,weconcludethepaperinSection5.
2DeepLearning
RestrictedBoltzmannMachines(RBMs)arethebuildingblocksofDBNs.Hence,inthissection rstwebrie ydescribeRBMsandthenwillexploreDBNs.
2.1RestrictedBoltzmannMachines
BoltzmannMachines(BMs)andRestrictedBoltzmannMachines(RBMs)wereintroducedin1980s.Buttheyattractedmoreattentionsince2006afterHintonetal.paper[8].HeshowedthataverypowerfulneuralnetworkcanbemadebystackingRBMs.RBMsareakindofMarkovRandomFields(MRF)whichhaveatwo-layerstructure.Onelayeriscalledvisibleandanotheriscalledhiddenlayer.Theyarerestrictedtohavenovisible-visibleor

hidden-hiddenconnectionandcon-nectionsareinterlayer.AgraphicaldepictionofRBMisshowninFigure1.
Figure1:StructureofaRestrictedBoltzmannMachine.
Ajointcon guration,(v,h)ofvisibleandhiddenunitshasanenergygivenby[9]:
E(v,h)= aivi bjhj vihjwi,j
i∈visiblej∈hiddeni,j(1)
wherevi,hjarethebinarystatesofvisibleunitiandhiddenunitjandai,bjaretheirbiasesandwijistheweightbetweenthem.Thenetworkassignsaprobabilitytoeverypossiblepairofavisibleandahiddenvectorviathisenergyfunction[10]:
e E(v,h)
p(v,h)= E(v,h)v,he(2)
Theprobabilitythatthenetworkassignstoavisiblevector,v,isgivenbysummingoverallpossiblehiddenvectors: E(v,h)ep(v)=h
E(v,h)(3)ev,h
logp(v))=<vihj>data <vihj>model ωi,j(4)
wheretheanglebracketsareusedtodenoteexpectationsunderthedistributionspeciesbythesub-scriptthatfollows.Thisleadstoaverysimplelearningruleforperformingstochasticsteepestascentinthelogprobabilityofthetrainingdata:
ωij= (<vihj>data <vihj>model)
(5)
Image Denoising by Sparse Code Shrinkage Aapo Hyv rinen, Patrik Hoyer and Erkki Oja Neural Networks Research Centre Helsinki University of Technology P.O....
? ? ? ? Restricted Boltzmann machines Deep Belief Networks Denoising ...image 72 Convolutional RBM (CRBM) [Lee et al, ICML 2009] For “filter...
Deep Belief Network Ba... 暂无评价 11页 1下载券 喜欢此文档的还喜欢 ... Network Engineering for Complex Belief Networks Suzanne M. Mahoney Information...
Deep Belief Networks,作为 DBN 的 breakthrough 3.Deep Learning Algorithm 的...Denoising Auto-Encoders - easy steps into unsupervised pre-training for deep...
卷积深度信念网络ConvolutionalDeepBeliefNetworks_计算机软件及应用_IT/计算机_...For each column: (top) input image. (middle) reconstruction from the ...
Deep belief nets Stacked denoising auto-encoders ...Matching Using Sparse Coding for Image ...Y. Ng. Convolutional deep belief networks for ...
Deep Belief Networks Denoising Autoencoders K-means Spatial pyramids / ...a fast computer (for learning 25 features from 10000 8x8 image patches)....
widely applied to image fusion[4], segmentation[5] and image denoising[6]...for Boundary and Surface Representation, " Neural Networks, vol.8, pp.1005...
(sparse/denoising) Autoencoder Sparse Coding SHALLOW SP Deep Belief Net ... Visualizing and Understanding Convolutional Networks, arXiv 1311.2901, 2013 6...
Loopy belief propagation and probabilistic image processing,” Neural Networks for Signal P_专业资料。Abstract. The hyperparameter estimation in the ...
热门文档