文档下载 文档下载导航
设为首页 | 加入收藏
搜索 请输入内容:  
 导航当前位置: 文档下载 > 所有分类 > 工程科技 > 信息与通信 > !Synthesis distortion estimation in 3D video using frequency and spatial analysis
侵权投诉

!Synthesis distortion estimation in 3D video using frequency and spatial analysis

SYNTHESISDISTORTIONESTIMATIONIN3DVIDEOUSINGFREQUENCYANDSPATIALANALYSIS

LuFang

Ngai-ManCheung

DongTian AnthonyVetro

HuifangSun LuYu

Univ.ofSci.andTech.ofChinaSingaporeUniv.ofTech.andDesignMitsubishiElectricResearchLabsZhejiangUniv.

ABSTRACT

Weproposeananalyticalmodeltoestimatethesynthesizedviewqualityin3Dvideo.Speci cally,weestimatethedepth-errorinduceddistortionusinganapproachthatcombinesfrequencyandspatialdomainanalysis.Wealsoproposetodecomposethespatial-variantvideosignalsintogradient-basedrepresentationstocapturetheinteractionbetweenimagegradients,deptherrorsandsynthesisdistortion.Experimentresultswithvideosequencesandcoding/renderingtoolsusedinMPEG3DVactivitiesshowthatouranalyticalmodelcanaccuratelyestimatethesynthesisnoisepower.IndexTerms—3DVideo,SynthesisDistortionEstimation,FrequencyandSpatialAnalysis,DepthError

I.INTRODUCTION

3Dvideo(3DV)hasattractedmuchattentionrecently[1]–[5].3Ddatasetsusuallyconsistofmultiplevideosequences(texturedata)capturedbycamerasatdifferentpositions,alongwiththeassociateddepthimages.Thequalityofthesynthesizedviewisimperativein3DVapplications[6],[7].Thesynthesisquality,however,dependsonseveralfactorsandcomplicatedinteractionsbetweenthem.Inparticular,textureanddepthimagesmaycontainerrorsduetoimperfectsensingorlossycompression[8],[9],anditisnotclearhowtheseerrorsinteractandaffecttherenderingquality.Unliketextureerrors,whichcausedistortionintheluminance/chrominancelevel,deptherrorscausepositionerrorsinsynthesis[10],i.e.,pixelsarewarpedtoslightlyshiftedpositionsduringsynthesis.Theeffectofdeptherrorsisverysubtle.Forinstance,theimpactofdeptherrorswouldvarywiththeimagecontents,andimageswithlesstexturestendtobemoreresilienttothedeptherrors.

Anaccurateanalyticalmodeltoestimatethesynthesisqualityisveryvaluableforthedesignof3DVsystems.NguyenandDo[11]analyzedtherenderingqualityofimage-basedrendering(IBR)algorithmsandusedTaylorseriesexpansiontoderivetheupperboundofthemeanabsoluteerror(MAE)inthesynthesisoutput.Liuetal.[12]approximatederrorsduetodepthmapartifactsusingalinearmodelofaveragemagnitudeofmean-squareddisparityerrorsoveranentireframeandamotionsensitivityfactorcomputedfromtheenergydensity.AnautoregressivemodelwasproposedbyKimetal.[13]toestimatethesynthesisdistortionattheblocklevelandwasshowntobeeffectiveforrate-distortionoptimizedmodeselection.AdistortionmodelasafunctionoftheviewlocationwasalsoproposedbyVelisavljevicetal.[14]forbitallocation.Takahashi[15]proposedanoptimizedviewinterpolationschemebasedonfrequencydomainanalysisofdepthmaperror.

Inourpreviouswork[16],weproposedtoestimatethesynthesisqualityusingpowerspectraldensity(PSD).Thisassumedthattheunderlyingvideoframesignalsarespatialinvariant.However,signalsalongstrongtextureedgeschangemuchquicklythanthoseinthenon-edgeregions,i.e.,autocovariancefunctiondecaysmuch

fasterinedgepixels.Differentfrompreviousapproaches,weproposeinthisworkamodelthatcombinesfrequencyanalysisandspatialanalysistoaccountforthenon-stationaryinthesignals.Speci cally,ourcontributionsare:1)Weproposeananalyticalmodelcombiningfrequencyandspatialdomainanalysistoestimatethesynthesisviewquality,givendepthmaperrors,textureimagecharacteristics(smoothortextural),textureimagequalityandthecameracon gurationastheinputs;2)Inparticular,weproposetodecomposethespatialvariantsignalsintogradient-basedrep-resentationstofacilitateanalysis.Theanalysisresultsshowthatlinearapproximationsofthespatialvariantsignalscanleadtoacomputationally-ef cientandyetaccurateestimation;3)WeverifyourmodelwithsubstantialexperimentsusingvideosequencesandcodingandrenderingtoolsfromtheMPEG3DVactivities[17],[18].Therestofthepaperisasfollows.SectionsIIdiscussesouranalyticalmodel.SectionIIIpresentsexperimentresultsandSectionIVconcludesthepaper.

II.ESTIMATENOISEPOWERDUETODEPTHCODINGWe rstdiscussourviewsynthesismodel,whichconsistsofframewarpingfollowedbyblending.Tworeferencetextureframescapturedbytheleftandrightcameras(denotedbyXl(m,n)andXr(m,n)respectively)alongwiththeirassociateddepthimages(denotedbyDl(m,n)andDr(m,n)respectively)areusedtogeneratethesynthesizedframeU(m,n)atacertainvirtualcameraposition.First,inframewarping,pixelsarecopiedfromXltoformanintermediateframeUl,fromposition(m′,n)to(m,n).Weassumethecamerasarerecti edandarrangedlinearly,andthereexistsonlyhorizontaldisparitygivenbym m′determinedbythedepthimages,cameraparametersandcameradistance[16].Likewise,pixelsarecopiedfromXrtoformtheintermediateframeUr.Then,UlandUraremerged(blended)togenerateU.Weassumemergingbylinearcombination:U(m,n)=αUl(m,n)+(1 α)Ur(m,n).Heretheweightαisdeterminedbythedistancesbetweenthevirtualcamerapositionandtheleft/rightreferencecamerapositions.

Inpractice,thetextureanddepthimagesarelossyencoded.Weassumethatwhenthereconstructedtexture/depthimages l,X r,D l,D r)arefedintothesynthesispipeline,weobtain(X

renderingoutputW.LetV=U Wbethenoiseintherenderingoutputduetocodingerrorsintexture/depthimages.In[16],weshowthatunderreasonableassumptionsthetotalsynthesisnoisepower(E[V2])canbeestimatedbysummingtwocomponents:oneisthesynthesisnoisepowerduetotextureimagecoding(E[N2]),theotheristhesynthesisnoisepowerduetodepthimagecoding(E[Z2]),i.e.,

E[V2]=E[N2]+E[Z2].(1)WediscussedtheestimationofE[N2]in[16]andthefocusinthis

paperisontheestimationofE[Z2].

第1页

你可能喜欢

热门文档

站点地图 | 文档上传 | 侵权投诉 | 手机版
本站所有资源均来自互联网,本站只负责收集和整理,均不承担任何法律责任,如有侵权等其它行为请联系我们.
文档下载 Copyright 2013 doc.guandang.net All Rights Reserved.  -  email