DenseCorrespondenceFindingforParametrization-freeAnimation
ReconstructionfromVideo
NaveedAhmedMPIInformatik
ChristianTheobaltStanfordUniversity
ChristianR¨osslMagdeburgUniversity
SebastianThrunStanfordUniversity
Hans-PeterSeidelMPIInformatik
Abstract
Wepresentadense3Dcorrespondence ndingmethod
thatenablesspatio-temporallycoherentreconstructionofsurfaceanimationsfrommulti-viewvideodata.Givenasin-putasequenceofshape-from-silhouettevolumesofamov-ingsubjectthatwerereconstructedforeachtimeframein-dividually,ourmethodestablishesdensesurfacecorrespon-dencesbetweensubsequentshapesindependentlyofsurfacediscretization.Thisisachievedintwosteps: rst,weob-tainsparsecorrespondencesfromrobustopticalfeaturesbetweenadjacentframes.Second,wegeneratedensecor-respondenceswhichserveasmapbetweenrespectivesur-faces.Byapplyingthisproceduresubsequentlytoallpairsoftimestepswecantriviallyalignoneshapewithallothers.Thus,theoriginalinputcanbereconstructedasasequenceofmesheswithconstantconnectivityandsmalltangentialdistortion.Weexemplifytheperformanceandaccuracyofourmethodusingseveralsyntheticandcapturedreal-worldsequences.
1.Introduction
Inrecentyears,evermoreef cientcomputersandin-creasinglyaccurateimagingdeviceshaverendereditfeasi-bletocapturecomputeranimationsfromsubjectsperform-inginthereal-worldratherthanbyhand-craftingthemwiththetraditionaltoolboxoftheanimator.Tothisend,ava-rietyofmethodshavebeendevelopedthatreconstructbothtime-varyingshapeandappearanceofarbitraryreal-worldperformersfrommulti-viewpointvideo,Sect.2.
Mostofthesemethodsprovideconvincingshapeandappearanceforeachtimestepofaninputanimationindi-vidually.However,theyfallshortofreconstructingspatio-temporallycoherentscenegeometryforarbitrarysubjectssincethechallenging3Dcorrespondenceproblemisnotaddressed.Spatio-temporalcoherenceisanimportantandhighly-desirablepropertyincapturedanimations,asitgreatlyfacilitatesorevenisinevitableformanytaskssuchasediting,compressionorspatio-temporalpostprocessing.
978-1-4244-2243-2/08/$25.00 ©2008 IEEE
Wethereforeproposeanewspatio-temporaldense3Dcorrespondence ndingmethodthatenablesustocapturecoherentdynamicscenegeometryusingstandardshape-from-silhouettemethods,Sect.3.Ouralgorithmistailoredtothecharacteristicsofvideo-basedreconstructionmethodswhichoftencapturehighspatialdetailintheinputvideoframes,butproviderelativelysparselysampled3Dgeome-trywithamuchlowerlevelofshapedetailandwithacon-siderablelevelofnoise.
Ina rststep,shape-from-silhouettesurfacesarerecon-structedforeachtimestepofvideoyieldingasequenceofshapesmadeoftrianglemesheswithvaryingconnectivity.Thereafter,sparse3Dcorrespondencesbetweensubsequentpairsofsurfacesarecomputedbymatching3Dpositionsofopticalfeaturesthatcanbeaccuratelyextractedfromhigh-resolutioninputvideoframes,Sect.3.1.Thesesparsecorre-spondencesrepresentcontrolpointsforanchoringappropri-atebivariatescalarfunctionsoneachreconstructedsurfacemesh,Sect.3.2.Thechoiceofthesefunctionsenablesustoestablishdensecorrespondenceessentiallybymatchingfunctionvalues.Thedensecorrespondencescanbeusedtostraightforwardlyalignonemeshtoallotherreconstruc-tionsbyperformingasequenceofpairwiseregistrations,Sect.3.3.Theoutputofourapproachisaspatio-temporallycoherentanimation,i.e.asequenceofmesheswithconstantgraphstructureandlowtangentialdistortion.
2.RelatedWork
Technologicalprogressinrecentyearshasmadeitfeasibletoreconstructshapeandappearanceofdynamicscenesusingvideo[16]orvideoplusactivesensing[28].Multi-viewvideomethodsbasedontheshape-from-silhouette[17]orstereoprinciple[30]beartheintriguingadvantagethattheyenablereconstructionofarbitrarymov-ingsubjects.Unfortunately,Noneofthesemethodsisde-signedtoreconstructscenegeometrywithcoherentcon-nectivityovertimesincethe3Dcorrespondenceproblemisnotaddressed.Model-basedapproachesemployshapepriors[7,6]whichlimitsthemtocertaintypesofscenes.Thealgorithmproposedinthispaperenablescoherentdy-
[6, 17], animation of faces and whole bodies ...computation of dense correspondence between the ...Registration establishes a common parametrization ...
finding intrinsic correspondence between three-...Unlike synthetic object animation, where such a ...Global parametrization is often readily available ...
热门文档