Learning deep features for multiple object tracking by using a multi-task learning strategy

Learning deep features for multiple object tracking by using a multi-task learning strategy

LEARNINGDEEPFEATURESFORMULTIPLEOBJECTTRACKING

BYUSINGAMULTI-TASKLEARNINGSTRATEGY

LiWang

NamTrungPham Tian-TsongNg GangWang KapLukChan KariantoLeman

SchoolofElectricalandElectronicEngineering,NanyangTechnologicalUniversity,Singapore

InstituteforInfocommResearch,A STAR,Singapore

ABSTRACT

Model-freeobjecttrackingisstillchallengingbecauseofthelimitedpriorknowledgeandtheunexpectedvariationofthetargetobject.Inthispaper,weproposeafeaturelearningalgorithmformodel-freemultipleobjecttracking.First,wepre-learngenericfeaturesinvarianttodiversemotiontrans-formationsfromauxiliaryvideodatabyusingadeepnetworkofanto-encoder.Then,weadaptthepre-learnedfeaturesac-cordingtomultipletargetobjectsrespectivelyinamulti-tasklearningmanner.Wetreatthefeatureadaptationforeachtar-getobjectasonesingletask.Wesimultaneouslylearnthecommonfeaturesharedbyalltargetobjectsandtheindivid-ualfeatureofeachobject.Experimentalresultsdemonstratethatourfeaturelearningalgorithmcansigni cantlyimprovemultipleobjecttrackingperformance.

IndexTerms—Multipleobjecttracking,deepfeaturelearning,multi-tasklearning

1.INTRODUCTION

Model-freemultipleobjecttrackingaimstolocatemultipletargetobjectsinavideosequencegiventhetargetobjects’lo-cationsinthe rstseveralframes.Ithasawiderangeofappli-cations,suchashuman-machineinteraction[1]andsurveil-lancevideoanalytics[2].Incontrasttoanotherlineofmulti-pleobjecttrackingusingdetectionassociation[3,4,5,6,7],model-freemultipleobjecttrackingisanextensionofsingleobjecttracking,whichdoesnotdependonclass-speci cde-tectorandcanonlinepredictlocationsofarbitraryobjects.

Model-freeobjecttrackingapproachescanbedividedintotwocategories:generative[8,9,10,11,12]anddiscrimina-tive[13,14,15,16,17].Forathoroughreview,interestedreaderscouldreferto[18].Mostofthetrackingmethodsre-viewedaboveareproposedintermsofsingleobjecttrack-ing.Allofthemcouldbenaturallyextendedtothecasesofmultipleobjecttracking.Forexample,wecouldtrackmul-tipletargetobjectsbysimplyrunningmultipleinstancesofasingleobjecttrackeronalltargetobjects.However,thiskindofstraightforwardmanipulationhasnotutilizedrelation-shipsbetweenmultipletargetobjects,whileexploringtheob-jectsimilarityinformation[19,20]isimportant.Recently,

Fig.1.Theoverviewofourfeaturelearningalgorithm.First,thegenericfeatureWpispre-learnedof inefromKauxiliaryvideosequences.Then,theindividualfeatureWi,i=1,...,MforeachobjectandthecommonfeatureWcsharedbyMobjectsaresimultaneouslylearnedinouronlinefeatureadaptationmodulewhichusestheproposedmulti-tasklearningstrategy.EachrowofatransformationmatrixWcanbevisualizedasa lterpatchasshowninthe gure.Hereonly25 lterpatchesarevisualizedforeachW.

aso-calledstructure-preservingobjecttracker(SPOT)[21]addressesthisproblembyconsideringspatialconstraintsbe-tweenobjects.Inthispaper,wearguethatexploringrelation-shipsbetweenobjectswillalsobene tfeaturelearningformultipleobjecttracking.

Featurerepresentationisimportantforvisualobjecttrack-ing.Traditionally,rawpixelvaluesandhand-craftedfeaturesaresimplyusedtorepresenttargetobjects,suchasHaar-likefeatures[22],histogramsoforientedgradients(HOG)[23]andlinearbinarypatterns(LBP)[24].Hand-craftedfeaturesaimtodepictsomepre-de nedimagepatterns.However,theyarenotabletocapturemorespeci candcomplexcharacter-isticsoftargetobjects.Recently,deeplearninghasreceivedmuchattentionsandachievedmanypromisingresultsinim-ageclassi cation[25]andspeechrecognition[26].Thedeeparchitectureisabletolearnincreasinglycomplicatedinvari-antfeatureswhicharenoteasilydescribedbyrawpixelvaluesandhand-craftedfeatures.WangandYeung[27]havealreadyproposedaso-calleddeeplearningtracker(DLT)toextractadeepcompactimagerepresentationforvisualobjecttracking.However,theyhavenotconsideredhowtolearndeepfeaturesformultipleobjecttracking.

Inthispaper,weproposeamulti-tasklearningstrategyto