Method and electronic device for object tracking in a light-field capture

Patent metadata

Field	Value
Publication number	US-9947106-B2
Application number	US-201514975665-A
Country	US
Kind code	B2
Filing date	Dec 18, 2015
Priority date	Dec 18, 2014
Publication date	Apr 17, 2018
Grant date	Apr 17, 2018

Abstract

A method and an electronic device for object tracking in a sequence of light-field captures. A data acquisition unit acquires a sequence of light-field captures, wherein each light-field capture comprises a plurality of views. A feature determining unit determines features of an initial visual appearance model for an object of interest in a reference view of a first light-field capture. A feature matching unit matches the features in the reference view and in the further views of the first light-field capture. A feature discarding unit discards features that cannot be well matched in all views of the first light-field capture. An appearance model building unit builds an updated visual appearance model for the object of interest based on the remaining features. Finally, a movement tracking unit tracks the movement of the object of interest in the sequence of light-field captures using the visual appearance model.

First claim

What is claimed, is:

1. A method for object tracking in a sequence of light-field captures, the method comprising:
acquiring a sequence of light-field captures with a light-field camera, each light-field capture having a plurality of views, the plurality of views corresponding to displaced viewpoints of the camera in a two dimensional plane, which is perpendicular to a main axis of the camera, one view among the plurality of views being a reference view;
determining features of an initial visual appearance model for an object of interest in the reference view of a first light-field capture;
matching the features in the reference view and in the further views of the first light-field capture;
discarding features that cannot be matched in all or most of the views of the first light-field capture;
building an updated visual appearance model for the object of interest based on the remaining features; and
tracking a movement of the object of interest in the sequence of light-field captures using the visual appearance model.

2. The method according to claim 1 , wherein the views of each light-field capture are acquired in a timely synchronized manner.

3. The method according to claim 1 , wherein the matching of the features in the reference view and in the further views of the first light-field capture is based on a similarity metric score.

4. A non-transitory computer readable storage medium having stored therein instructions enabling object tracking in a sequence of light-field captures, which, when executed by a computer, cause the computer to:
acquire a sequence of light-field captures with a light-field camera, each light-field capture having a plurality of views, the plurality of views corresponding to displaced viewpoints of the camera in a two dimensional plane, which is perpendicular to a main axis of the camera, and one view among the plurality of views being a reference view;
determine features of an initial visual appearance model for an object of interest in the reference view of a first light-field capture;
match the features in the reference view and in the further views of the first light-field capture;
discard features that cannot be matched in all or most of the views of the first light-field capture;
build an updated visual appearance model for the object of interest based on the remaining features; and
track a movement of the object of interest in the sequence of light-field captures using the visual appearance model.

5. The non-transitory computer readable storage medium according to claim 4 , wherein the instructions cause the computer to acquire the views of each light-field capture in a timely synchronized manner.

6. The non-transitory computer readable storage medium according to claim 4 , wherein the matching of the features in the reference view and in the further views of the first light-field capture is based on a similarity metric score.

7. An electronic device for object tracking in a sequence of light-field captures, wherein the electronic device comprises:
a processor coupled to a memory, the processor being configured to:
acquire a sequence of light-field captures with a light-field camera, wherein each light-field capture comprises a plurality of views, the plurality of views corresponding to displaced viewpoints of the camera in a two dimensional plane, which is perpendicular to a main axis of the camera, one view among the plurality of views being a reference view;
determine features of an initial visual appearance model for an object of interest in the reference view of a first light-field capture;
match the features in the reference view and in the further views of the first light-field capture;
discard features that cannot be matched in all or most of the views of the first light-field capture;
build an updated visual appearance model for the object of interest based on the remaining features;
track a movement of the object of interest in the sequence of light-field captures using the visual appearance model; and
output data relative to the movement of the object of interest in the sequence of light-field captures.

8. The electronic device according to claim 7 , wherein the processor is coupled to a light-field camera.

9. The electronic device according to claim 7 , wherein the processor is configured to acquire the views of each light-field capture in a timely synchronized manner.

10. An electronic device for object tracking in a sequence of light-field captures, the electronic device comprising a processing device and a memory device having stored therein instructions, which, when executed by the processing device, cause the electronic device to:
acquire a sequence of light-field captures with a light-field camera, each light-field capture having a plurality of views, the plurality of views corresponding to displaced viewpoints of the camera in a two dimensional plane, which is perpendicular to a main axis of the camera, one view among the plurality of views being a reference view;
determine features of an initial visual appearance model for an object of interest in the reference view of a first light-field capture;
match the features in the reference view and in the further views of the first light-field capture;
discard features that cannot be matched in all or most of the views of the first light-field capture;
build an updated visual appearance model for the object of interest based on the remaining features; and
track a movement of the object of interest in the sequence of light-field captures using the visual appearance model.

11. The electronic device according to claim 7 , wherein the processor is configured to match the features in the reference view and in the further views of the first light-field capture based on a similarity metric score.

12. An apparatus for processing image data, in particular video post-production system or video surveillance system, comprising the electronic device according to claim 7 or 10 .

13. The electronic device according to claim 10 , wherein the electronic device is coupled to a light-field camera.

14. The electronic device according to claim 10 , wherein the instructions cause the electronic device to acquire the views of each light-field capture in a timely synchronized manner.

15. The electronic device according to claim 10 , wherein the processing device causes the electronic device to match the features in the reference view and in the further views of the first light-field capture based on a similarity metric score.

Assignees

• Thomson Licensing
• Thomson Licensing Dtv

Inventors

• Le Clerc Francois
• Robert Philippe
• Varanasi Kiran

Classifications

• G06V10/7557

  based on appearance, e.g. active appearance models [AAM] · CPC title

• G06K9/00771

  Physics · mapped topic

• G06T2207/10016

  Video; Image sequence · CPC title

• G06T2207/10052

  Images from lightfield camera · CPC title

• G06K9/621

  Physics · mapped topic