Object tracking for artificial vision

Patent metadata

Field	Value
Publication number	US-10062303-B2
Application number	US-201715609953-A
Country	US
Kind code	B2
Filing date	May 31, 2017
Priority date	Sep 30, 2009
Publication date	Aug 28, 2018
Grant date	Aug 28, 2018

Abstract

This invention concerns the tracking of objects in video data for artificial vision; for instance for a bionic eye. More particularly, the invention concerns a vision enhancement apparatus for a vision-impaired user. In other aspects, the invention concerns a method for enhancing vision and software to perform the method. The image processor operates to process video data representing images of a scene. Automatically detect and track a user selected object, such as a face, in the images. And, automatically modify the video data, by reserving a user selected area of the displayed images for displaying the tracked object as a separate video tile within the scene. The separate video tile remains in the selected area despite movement of the camera relative to the scene, or movement of the user relative to the object or the scene.

First claim

The invention claimed is:

1. A vision enhancement apparatus for a vision-impaired user, comprising an image processor arranged to:
process high resolution video data representing images of a scene and display the resulting images to the user;
automatically detect and track a user selected object in the images; and
automatically modify the video data, by reserving a user selected area of the displayed images for displaying the tracked object as a separate video tile within the images of the scene to provide modified video data that includes:
the displayed images of the scene that includes the tracked object; and
the separate video tile, at the user selected area within the displayed images, to display the tracked object,

wherein the tracked object remains displayed at the separate video tile despite movement of the user or the tracked object.

2. A vision enhancement apparatus according to claim 1 , wherein the object is identified within a first box and the separate video tile is a second box.

3. A vision enhancement apparatus according to claim 2 , wherein the object in the first box is displayed in the separate video tile.

4. The vision enhancement apparatus according to claim 2 , wherein the second box is the same size and shape as the first box.

5. A vision enhancement apparatus according to claim 1 , wherein the user selects the location for the separate video tile such that it is located in a part of the displayed images where it is visible to the vision-impaired user.

6. A vision enhancement apparatus according to claim 5 , wherein in the event the user is suffering from macular degeneration the separate video tile is located in their peripheral vision.

7. A vision enhancement apparatus according to claim 1 , wherein the separate video tile will remain in that part of the user's field of vision despite movement of the user, the tracked object or a camera mounted on the head of the user for filming the scene in front of the user.

8. A vision enhancement apparatus according to claim 1 , wherein the user has a high resolution video camera mounted on their head so that it films the scene in front of them.

9. A vision enhancement apparatus according to claim 1 , wherein the processor automatically detects faces in the scene and processes the images to identify the faces.

10. A vision enhancement apparatus according to claim 9 , wherein the apparatus announces newly identified faces to the user and then tracks their movements within the scene.

11. A vision enhancement apparatus according to claim 9 , wherein while a face remains in the scene it is continually tracked so the image of it is continually updated to show its changing facial expressions in the separate video tile.

12. A vision enhancement apparatus according to claim 3 , wherein the user is able to select more than one object to be displayed in respective separate video tiles of the displayed images.

13. A vision enhancement apparatus according to claim 12 , wherein the user selects which objects are displayed and which video tile each is displayed in within the displayed images.

14. A vision enhancement apparatus according to claim 1 , wherein the user is able to switch between the detected objects, to select the one which is displayed in the video tile.

15. A vision enhancement apparatus according to claim 1 , wherein the object is enlarged, or otherwise manipulated, to increase the user's perception of it.

16. A vision enhancement apparatus according to claim 1 , wherein the modified video data is, provided to a visual display, being vision spectacles, a retina, cortical or optic nerve implant.

17. A vision enhancement apparatus according to claim 1 , wherein the user selected object is the users hands.

18. A vision enhancement apparatus according to claim 17 , wherein the user selected object is in the users hands and the image processor operates to segment the object, to enable to user to interactively recognise it.

19. A vision enhancement apparatus according to claim 1 , wherein the user selected object includes text or an object with text on it.

20. A vision enhancement apparatus according to claim 19 , wherein the user is able to selectively switch between a still image of the text and a video image of the scene including the object.

21. A vision enhancement apparatus according to claim 1 , wherein the apparatus automatically obtains and present information to the user about the selected object.

22. A vision enhancement apparatus according to claim 1 , wherein the entire apparatus is distributed about a body of the user.

23. A method for enhancing vision, comprising the steps of:
processing high resolution video data representing images of a scene and displaying the resulting images to the user;
automatically detecting and tracking a user selected object in the images; and
automatically modifying the video data, by reserving a user selected area of the displayed images for displaying the tracked object as a separate video tile within the images of the scene to provide modified video data that includes:
the displayed images of the scene that includes the tracked object; and
the separate video tile, at the user selected area within the displayed images, to display the tracked object,

wherein the tracked object remains displayed at the separate video tile despite movement of the user or the tracked object.

24. A non-transitory computer readable medium that provides instructions, which when executed by a processor, cause the processor to perform the processing steps of claim 23 .

Assignees

• Nat Ict Australia Ltd

Inventors

• Barnes Nick
• Shen Chunhua

Classifications

• H04N7/002

  Special television systems not provided for by H04N7/007 - H04N7/18 (still pictures via a television channel H04N1/00098) · CPC title

• G09B21/008 Primary

  using visual presentation of the information for the partially sighted · CPC title

• A61F9/08

  Devices or methods enabling eye-patients to replace direct visual perception by another kind of perception · CPC title

• G06T11/60

  Creating or editing images; Combining images with text · CPC title

• A61N1/36046

  of the eye · CPC title