Image processing apparatus, image processing method, and program

Patent metadata

Field	Value
Publication number	US-9292974-B2
Application number	US-201213824140-A
Country	US
Kind code	B2
Filing date	Sep 4, 2012
Priority date	Oct 27, 2011
Publication date	Mar 22, 2016
Grant date	Mar 22, 2016

Abstract

An information processing system that acquires video data captured by an image pickup unit; detects an object from the video data; detects a condition corresponding to the image pickup unit; and controls a display to display content associated with the object at a position other than a detected position of the object based on the condition corresponding to the image pickup unit.

First claim

The invention claimed is:

1. An information processing system comprising:
an image pickup circuit configured to acquire video data;
a storage configured to store content; and
a processor configured to
detect an object from the video data;
detect a condition corresponding to the image pickup circuit;
retrieve content associated with the object based on the object detected from the video data; and
control a display to display the content associated with the object at a position other than a detected position of the object based on the condition corresponding to the image pickup circuit, the controlling including controlling the display to display the content associated with the object when the object is detected in the acquired video data, and controlling the display to continue displaying the content associated with the object when the object is no longer detected or when the object is partially observable in the acquired video data.

2. The information processing system of claim 1 , wherein
the object detected from the video data is at least one of a shape, symbol, character string, design, object, part of an object and an image displayed on a display.

3. The information processing system of claim 1 , wherein
detecting a condition corresponding to the image pickup circuit includes detecting at least one of a position and a posture of the image pickup circuit.

4. The information processing system of claim 3 , wherein the processor is further configured to
detect at least one of a position and a posture of the detected object.

5. The information processing system of claim 4 , wherein the processor is further configured to
control the position on the display at which the content associated with the object is displayed based on the at least one of the detected position and the detected posture of the image pickup circuit and the at least one of the detected position and the detected a posture of the detected object.

6. The information processing system of claim 1 , further comprising:
a sensor that detects the condition corresponding to the image pickup circuit.

7. The information processing system of claim 6 , wherein
the sensor includes at least one of a positioning sensor, an acceleration sensor and a gyrosensor.

8. The information processing system of claim 1 , further comprising:
a memory that stores information corresponding to at least one object in association with information corresponding to the content, wherein
the information corresponding to the content includes at least one of a marker identifier, a related content identifier and a size corresponding to the content to be displayed.

9. The information processing system of claim 1 , further comprising:
a memory that stores information corresponding to the detection of the object, wherein
the information corresponding to the detection of the object includes at least one of an object identifier, an object position, an object posture, an object detection time and a time corresponding to when the object was no longer detected in the acquired video data.

10. The information processing system of claim 9 , wherein the processor is further configured to
control the position on the display at which the content associated with the object is displayed based on the condition corresponding to the image pickup circuit and the information corresponding to the detection of the object.

11. The information processing system of claim 1 , wherein the processor is further configured to
control the display to move the content associated with the object on the display based on a change in at least one of a position and a posture of the image pickup circuit.

12. The information processing system of claim 1 , wherein
the detected condition corresponding to the image pickup circuit includes at least one of a position and a posture of the image pickup circuit, and
the processor is further configured to control the display to stop displaying the content associated with the object when at least one of the position and the posture of the image pickup circuit satisfies a predetermined condition.

13. The information processing system of claim 12 , wherein the processor is further configured to
control the display to display an indication that the display is to stop displaying the content associated with the object when at least one of the position and the posture of the image pickup circuit satisfies the predetermined condition.

14. The information processing system of claim 12 , wherein
the predetermined condition corresponds to a distance between the image pickup circuit and the detected object, and
the processor is further configured to control the display to stop displaying the content associated with the object when the distance between image pickup circuit and the detected object exceeds a predetermined threshold value.

15. The information processing system of claim 12 , wherein
the predetermined condition corresponds to a difference in an angle between an optical axis of the image pickup circuit and an axis extending from the image pickup circuit to the detected object, and
the processor is further configured to control the display to stop displaying the content associated with the object when the difference in angle exceeds a predetermined threshold value.

16. The information processing system of claim 12 , wherein
the predetermined condition corresponds to a time elapsed from when the object was first detected in the acquired video data, and
the processor is further configured to control the display to stop displaying the content associated with the object when time elapsed exceeds a predetermined threshold value.

17. The information processing system of claim 12 , wherein
the predetermined condition corresponds to a time elapsed from when the object was last detected in the acquired video data, and
the processor is further configured to control the display to stop displaying the content associated with the object when time elapsed exceeds a predetermined threshold value.

18. The information processing system of claim 1 , wherein the processor is configured to control the display to start displaying the content associated with the object when the object is detected in the acquired video data and when a distance between the object and the image pickup circuit is less than a predetermined threshold value.

19. An information processing method performed by an information processing system, the method comprising:
acquiring video data captured by an image pickup unit;
detecting an object from the video data;
detecting a condition corresponding to the image pickup unit;
retrieving content associated with the object based on the object detected from the video data; and
controlling a display to display the content associated with the object at a position other than a detected position of the object based on the condition corresponding to the image pickup unit, the controlling including controlling the display to display the content associated with the object when the object is detected in the acquired video data, and controlling the display to continue displaying the content associated with the object when the object is no longer detected or when the object is partially observable in the acquired video data.

20. The information processing method of claim 19 , wherein the controlling includes controlling the display to start displaying the content associated with the object when the object is detected in the acquired video data and

Assignees

• Kaino Akihiko
• Fukuchi Masaki
• Kashitani Tatsuki
• Ooi Kenichiro
• Guo Jingjing
• Sony Corp

Inventors

• Kaino Akihiko
• Fukuchi Masaki
• Kashitani Tatsuki
• Ooi Kenichiro
• Guo Jingjing

Classifications

• G06T2207/30244

  Camera pose · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06T2219/004

  Annotating, labelling · CPC title

• G06T7/70

  Determining position or orientation of objects or cameras (camera calibration G06T7/80) · CPC title

• G06T7/73

  using feature-based methods · CPC title