Information processing to notify potential source of interest to user

Patent metadata

Field	Value
Publication number	US-10909759-B2
Application number	US-201113980978-A
Country	US
Kind code	B2
Filing date	Dec 19, 2011
Priority date	Jan 28, 2011
Publication date	Feb 2, 2021
Grant date	Feb 2, 2021

Abstract

An apparatus comprising a memory storing instructions is provided. The apparatus includes a control unit for executing the instructions to send signals to display, for a user, a first virtual image superimposed onto a representation of real space. The control unit further executes instructions to send signals to identify a potential source of interest for the user, the potential source of interest being outside of a user focus area in the representation of real space. The control unit further executes instructions to send signals to notify the user of the potential source of interest.

First claim

The invention claimed is:

1. An apparatus, comprising:
a memory configured to store instructions;
an imaging unit configured to image a real space to generate a representation of the real space;
a display unit; and
a Central Processing Unit (CPU) configured to:
transmit a first plurality of signals to display a first virtual image superimposed onto the representation of the real space, wherein the superimposed first virtual image is displayed on the display unit;
transmit a second plurality of signals to identify a potential source of interest for a user, wherein the potential source of interest is at least one of a real object or a region outside an angle of view of the imaging unit;
determine the potential source of interest based on an approach speed of the apparatus towards the potential source of interest that exceeds a threshold speed;
determine that the potential source of interest is the same as a previously notified potential source of interest based on a position of the potential source of interest and a user action history, wherein the user action history comprises a history of user movement;
increment a number of times that a user notification is generated for the potential source of interest, wherein the increment of the number of times is based on the determination that the potential source of interest is the same as the previously notified potential source of interest;
compare the incremented number of times that the user notification is generated for the potential source of interest with a threshold upper limit;
control, based on the determined potential source of interest, the display unit to display the superimposed first virtual image as a translucent image; and
control notification of the determined potential source of interest based on the comparison.

2. The apparatus of claim 1 , wherein the CPU is further configured to analyze input signals to create the representation of the real space.

3. The apparatus of claim 1 , wherein the potential source of interest corresponds to a potential source of danger for the user.

4. The apparatus of claim 1 , wherein the user notification includes a first object indicative of the position of the determined potential source of interest.

5. The apparatus of claim 1 , wherein the CPU is further configured to control the notification of the determined potential source of interest based on the display of the superimposed first virtual image as the translucent image.

6. The apparatus of claim 1 , wherein the CPU is further configured to transmit a third plurality of signals to generate at least one of an audio alarm, a tactile alarm, or a visual alarm.

7. The apparatus of claim 6 , wherein the visual alarm comprises a second virtual image.

8. The apparatus of claim 1 , wherein the CPU is further configured to:
transmit a third plurality of signals to monitor a user action; and
identify the potential source of interest based on the monitored user action.

9. The apparatus of claim 8 , wherein the CPU is further configured to determine that the user notification about the potential source of interest is based on the monitored user action.

10. The apparatus of claim 8 , wherein the monitored user action comprises a series of actions.

11. The apparatus of claim 10 , wherein the CPU is further configured to determine that the user notification is based on a comparison of a number of actions in the series of actions with a threshold number of user actions.

12. The apparatus of claim 10 , wherein the CPU is further configured to determine that the series of actions indicate that the user has one of avoided the potential source of interest or approached the potential source of interest.

13. The apparatus of claim 8 , wherein the monitored user action is a reaction to a type of the potential source of interest.

14. The apparatus of claim 1 , wherein the CPU is further configured to identify the potential source of interest based on at least one of:
an analysis of the representation of the real space,
reception of a first signal that indicates a distance between the apparatus and the potential source of interest, or
reception of a second signal from an external device.

15. The apparatus of claim 14 , wherein the analysis comprises a search of the representation of the real space for the potential source of interest.

16. The apparatus of claim 1 , wherein
the apparatus is a user device, and
the CPU is further configured to transmit the second plurality of signals to a remote server for identification of the potential source of interest.

17. The apparatus of claim 1 , wherein
the apparatus is a server, and
the CPU is further configured to transmit the second plurality of signals to a user device for identification of the potential source of interest.

18. The apparatus of claim 1 , wherein the CPU is further configured to determine that the user has approached the region based on a comparison of a distance, between a boundary of the region and a current user position, with a threshold value.

19. The apparatus of claim 1 , wherein the CPU is further configured to:
receive a beacon from the potential source of interest, wherein the beacon comprises an identifier of the potential source of interest, and position data of the potential source of interest; and
determine the potential source of interest based on the position data in the beacon and a current user position.

20. The apparatus of claim 1 , wherein the CPU is further configured to determine the potential source of interest based on a variation of a level of one of a floor or a ground.

21. A method, comprising:
in an apparatus:
imaging a real space by an imaging device;
generating a representation of the real space based on the imaging of the real space;
displaying a virtual image superimposed onto the representation of the real space, wherein the superimposed virtual image is displayed on a display unit;
determining a potential source of interest for a user based on an approach speed of the apparatus towards the potential source of interest that exceeds a threshold speed, wherein the potential source of interest is at least one of a real object or a region outside an angle of view of the imaging device;
determining that the potential source of interest is the same as a previously notified potential source of interest based on a position of the potential source of interest and a user action history, wherein the user action history comprises a history of user movement;
incrementing a number of times that a user notification is generated for the potential source of interest, wherein the increment of the number of times is based on the determination that the potential source of interest is the same as the previously notified potential source of interest;
comparing the incremented number of times that the user notification is generated for the potential source of interest with a threshold upper limit;
controlling, based on the determined potential source of interest, the display unit to display the superimposed virtual image as a translucent image; and
controlling notification of the determined potential source of interest based on the comparison.

22. A non-transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by a processor of an apparatus, cause the processor to execute operations, the operations comprising:
generating a representation of a real space by imaging the real space;
disp

Assignees

• Kaino Akihiko
• Iwai Yoshiaki
• Oi Kenichiro
• Homma Shunichi
• Wu Jianing
• Sony Corp

Inventors

• Kaino Akihiko
• Iwai Yoshiaki
• Oi Kenichiro
• Homma Shunichi
• Wu Jianing

Classifications

• G08G1/005 Primary

  including pedestrian guidance indicator · CPC title

• G06T11/00

  Two-dimensional [2D] image generation · CPC title

• G06F3/14

  Digital output to display device {; Cooperation and interconnection of the display device with other functional units} · CPC title

• G06F3/005 Primary

  Input arrangements through a video camera · CPC title

• G06F3/048

  Interaction techniques based on graphical user interfaces [GUI] · CPC title