Wearable device, display control method, and computer-readable recording medium

Patent metadata

Field	Value
Publication number	US-10490101-B2
Application number	US-201715589309-A
Country	US
Kind code	B2
Filing date	May 8, 2017
Priority date	Nov 12, 2014
Publication date	Nov 26, 2019
Grant date	Nov 26, 2019

Abstract

A wearable device is provided that includes a microphone, a display, and a controller. The controller controls to identify a direction of emitted sound based on sound picked up by the microphone, and to display information corresponding to the sound at a position on the display corresponding to the identified direction of the emitted sound.

First claim

What is claimed is:

1. A wearable device comprising: a microphone; a display; and a processor configured to execute a process, the process comprising: analyzing sound picked up by the microphone; distinguishing an origin of the sound; determining if the origin of the sound corresponds to a voice of a wearer of the wearable device; if the origin of the sound corresponds to the voice of the wearer, selecting, based on a predetermined setting, between not displaying information corresponding to the voice of the wearer, or displaying information corresponding to the voice of the wearer in a different state than that of a voice other than the wearer's voice, and if the origin of the sound does not correspond to the voice of the wearer, identifying a direction of the sound based on the sound picked up by the microphone and displaying information corresponding to the sound at a position on the display corresponding to the identified direction of the sound; detecting a direction of gaze of the wearer of the wearable device; determining a position on the display that corresponds to a center in a field of view of the wearer of the wearable device according to the direction of detected gaze of the wearer of the wearable device; and correcting the position where the information is displayed such that closer the detected gaze is to the direction of sound, the closer the position where the information is displayed is to the center of the field of view.

2. The wearable device of claim 1 , wherein the display is a retinal display or a transmission type display.

3. The wearable device of claim 1 , wherein the gaze of the wearer is detected based on a potential difference surrounding an eyeball of the wearer of the wearable device.

4. The wearable device of claim 1 , wherein the process further comprises:
displaying an icon or text according to the distinguished origin.

5. The wearable device of claim 4 , wherein the origin of the sound is at least one of a person, a vehicle, thunder, or a chime.

6. The wearable device of claim 1 , wherein the process further comprises:
wirelessly transmitting information regarding the sound picked up by the microphone; and
displaying an icon or text according to a distinguished origin of the sound, based on emission-origin-distinguishing result information, received as a response to the wireless transmission.

7. The wearable device of claim 1 , wherein the process further comprises:
erasing the information displayed on the display or lowering a visibility of the information displayed on the display, based on a period of time elapsed since sound was picked up by the microphone.

8. The wearable device of claim 1 , wherein
if the origin of the sound corresponds to the voice of the wearer, either not displaying information corresponding to the voice of the wearer or displaying information corresponding to the voice of the wearer at a fixed position on the display, and
if the origin of the sound does not correspond to the voice of the wearer, identifying a direction of the sound and displaying information corresponding to the sound at a position on the display corresponding to the identified direction of the sound.

9. A display control method in which a computer executes processing comprising: by a processor: analyzing sound picked up by a microphone; distinguishing an origin of the sound; determining if the origin of the sound corresponds to a voice of a wearer of the wearable device; if the origin of the sound corresponds to the voice of the wearer, selecting, based on a predetermined setting, between not displaying information corresponding to the voice of the wearer, or displaying information corresponding to the voice of the wearer in a different state than that of a voice other than the wearer's voice, and if the origin of the sound does not correspond to the voice of the wearer, identifying a direction of the sound based on the sound picked up by a microphone and displaying information corresponding to the sound at a position on a display corresponding to the identified direction of the sound; detecting a direction of gaze of the wearer of a wearable device executing the processing; determining a position of the display that corresponds to a center in a field of view of the wearer of the wearable device according to the direction of detected gaze of the wearer of the wearable device; and correcting a position where the information is displayed such that closer the detected gaze is to the direction of sound, the closer the position where the information is displayed is to the center of the field of view.

10. The display control method of claim 9 , wherein:
the display is a retinal display or a transmission type display.

11. The display control method of claim 9 , wherein the gaze of the wearer of the device executing the processing is detected based on a potential difference surrounding an eyeball of the wearer.

12. The display control method of claim 9 , further comprising:
by the processor:
displaying an icon or text according to the distinguished origin.

13. The display control method of claim 12 , wherein the origin of the sound is at least one of a person, a vehicle, thunder, or a chime.

14. The display control method of claim 9 , further comprising:
by the processor:
wirelessly transmitting information regarding the sound picked up by the microphone; and
displaying an icon or text according to a distinguished origin of the sound based on emission-origin-distinguishing result information, received as a response to the wireless transmission.

15. The display control method of claim 9 , further comprising:
by the processor, erasing the information displayed on the display or lowering a visibility of the information displayed on the display based on a period of time elapsed since sound was picked up by the microphone.

16. A non-transitory computer-readable recording medium storing a display control program that causes a computer to execute a process, the process comprising: analyzing sound picked up by a microphone; distinguishing an origin of sound; determining if the origin of the sound corresponds to a voice of a wearer of the wearable device; if the origin of the sound corresponds to the voice of the wearer, selecting, based on a predetermined setting, between not displaying information corresponding to the voice of the wearer, or displaying information corresponding to the voice of the wearer in a different state than that of a voice other than the wearer's voice, and if the origin of the sound does not correspond to the voice of the wearer, identifying a direction of the sound based on the sound picked up by a microphone and displaying information corresponding to the sound at a position on a display corresponding to the identified direction of the sound; detecting a direction of gaze of the wearer of a wearable device executing the process; determining a position on the display that corresponds to a center in a field of view of the wearer of the wearable device according to the direction of detected gaze of the wearer of the wearable device; and correcting a position where the information is displayed such that closer the detected gaze is to the direction of sound, the closer the position where the information is displayed is to the center of the field of view.

17. The computer-readable recording medium of claim 16 , wherein:
the display is a retinal display or a transmission type display.

Assignees

• Fujitsu Ltd

Inventors

• Teshima Mamiko

Classifications

• G02B2027/014

  comprising information/image processing systems · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G02B27/0172

  characterised by optical features · CPC title

• G09B21/00

  Teaching, or communicating with, the blind, deaf or mute (audible presentation of material to be studied G09B5/04; devices or methods for replacing direct visual or auditory perception by another kind of perception A61F9/08, A61F11/04; audible indication of meter readings or of colour G01D7/12; watches for blind persons G04B25/02; speech analysis, speech recognition G10L; sound-recording or reproducing, per se G11B) · CPC title

• H04N5/44504

  Circuit details of the additional information generator, e.g. details of the character or graphics signal generator, overlay mixing circuits · CPC title