Environmental reproduction system for representing an environment using one or more environmental sensors

Patent metadata

Field	Value
Publication number	US-9626558-B2
Application number	US-201414568943-A
Country	US
Kind code	B2
Filing date	Dec 12, 2014
Priority date	Dec 22, 2010
Publication date	Apr 18, 2017
Grant date	Apr 18, 2017

Abstract

A environmental reproduction system multiplexes information from a variety of sources to reproduce a target environment having visual and/or auditory impediments. The visual impediments may include one or more environmental conditions, such as rain, sleet, snow, darkness, brightness, or any other type of environmental condition. The auditory impediments may include loud noises, such as construction noises, or sounds that are too low, such as an emergency vehicle sound not being heard. The environmental reproduction system may determine the environmental condition from one or more sources, such as an internal microphone, an external microphone, temperature sensor, a camera, a weather receiver, or other types of sensors. The environmental reproduction may be in communication with an environmental condition database and operative to apply a signal processing action to a recording (video and/or audio recording) of the target environment based on the determined environmental condition.

First claim

We claim:

1. A computer-implemented method comprising:
receiving a first image of an object;
determining a geographic location associated with the first image or the object;
determining to enhance an appearance of the object in the first image;
in response to determining to enhance the appearance of the object in the first image, identifying a second image based at least on the geographic location;
generating an enhanced image of the object based at least on the first image and the second image, wherein generating an enhanced image of the object based at least on the first image and the second image comprises combining the first image and the second image by overlaying a portion of the second image that includes at least a portion of the object over a portion of the first image that includes at least a portion of the object, and wherein the enhanced image overcomes at least one impediment to viewing the object; and
providing the enhanced image for display.

2. The method of claim 1 , wherein determining a geographic location associated with the first image or the object comprises:
obtaining geographic coordinates indicating where the first image of the object was taken.

3. The method of claim 1 , wherein determining to enhance an appearance of the object in the first image comprises:
identifying the object in the first image; and
in response to identifying the object in the first image, determining to enhance the appearance of the object in the first image.

4. The method of claim 1 , wherein determining to enhance an appearance of the object in the first image comprises:
determining that the object is likely to be obscured in the first image; and
in response to determining that the object is likely to be obscured in the first image, determining to enhance the appearance of the object in the first image.

5. The method of claim 1 , wherein in response to determining to enhance the appearance of the object in the first image, identifying a second image based at least on the geographic location, comprises:
identifying the second image as an image stored in association with the geographic location.

6. A system comprising:
one or more computers and one or more storage devices storing instructions that are operable, when executed by the one or more computers, to cause the one or more computers to perform operations comprising:
receiving a first image of an object;
determining a geographic location associated with the first image or the object;
determining to enhance an appearance of the object in the first image;
in response to determining to enhance the appearance of the object in the first image, identifying a second image based at least on the geographic location;
generating an enhanced image of the object based at least on the first image and the second image wherein generating an enhanced image of the object based at least on the first image and the second image comprises combining the first image and the second image by overlaying a portion of the second image that includes at least a portion of the object over a portion of the first image that includes at least a portion of the object, and wherein the enhanced image overcomes at least one impediment to viewing the object; and
providing the enhanced image for display.

7. The system of claim 6 , wherein determining a geographic location associated with the first image or the object comprises:
obtaining geographic coordinates indicating where the first image of the object was taken.

8. The system of claim 6 , wherein determining to enhance an appearance of the object in the first image comprises:
identifying the object in the first image; and
in response to identifying the object in the first image, determining to enhance the appearance of the object in the first image.

9. The system of claim 6 , wherein determining to enhance an appearance of the object in the first image comprises:
determining that the object is likely to be obscured in the first image; and
in response to determining that the object is likely to be obscured in the first image, determining to enhance the appearance of the object in the first image.

10. The system of claim 6 , wherein in response to determining to enhance the appearance of the object in the first image, identifying a second image based at least on the geographic location, comprises:
identifying the second image as an image stored in association with the geographic location.

11. A non-transitory computer-readable medium storing software comprising instructions executable by one or more computers which, upon such execution, cause the one or more computers to perform operations comprising:
receiving a first image of an object;
determining a geographic location associated with the first image or the object;
determining to enhance an appearance of the object in the first image;
in response to determining to enhance the appearance of the object in the first image, identifying a second image based at least on the geographic location;
generating an enhanced image of the object based at least on the first image and the second image wherein generating an enhanced image of the object based at least on the first image and the second image comprises combining the first image and the second image by overlaying a portion of the second image that includes at least a portion of the object over a portion of the first image that includes at least a portion of the object, and wherein the enhanced image overcomes at least one impediment to viewing the object; and
providing the enhanced image for display.

12. The medium of claim 11 , wherein determining a geographic location associated with the first image or the object comprises:
obtaining geographic coordinates indicating where the first image of the object was taken.

13. The medium of claim 11 , wherein determining to enhance an appearance of the object in the first image comprises:
identifying the object in the first image; and
in response to identifying the object in the first image, determining to enhance the appearance of the object in the first image.

14. The medium of claim 11 , wherein determining to enhance an appearance of the object in the first image comprises:
determining that the object is likely to be obscured in the first image; and
in response to determining that the object is likely to be obscured in the first image, determining to enhance the appearance of the object in the first image.

15. The medium of claim 11 , wherein in response to determining to enhance the appearance of the object in the first image, identifying a second image based at least on the geographic location, comprises:
identifying the second image as an image stored in association with the geographic location.

16. A computer-implemented method comprising:
receiving a first image of an object;
determining a geographic location associated with the object;
determining to enhance an appearance of the object in the first image;
in response to determining to enhance the appearance of the object in the first image, identifying the same object in a second image based at least on the geographic location;
generating an enhanced image of the object based at least on the first image and the second image so as to display environmental information, wherein the environmental information was obscured by at least one impediment prior to enhancing the image, and wherein the enhanced image overcomes the at least one impediment to viewing the object; and
providing the enhanced image for display.

Assignees

• Google Inc

Inventors

• Ho Casey Kwok Ching
• Nanavati Sharvil A

Classifications

• G06V30/422

  Technical drawings; Geographical maps · CPC title

• G06T5/00

  Image enhancement or restoration · CPC title

• G06K9/00476 Primary

  Physics · mapped topic

• G09B21/006

  using audible presentation of the information · CPC title

• G06T5/50 Primary

  using two or more images, e.g. averaging or subtraction · CPC title