Determining regions of interest based on user interaction

Patent metadata

Field	Value
Publication number	US-9842268-B1
Application number	US-201615079519-A
Country	US
Kind code	B1
Filing date	Mar 24, 2016
Priority date	Mar 27, 2015
Publication date	Dec 12, 2017
Grant date	Dec 12, 2017

Abstract

A system and method provide for determining regions of interest within an image based on viewer interaction with the image. At least one image associated with a location is provided for display in a viewport, and pose data related to user interaction with the at least one image is identified. Weights are assigned to portions of the at least one image based on the pose data, the weights indicating at least a period of time the portion of the at least one image is generally at a center of the viewport. Based on the assigned weights, image regions of interest of the at least one image are determined.

First claim

The invention claimed is:

1. A computer-implemented method, comprising:
providing, using one or more computing devices, at least one image associated with a location for display in a viewport;
identifying, using the one or more computing devices, pose data related to user interaction with the at least one image;
assigning, using the one or more computing devices, weights to portions of the at least one image based on the pose data, the weights indicating at least a period of time the portion of the at least one image is generally at a center of the viewport;
determining, using the one or more computing devices, image regions of interest of the at least one image based on the assigned weights;
generating, using the one or more processors, heat map data for the location based on the assigned weights for the at least one image, the heat map data indicating one or more areas of interest at the location; and
overlaying the heat map data onto a 3D panoramic view of the location;
wherein portions with higher assigned weights are at a central portion of the viewport for a longer period of time than portions with lower assigned weights.

2. The method of claim 1 , further comprising orienting the at least one image in the viewport based on the determined image regions of interest.

3. The method of claim 2 , wherein orienting the at least one image comprises adjusting at least one of a zoom level, a pan, and a point of view from which the image is captured.

4. The method of claim 1 , wherein the heat map data includes one or more heat spot indicators, each heat spot indicator corresponding to the weight assigned to a given area of interest at the location.

5. The method of claim 1 , further comprising:
determining whether the view port is zoomed in on a portion of a given set of images based on the identified pose data; and
assigning a weight to the zoomed in portion of the given set of images according to the determination.

6. The method of claim 1 , further comprising:
determining a number of times a portion of a given set of images is at the central portion of the view port based on the identified pose data; and
assigning a weight to the portion of the given set of images according to the determination.

7. A system, comprising:
a memory storing at least one image corresponding to a geographic location; one or more processors in communication with the memory, the one or more processors configured to:
provide at least one image associated with a location for display in a viewport; identify pose data related to user interaction with the at least one image; assign weights to portions of the at least one image based on the pose data, the weights indicating at least a period of time the portion of the at least one image is generally at a center of the viewport; and
determine image regions of interest of the at least one image based on the assigned weights;
generate heat map data for the location based on the assigned weights for the at least one image, the heat map data indicating one or more areas of interest at the location;
and overlay the heat map data onto a 3D panoramic view of the location;
wherein portions with higher assigned weights are at a central portion of the viewport for a longer period of time than portions with lower assigned weights.

8. The system of claim 7 , wherein the one or more processors are further configured to orient the at least one image in the viewport based on the determined image regions of interest.

9. The system of claim 8 , wherein orienting the at least one image comprises adjusting at least one of a zoom level, a pan, and a point of view from which the image is captured.

10. The system of claim 7 , wherein the heat map data includes one or more heat spot indicators, each heat spot indicator corresponding to the weight assigned to a given area of interest at the location.

11. The system of claim 7 , wherein the one or more processors are further configured to:
determine whether the viewport is zoomed in on a portion of a given set of images based on the identified pose data; and
assign a weight to the zoomed in portion of the given set of images according to the determination.

12. The system of claim 7 , wherein the one or more processors are further configured to:
determine a number of times a portion of a given set of images is at the central portion of the view port based on the identified pose data; and
assign a weight to the portion of the given set of images according to the determination.

13. A non-transitory computer readable medium storing instruction executable by one or more processors for performing a method, comprising:
providing at least one image associated with a location for display in a viewport;
identifying pose data related to user interaction with the at least one image;
assigning weights to portions of the at least one image based on the pose data, the weights indicating at least a period of time the portion of the at least one image is generally at a center of the viewport; and
determining image regions of interest of the at least one image based on the assigned weights;
generating heat map data for the location based on the assigned weights for the at least one image, the heat map data indicating one or more areas of interest at the location;
and overlaying the heat map data onto a 3D panoramic view of the location;
wherein portions with higher assigned weights are at a central portion of the viewport for a longer period of time than portions with lower assigned weights.

14. The non-transitory computer readable medium of claim 13 , the method further comprising orienting the at least one image in the viewport based on the determined image regions of interest.

15. The non-transitory computer readable medium of claim 14 , wherein orienting the at least one image comprises adjusting at least one of a zoom level, a pan, and a point of view from which the image is captured.

Assignees

• Google Inc
• Google Llc

Inventors

• Krafka Kyle John
• Sheridan Alan

Classifications

• G06F3/04815 Primary

  Interaction with a metaphor-based environment or interaction object displayed as three-dimensional [3D], e.g. changing the user viewpoint with respect to the environment or object · CPC title

• G06V10/235

  based on user input or interaction · CPC title

• G06T11/26

  Drawing of charts or graphs · CPC title

• G06V10/25

  Determination of region of interest [ROI] or a volume of interest [VOI] · CPC title

• G06V10/242

  by image rotation, e.g. by 90 degrees · CPC title