Smartphone-based methods and systems

Patent metadata

Field	Value
Publication number	US-9354778-B2
Application number	US-201414452239-A
Country	US
Kind code	B2
Filing date	Aug 5, 2014
Priority date	Dec 6, 2013
Publication date	May 31, 2016
Grant date	May 31, 2016

Abstract

The present disclosure generally concerns sensor-equipped consumer electronic devices, such as smartphones and tablet computers. One implementation is directed toward a smartphone configured to operate in an image discovery mode. Results from the image discovery mode may be displayed on a smartphones display screen. The smartphone may concurrently operate an audio discovery mode. Results from the audio discovery mode are added to a discovery history, but are not displayed on the smartphone's display screen while operating in the image discovery mode. Of course, many other combinations and implementations are provided as well.

First claim

What is claimed is:

1. A method utilizing a portable device comprising a camera, a microphone, a movement sensor and a touchscreen display, said method comprising the acts of:
receiving image data captured by the camera of the portable device;
communicating data associated with the image data to a remotely located server;
in response to said communicating, receiving identifying information associated with an object depicted in the image data, the identifying information comprising an image or graphic;
causing the touchscreen display to simultaneously display:
i) a first tile comprising an image or graphic, the first tile associated with a first image-recognized object, and a second tile comprising an image or graphic, the second tile associated with a second image-recognized object; and
ii) live image data captured by the camera while the portable device remains in an image discovery mode, in which the image discovery mode operates to identify objects depicted in captured image data; and

operating the portable device in an image discovery mode rather than an audio discovery mode based on data obtained from the movement sensor, in which the audio discovery mode operates to identify audio captured with the microphone.

2. The method of claim 1 in which the movement sensor comprises an accelerometer or gyroscope.

3. The method of claim 1 in which the first tile, second tile and live image data are displayed on the same plane of the touchscreen display.

4. The method of claim 1 in which, in response to input received through the touchscreen display, displaying a third tile in a stacked arrangement with the first tile and the second tile.

5. The method of claim 4 further comprising, causing the stacked arrangement to grow in tiles until a predetermined height is reached, and then tiles resulting from content discovery cause a bottom tile in the stacked arrangement to sink out of sight on the touchscreen display.

6. The method of claim 1 in which the live image data is displayed in a viewfinder window.

7. The method of claim 6 in which upon the viewfinder window being occluded, said method further comprises deactivating the camera.

8. The method of claim 6 further comprising causing the portable device to vibrate to confirm recognition of an object depicted in the viewfinder window.

9. A smartphone comprising:
a touchscreen display;
a movement sensor;
a camera for capturing image data;
a microphone for capturing audio data;
memory for storing software instructions;
one or more processors programmed with said software instructions for:
processing image data captured by said camera;
communicating data associated with the processed image data to a remotely located server;
in response to said communicating, receiving identifying information associated with an object depicted in the image data, the identifying information comprising an image or graphic, the identifying information associated with fingerprint data extracted from data representing the image data;
causing the touchscreen display to simultaneously display:
i) a first tile including an image or graphic, the first tile associated with a first image-recognized object, and a second tile including an image or graphic, the second tile associated with a second image-recognized object; and
ii) a viewfinder area for displaying live image data captured by said camera while the smartphone remains in an image discovery mode, in which the image discovery mode operates to identify objects depicted in captured image data from fingerprint analysis; and

upon the viewfinder area being occluded, deactivating said camera.

10. The smartphone of claim 9 in which said one or more processors are programmed for making an inference from data obtained from said movement sensor of a preference for operating said smartphone in the image discovery mode rather than an audio discovery mode, in which the audio discovery mode operates to identify audio captured with said microphone.

11. The smartphone of claim 10 in which said movement sensor comprises an accelerometer or gyroscope.

12. The smartphone of claim 9 in which the first tile, second tile and viewfinder area are displayed on the same plane of said touchscreen display.

13. The smartphone of claim 9 in which said one or more processors are programmed for, in response to input received through said touchscreen display, displaying a third tile in a stacked arrangement with the first tile and the second tile.

14. The smartphone of claim 13 in which said one or more processors are programmed for causing the stacked arrangement to grow in tiles until a predetermined height is reached, and then tiles resulting from further content discovery cause a bottom tile in the stack to sink out of sight on the touchscreen display.

15. The smartphone of claim 9 in which said one or more processors are programmed for controlling a vibration to confirm recognition of an object depicted in the viewfinder area.

16. The smartphone of claim 9 in which said one or more processors programmed with said software instructions include programming for: determining whether to disable an audio discovery mode based on information from said movement sensor, in which the audio discovery mode operates to identify audio captured with said microphone.

17. The smartphone of claim 9 in which said one or more processors programmed with said software instructions include programming for: switching between an audio discovery and the image discovery mode based on information obtained from said movement sensor, in which the audio discovery mode operates to identify audio captured with said microphone.

18. A system comprising:
A. a smartphone comprising:
a touchscreen display;
a camera for capturing image data;
a microphone for capturing audio data;
one or more processors configured for:
communicating data associated with captured image data to a remotely located server;
in response to said communicating, receiving payoff information associated with an object depicted in the image data, the payoff information comprising an image or graphic;
causing the touchscreen display to simultaneously display:
i) stacked tiles comprising a plurality of tiles including a first tile including an image or graphic, the first tile associated with a first image-recognized object, and a second tile including an image or graphic, the second tile associated with a second image-recognized object; and
ii) a viewfinder window where live image data captured by said camera is displayed while said smartphone remains in an image discovery mode, in which the image discovery mode operates to identify objects depicted in captured image data; and

while operating in the image discovery mode, concurrently operating an audio discovery mode, and including a third tile obtained from the audio discovery mode in a tile history but not in the stacked tiles, in which the audio discovery mode operates to identify audio captured with said microphone;

B. said remotely located server comprising one or more processors configured for:
processing data associated with remotely captured imagery to extract identifying information therefrom;
indexing the identifying information in a data repository to find payoff information associated with an object depicted in the captured imagery; and
control communication of the payoff information to said smartphone.

19. The system of claim 18 in which said smartphone further comprises a movement sensor, and in which said smartphone's one or more processo

Assignees

• Digimarc Corp

Inventors

• Cornaby Colin P
• Weaver Matthew M

Classifications

• G06F3/04886

  by partitioning the display area of the touch-screen or the surface of the digitising tablet into independently controllable areas, e.g. virtual keyboards or menus · CPC title

• G06F3/04842

  Selection of displayed objects or displayed text elements (G06F3/0482 takes precedence) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F1/1694

  the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer · CPC title

• G06F3/0237

  using prediction or retrieval techniques · CPC title