Interface for real-time audio recognition

Patent metadata

Field	Value
Publication number	US-9280599-B1
Application number	US-201213405023-A
Country	US
Kind code	B1
Filing date	Feb 24, 2012
Priority date	Feb 24, 2012
Publication date	Mar 8, 2016
Grant date	Mar 8, 2016

Abstract

An audio recognition service recognizes an audio sample across multiple content types. At least a partial set of results generated by the service are returned to a client while the audio sample is still being recorded and/or transmitted. The client additionally displays the results in real-time or near real-time to the user. The audio sample can be sent over a first HTTP connection and the results can be returned over a second HTTP connection. The audio recognition service further processes check-in selections received from the client for content items indicated by the results. Responsive to receiving the check-in selections, the service determines whether a user is eligible for a reward. If the user is eligible, the service provides the reward.

First claim

What is claimed is:

1. A computer-implemented method for providing audio recognition results via a user interface of a client device, the method comprising:
recording, at the client device, an audio sample;
transmitting, by the client device to a server, at least a portion of the recorded audio sample including a first window of the audio sample and, subsequent to the first window, a second window of the audio sample, wherein the transmitted portion of the audio sample is less than an entirety of the recorded audio sample;
receiving, at the client device from the server and while transmitting the audio sample to the server, a first recognition result in response to the server analyzing the first window of the transmitted audio sample, and a second recognition result in response to the server analyzing the first window and the second window of the transmitted audio sample; and
providing for display, in a user interface of the client device, the first and the second recognition results prior to the client device transmitting the entirety of the audio sample to the server, wherein the audio sample contains audio associated with multiple content types.

2. The computer-implemented method of claim 1 , wherein the first recognition result includes recognizing the recorded audio from a first content type and the second recognition result includes recognizing the recorded audio from a second content type, the first content type being different from the second content type.

3. The computer-implemented method of claim 2 , wherein the first content type is a music type and the second content type is a video type.

4. The computer-implemented method of claim 2 , wherein the first recognition result includes information specific to the first content type and the second recognition result includes information specific to the second content type.

5. The computer-implemented method of claim 2 , wherein the first recognition result includes a first visual element representative of the first content type, and the second recognition result includes a second visual element representative of the second content type.

6. The computer-implemented method of claim 2 , wherein first and second recognition results are ordered based on a user defined priority.

7. The computer-implemented method of claim 6 , wherein the user defined priority indicates that the first and second recognition results are to be ordered based on content type.

8. The computer-implemented method of claim 1 , wherein the first recognition result includes purchase information for an item.

9. The computer-implemented method of claim 1 , wherein the first recognition result includes at least one of the following types of information: identification information for a content item associated with the audio sample, non-identification information relating to the content item, or social information relating to the content item.

10. The computer-implemented method of claim 1 , further comprising:
providing for display, in the user interface, a check-in control for checking-in to the first recognition result.

11. A computer system configured to provide audio recognition results, the computer system comprising a non-transitory computer-readable medium encoding instructions thereon that, in response to execution by the computer system, configure the computer system to:
record, at the computer system, an audio sample;
transmit, by the computer system to a server, at least a portion of the recorded audio sample including a first window of the audio sample and, subsequent to the first window, a second window of the audio sample, wherein the transmitted portion of the audio sample is less than an entirety of the recorded audio sample;
receive, at the computer system from the server and while transmitting the audio sample to the server, a first recognition result in response to the server analyzing the first window of the transmitted audio sample, and a second recognition result in response to the server analyzing the first window and the second window of the transmitted audio sample; and
provide for display, in a user interface of the computer system, the first and the second, recognition results prior to the computer system transmitting the entirety of the audio sample to the server, wherein the audio sample contains audio associated with multiple content types.

12. A computer-implemented method for providing audio recognition results via a user interface of a client device, the method comprising:
recording, at a client device, an audio sample;
sending, at the client device to a server, a window of the audio sample, the window comprising less than the entirety of the recorded audio sample;
receiving, at the client device from the server and while sending the window of audio sample to the server, a set of recognition results corresponding to the window of the audio sample, the set of recognition results including a first subset of recognition results provided at a first time and having a first content type and a second subset of recognition results provided at a second time, subsequent to the first time, and having a second content type the second subset of recognition results based on the first subset of recognition results; and
providing for display, in a user interface of the client device, of the set of recognition results.

13. The computer-implemented method of claim 12 , wherein the set of recognition results are received prior to the client device sending the entirety of the audio sample to the server.

14. The computer-implemented method of claim 12 , wherein providing for display of the set of recognition results includes grouping the set of recognition results by content type.

15. The computer-implemented method of claim 12 , further comprising:
providing for display, in the user interface, a query stop control; and
responsive to a user selection of the query stop control, terminating the sending of the window of the audio sample to the server.

16. The computer-implemented method of claim 12 , wherein the first content type is a music type and the second content type is a video type.

Assignees

• Wiseman Matthew
• Roblek Dominik
• Bernstein Yaniv
• Sharifi Matthew
• Chen Annie
• Jones Tim C
• Graff Ehud
• Google Inc

Inventors

• Wiseman Matthew
• Roblek Dominik
• Bernstein Yaniv
• Sharifi Matthew
• Chen Annie
• Jones Tim C
• Graff Ehud

Classifications

• G06Q30/0601

  Electronic shopping [e-shopping] · CPC title

• G06F17/30758 Primary

  Physics · mapped topic

• G06F16/632

  Query formulation · CPC title

• G10L15/26

  Speech to text systems (G10L15/08 takes precedence) · CPC title

• G06F16/638

  Presentation of query results · CPC title