Methods and systems for searching utilizing acoustical context

Patent metadata

Field	Value
Publication number	US-10409860-B2
Application number	US-201214007482-A
Country	US
Kind code	B2
Filing date	Mar 28, 2012
Priority date	Mar 28, 2011
Publication date	Sep 10, 2019
Grant date	Sep 10, 2019

Abstract

Systems, controllers and methods for contextual-based searching are provided. A system includes one or more devices configured to collect at least one audio stream and a contextual search system. The contextual search system includes a data analyzer and a search engine. The data analyzer is configured to receive the at least one audio stream from among the one or more devices and to determine contextual information from the received at least one audio stream. The search engine is configured to perform a search of at least one search term using the contextual information, to produce a search result.

First claim

What is claimed:

1. A system for contextual-based searching comprising:
one or more devices configured to collect at least one audio stream;
one or more sensors for acquiring sensor data related to an environment proximate to one or more of the devices, wherein the one or more sensors are integral or operatively coupled to the one or more devices and include a temperature sensor, a biometric sensor, an image sensor, a light detection sensor, a proximity sensor, a gravity detection sensor, a magnetic field detection sensor, an electrical field detection sensor, a vibration sensor, a pressure senor, a humidity sensor, a moisture sensor, a toxin detection senor, a nutrient detection sensor, an eye-tracker, a pheromone detection sensor, or a combination thereof; and
a contextual search system including:
a data analyzer configured to receive the at least one audio stream and the sensor data from among the one or more devices and to determine contextual information from the received at least one audio stream and the sensor data;
a classifier that transcribes the at least one audio stream and detects an event in the at least one transcribed audio stream;
a search engine configured to perform a search of at least one search term using the contextual information and the detected event to produce a search result,
wherein, in response to the search result, a sensor module is activated and at least one further audio stream is collected by the one or more devices by utilizing the sensor module; and
wherein the contextual search system determines an effectiveness of the search result based on the at least one further audio stream.

2. The system according to claim 1 , wherein the system provides the search result to at least one device among the one or more devices.

3. The system according to claim 1 , wherein the contextual information indicates a search intent of the search term.

4. The system according to claim 1 , wherein the classifier classifies the at least one audio stream into classified information, the data analyzer using the classified information and the at least one search term to determine the contextual information to produce the search result.

5. The system according to claim 1 , wherein each audio stream of the at least one audio stream provides a key word, a speech event, a non-speech event, a topic, a transcription, a speaker identity, an emotional state associated with each audio stream, or a combination thereof.

6. The system according to claim 1 , wherein the search result is produced from contextual information obtained from at least one or more sensors proximate to each other or from at least one or more sensors mounted in or on separate devices that are proximate to each other.

7. The system according to claim 1 , wherein the data analyzer is configured to receive non-acoustic information and wherein the non-acoustic information includes at least one of an image or biometric information.

8. The system according to claim 1 , further including a storage device configured to store at least one of the at least one audio stream, non-acoustic information, the contextual information, the at least one search term, the search result or a log of classified information of non-acoustic information over a period of time.

9. The system according to claim 1 , wherein each device of the one or more devices has at least a communication capability for interchanging information with at least one of others of the one or more devices or a server.

10. The system according to claim 9 , wherein at least one of the server or the one or more of the devices includes the contextual search system.

11. The system according to claim 1 , wherein each of at least some of the devices include a user interface, the search being performed responsive to an indication received from the user interface.

12. The system according to claim 1 , wherein the contextual search system continuously monitors the at least one audio stream, and a further search is performed automatically responsive to an indication in the monitored audio stream.

13. The system according to claim 1 , wherein each of at least some of the one or more devices have the one or more of the sensors for acquiring the sensor data related to the environment proximate to the one or more devices and at least one or more microphones configured to collect the at least one audio stream proximate to the one or more devices.

14. The system according to claim 1 , wherein the one or more devices are configured to collect the at least one audio stream using at least one of the electromechanical transducer, an optical transducer, or a piezoelectric transducer.

15. The system according to claim 1 , wherein the one or more devices are configured to collect at least two audio streams from transducers including at least one ear canal microphone and an external microphone.

16. The system according to claim 1 , wherein at least one of the one or more devices includes at least one of a mobile telephone device, an earpiece, a hearing aid, a watch, a television remote, a car key fob, a portable game controller, a navigation device, or a computer having at least one sensor.

17. The system according to claim 1 , wherein further comprising an occlusion element configured to fully or substantially isolate an ear canal from an external audio field.

18. The system according to claim 1 , wherein the one or more devices are configured to collect the least one audio stream using least one of a throat microphone, a jaw microphone, or a bone conduction microphone.

19. A controller for contextual-based searching, the controller configured to interact with one or more devices, each device of the one or more devices having at least a communication capability for interchanging information with at least one of others of the one or more devices, the controller or a communication system, at least one of the one or more of the devices or the controller configured to collect at least one audio stream and sensor data proximate to the one or more of the devices or the controller, the controller comprising: a contextual search system including:
a data analyzer configured to receive the at least one audio stream and the sensor data from among at least one of the one or more devices or the controller and to determine contextual information from the received at least one audio stream and the sensor data,
a classifier that transcribes the at least one audio stream and detects an event in the at least one transcribed audio stream;
a search engine configured to perform a search of at least one search term using the contextual information and the detected event to produce a search result,
wherein, in response to the search result, a sensor module is activated and at least one further audio stream is collected by the one or more devices by utilizing the sensor module, and
wherein the contextual search system determines an effectiveness of the search result based on the at least one further audio stream.

20. The controller according to claim 19 , further including a communication module configured to receive the at least one audio stream from among the one or more devices.

21. The controller according to claim 19 , wherein two or more transducers from at least one of the one or more of the devices are selected across a network and used as a beamforming array.

22. The controller according to claim 19 , wherein each of at least some of the one or more devices have the sensor module including a mi

Assignees

• Goldstein Steven W
• Cohen Jordan
• Staton Techiya Llc

Inventors

• Goldstein Steven W
• Cohen Jordan

Classifications

• G06F16/9032

  Query formulation · CPC title

• G10L25/63

  for estimating an emotional state · CPC title

• G10L2015/088

  Word spotting · CPC title

• G06F16/638

  Presentation of query results · CPC title

• G10L17/00

  Speaker identification or verification techniques · CPC title