Systems and methods for a dual modality sensor system

Patent metadata

Field	Value
Publication number	US-9437002-B2
Application number	US-201414496037-A
Country	US
Kind code	B2
Filing date	Sep 25, 2014
Priority date	Sep 25, 2014
Publication date	Sep 6, 2016
Grant date	Sep 6, 2016

Abstract

The present disclosure provides systems and methods for using two imaging modalities for imaging an object at two different resolutions. For example, the system may utilize a first modality (e.g., ultrasound or electromagnetic radiation) to generate image data at a first resolution. The system may then utilize the other modality to generate image data of portions of interest at a second resolution that is higher than the first resolution. In another embodiment, one imaging modality may be used to resolve an ambiguity, such as ghost images, in image data generated using another imaging modality.

First claim

What is claimed is:

1. A method for generating an image of an object within a region, comprising:
transmitting ultrasound, via a first ultrasonic transmitter of an ultrasonic transmission system, into the region;
receiving, via a first ultrasonic receiver of an ultrasonic receiver system, ultrasonic reflections of the transmitted ultrasound from a plurality of sites on the object within the region, the ultrasonic reflections comprising sound waves within a first frequency band, the ultrasonic reflections including direct ultrasonic reflection and rebounded ultrasonic reflection, wherein the direct ultrasonic reflection is reflected from a first portion of the object and the rebounded ultrasonic reflection is reflected from a second, different portion of the object;
generating, via a processor, coarse image data of the object at a first resolution based on the received ultrasonic reflections;
identifying a portion of interest on the object based on the coarse image data;
receiving electromagnetic radiation from the identified portion of interest on the object, the electromagnetic radiation comprising electromagnetic waves within a second frequency band, the second frequency band being different than the first frequency band;
generating fine image data of the portion of interest on the object at a second resolution based on the received electromagnetic radiation, wherein the second resolution is greater than the first resolution;
determining a kinematic value associated with the portion of interest on the object based on at least one of the received electromagnetic radiation and the received ultrasonic reflections, wherein positional data is generated based at least on the direct ultrasonic reflection and the rebounded ultrasonic reflection, wherein the kinematic value is determined based at least in part on the positional data; and
modifying the state of an entertainment device based on the determined kinematic value associated with the portion of interest on the object.

2. The method of claim 1 , wherein the portion of interest is identified based at least in part on a state of the entertainment device.

3. The method of claim 1 , further comprising transmitting electromagnetic energy into the region via an electromagnetic transmitter, and
wherein receiving electromagnetic radiation from the identified portion of interest on the object comprises receiving a reflected portion of the transmitted electromagnetic radiation.

4. The method of claim 1 , wherein at least one of the transmitted ultrasound and the received reflected ultrasound are rebounded from an auxiliary ultrasonic reflector.

5. The method of claim 1 , wherein calculating the kinematic value associated with the portion of interest on the object comprises a Doppler velocity calculation in which the first velocity (V dop ) is a function of: a calculated shift (Δ s ), the frequency of the transmitted ultrasound (f 0 ), an arrival velocity of the ultrasound (V in ), and a departure velocity of the ultrasound (V out ).

6. The method of claim 1 , wherein detecting a shift and calculating the first velocity component are performed using a derivation or equivalent equation to the following equation:

f
r

=

(

C
-

V
o

C
+

V
o

)

⁢

f
t

,

where f r is based on the frequency of the received ultrasound, C is based on a velocity of the ultrasound in a medium within the region, V o is based on a velocity of the site on the object relative to the medium, and f t is based on a frequency of the transmitted ultrasound.

7. The method of claim 1 , further comprising:
receiving, via a second ultrasonic receiver of the ultrasonic receiver system that is physically separated from the first ultrasonic receiver, an ultrasonic reflection from the object;
detecting a shift of the ultrasonic reflection received by the second ultrasonic receiver; and
calculating, via the processor, a second kinematic value associated with the portion of interest on the object based on the detected shift of the ultrasonic reflection received by the second ultrasonic receiver.

8. The method of claim 7 , further comprising:
receiving, via a third ultrasonic receiver of the ultrasonic receiver system that is physically separated from the first ultrasonic receiver, an ultrasonic reflection from the object;
detecting a shift of the ultrasonic reflection received by the third ultrasonic receiver; and
calculating, via the processor, a third kinematic value associated with the portion of interest on the object based on the detected shift of the ultrasonic reflection received by the third ultrasonic receiver.

9. The method of claim 8 , wherein the first, second and third ultrasonic receivers are non-collinear with respect to one another.

10. The method of claim 1 , wherein a plurality of kinematic values are calculated for a corresponding plurality of sites on the object.

11. The method of claim 10 , wherein the plurality of kinematic values are used to determine a translational velocity of the object.

12. The method of claim 1 , wherein the portion of interest on the object comprises a portion of a person selected from the group consisting of a hand, finger, arm, leg, foot, toe, torso, neck, head, mouth, lip, and eye.

13. The method of claim 1 , wherein determining the kinematic value of the portion of interest on the object comprises:
transmitting ultrasound, via the ultrasound transmission system, into the region, wherein the region is bounded by a first surface;
receiving, via the ultrasonic receiver system, the direct ultrasonic reflection from the object;
generating direct positional data associated with the object based on the direct ultrasonic reflection;
receiving the rebounded ultrasonic reflection from the object, wherein the rebounded ultrasonic reflection comprises ultrasound reflected by the object and the first surface prior to being received by the ultrasonic receiver;
generating rebounded positional data using the rebounded ultrasonic reflection of the object from the first surface; and
generating enhanced positional data by combining the direct positional data and the rebounded positional data.

14. The method of claim 1 , wherein determining the kinematic value of the portion of interest on the object comprises:
transmitting ultrasound into the region, wherein the region is bounded by a first surface;
receiving the direct ultrasonic reflection from the object;
generating direct positional data associated with the object based on the direct ultrasonic reflection;
receiving a rebounded ultrasonic reflection from the object, wherein the rebounded ultrasonic reflection comprises ultrasound reflected by the object and the first surface prior to being received;
generating rebounded positional

Assignees

• Elwha Llc

Inventors

• Cheatham Iii Jesse R
• Hyde Roderick A
• Ishikawa Muriel Y
• Kare Jordin T
• Mundie Craig J
• Myhrvold Nathan P
• Petroski Robert C
• Rudder Eric D
• Tan Desney S
• Tegreene Clarence T
• Whitmer Charles
• Wilson Andrew
• Wing Jeannette M
• Wood Jr Lowell L
• Wood Victoria Y H

Classifications

• G06V40/107

  Static hand or arm · CPC title

• G06V20/64 Primary

  Three-dimensional [3D] objects · CPC title

• G01S15/58

  Velocity or trajectory determination systems; Sense-of-movement determination systems {(velocity measurement in imaging systems G01S15/8979)} · CPC title

• G06T2207/10132

  Ultrasound image · CPC title

• A63F13/424

  involving acoustic input signals, e.g. by using the results of pitch or rhythm extraction or voice recognition · CPC title