Adaptive tracking system for spatial input devices

Patent metadata

Field	Value
Publication number	US-9740922-B2
Application number	US-201514606563-A
Country	US
Kind code	B2
Filing date	Jan 27, 2015
Priority date	Apr 24, 2008
Publication date	Aug 22, 2017
Grant date	Aug 22, 2017

Abstract

An adaptive tracking system for spatial input devices provides real-time tracking of spatial input devices for human-computer interaction in a Spatial Operating Environment (SOE). The components of an SOE include gestural input/output; network-based data representation, transit, and interchange; and spatially conformed display mesh. The SOE comprises a workspace occupied by one or more users, a set of screens which provide the users with visual feedback, and a gestural control system which translates user motions into command inputs. Users perform gestures with body parts and/or physical pointing devices, and the system translates those gestures into actions such as pointing, dragging, selecting, or other direct manipulations. The tracking system provides the requisite data for creating an immersive environment by maintaining a model of the spatial relationships between users, screens, pointing devices, and other physical objects within the workspace.

First claim

What is claimed is:

1. A system comprising:
a plurality of sensors, wherein a location of the plurality of sensors defines a spatial operating environment (SOE) that includes a plurality of objects; and
a tracking component running on a processor, wherein the tracking component receives from each sensor of the plurality of sensors feature data corresponding to each object of the plurality of objects detected by the respective sensor, wherein the tracking component generates and maintains from the feature data a coherent model of relationships between the plurality of objects and the SOE.

2. The system of claim 1 , wherein the coherent model includes spatial relationships between the plurality of objects.

3. The system of claim 2 , wherein the coherent model includes at least one of location, orientation, and motion of the plurality of objects.

4. The system of claim 1 , wherein the ATC generates coincidence between virtual space and physical space that includes the SOE.

5. The system of claim 1 , wherein a sensor detects a pose comprising location and orientation of the object relative to the sensor, wherein the pose comprises a six-degree-of-freedom (DOF) pose.

6. The system of claim 1 , wherein the plurality of objects include at least one of a body, an appendage of a body, a device, an article of clothing, a glove, a display device, a piece of furniture.

7. The system of claim 1 , wherein an origin of the coherent model is defined relative to a particular sensor of the plurality of sensors.

8. The system of claim 1 , wherein an output of the ATC controls at least one of a function of an application, a display component, and a remote component coupled to the processor.

9. The system of claim 1 , wherein the tracking component automatically detects a gesture of at least one object of the plurality of objects.

10. The system of claim 9 , wherein the tracking component identifies a gesture of at least one object of the plurality of objects.

11. The system of claim 9 , comprising translating the gesture to a gesture signal, and controlling a component coupled to the processor in response to the gesture signal.

12. The system of claim 11 , wherein the detecting comprises identifying the gesture, wherein the identifying includes identifying a pose and an orientation of a portion of the object.

13. The system of claim 12 , wherein the translating comprises translating information of the gesture to a gesture notation, wherein the gesture notation represents a gesture vocabulary, and the gesture signal comprises communications of the gesture vocabulary.

14. The system of claim 13 , wherein the gesture vocabulary represents in textual form at least one of instantaneous pose states of kinematic linkages of the object, an orientation of kinematic linkages of the object, and a combination of orientations of kinematic linkages of the object.

15. The system of claim 13 , wherein the gesture vocabulary includes a string of characters that represent a state of kinematic linkages of the object.

16. The system of claim 11 , wherein controlling the component comprises controlling a three-space object in six degrees of freedom simultaneously by mapping the gesture to the three-space object, wherein the plurality of objects includes the three-space object.

17. The system of claim 16 , comprising presenting the three-space object on a display device.

18. The system of claim 16 , comprising controlling movement of the three-space object by mapping a plurality of gestures to a plurality of object translations of the three-space object.

19. The system of claim 16 , wherein the detecting comprises detecting when an extrapolated position of the object intersects virtual space, wherein the virtual space comprises space depicted on a display device, wherein controlling the component comprises controlling a virtual object in the virtual space when the extrapolated position intersects the virtual object.

20. The system of claim 1 , comprising detecting an event of a source device, wherein the source device includes the tracking component running on the processor, generating at least one data sequence comprising device event data specifying the event and state information of the event, wherein the device event data and state information are type-specific data having a type corresponding to an application of the source device, and forming a data capsule to include the at least one data sequence, the data capsule having a data structure comprising an application-independent representation of the at least one data sequence.

21. The system of claim 20 , wherein the generating of the at least one data sequence comprises:
generating a first respective data set that includes first respective device event data;
generating a second respective data set that includes second respective state information; and
forming a first data sequence to include the first respective data set and the second respective data set.

22. The system of claim 21 , wherein the generating of the first respective data set includes forming the first respective data set to include identification data of the source device, the identification data including data identifying the source device.

23. The system of claim 21 , wherein the generating of the at least one data sequence comprises:
generating a first respective data set that includes first respective device event data;
generating a second respective data set that includes second respective state information; and
forming a second data sequence to include the first respective data set and the second respective data set.

24. The system of claim 23 , wherein the generating of the first respective data set includes generating a first respective data set offset, wherein the first respective data set offset points to the first respective data set of the second data sequence.

25. The system of claim 23 , wherein the generating of the second respective data set includes generating a second respective data set offset, wherein the second respective data set offset points to the second respective data set of the second data sequence.

26. The system of claim 21 , wherein the first respective data set is a description list, the description list including a description of the data.

27. The system of claim 20 , wherein the device event data is a tagged byte-sequence representing typed data, wherein the device event data includes a type header and a type-specific data layout.

28. The system of claim 20 , wherein the state information is a tagged byte-sequence representing typed data, wherein the state information includes a type header and a type-specific data layout.

29. The system of claim 20 , comprising:
generating at least one offset; and
forming the data capsule to include the at least one offset.

30. The system of claim 29 , comprising:
generating a first offset having a first variable length;
wherein the first offset points to the device event data of a first data sequence of the at least one data sequence.

31. The system of claim 29 , comprising:
generating a second offset having a second variable length;
wherein the second offset points to the state information of a first data sequence of the at least one data sequence.

32. The system of claim 29 , comprising:
forming a first code pat

Assignees

• Oblong Ind Inc

Inventors

• Csaszar Ambrus
• Kogan Dima
• Yarin Paul

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/661

  Transmitting camera control signals through networks, e.g. control via the Internet · CPC title

• H04N23/695

  Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects · CPC title

• H04N7/181

  for receiving images from a plurality of remote sources · CPC title

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title