Systems and methods for stable haptic feedback over packet-switched networks

Patent metadata

Field	Value
Publication number	US-9501946-B1
Application number	US-201414571377-A
Country	US
Kind code	B1
Filing date	Dec 16, 2014
Priority date	Dec 17, 2013
Publication date	Nov 22, 2016
Grant date	Nov 22, 2016

Abstract

Providing haptic feedback to a user over a packet-switched network includes sensing movement of a haptic device by a user, the haptic device being configured to control a first virtual object within a virtual environment, transmitting data associated with the movement of the haptic device from a client computer to a remote server via the packet-switched network, the server modeling movement of the first virtual object in the virtual environment and estimating an orientation and a position of the first virtual object within the virtual environment, the server determining a type of tactile feedback to be provided to the user, the server transmitting the type of tactile feedback data to the client computer via the packet-switched network, and providing tactile feedback to the user with the haptic device.

First claim

The invention claimed is:

1. A method for providing haptic feedback to a user over a packet-switched network, the method comprising:
sensing movement of a haptic device by a user, the haptic device being configured to control a first virtual object within a virtual environment;
transmitting data associated with the movement of the haptic device from a client computer to a remote server via the packet-switched network;
the server modeling movement of the first virtual object in the virtual environment and estimating an orientation and a position of the first virtual object within the virtual environment;
the server determining a type of tactile feedback to be provided to the user;
the server transmitting the type of tactile feedback data to the client computer via the packet-switched network; and
providing tactile feedback to the user with the haptic device.

2. The method of claim 1 , wherein the haptic device controls a virtual dental or surgical instrument in the virtual environment.

3. The method of claim 1 , wherein transmitting data associated with the movement of the haptic device from a client computer comprises transmitting the data to the server from a thin client or a mobile client.

4. The method of claim 1 , wherein transmitting data associated with the movement of the haptic device via the packet-switched network comprises transmitting the data using a user datagram protocol (UDP).

5. The method of claim 1 , wherein transmitting data associated with the movement of the haptic device via the packet-switched network comprises accounting for time-varying delays in the packet-switched network.

6. The method of claim 1 , wherein modeling movement of the first virtual object in the virtual environment comprises modeling the movements as a stochastic process.

7. The method of claim 6 , wherein estimating the orientation and the position of the first virtual object comprises estimating the orientation and the position using a recursive Bayesian filtering technique.

8. The method of claim 7 , wherein estimating the orientation and the position of the first virtual object comprises determining the orientation and the position using an optimal j-step look-ahead Kalman predictor.

9. The method of claim 8 , further comprising performing optimal smoothing using a Kalman filter after a contact has been registered between the first virtual object and a second virtual object in the virtual environment.

10. The method of claim 9 , wherein determining tactile feedback to be provided to the user comprises determining the nature of the contact between the first and second virtual objects in the virtual environment.

11. A non-transitory computer-readable medium that stores a haptic communication and control system, the system comprising:
logic configured to receive via a packet-switched network data associated with movement of a haptic device by a user;
logic configured to model movement of a first virtual object within a virtual environment and to estimate an orientation and a position of the first virtual object within the virtual environment based upon the received data;
logic configured to determine a type of tactile feedback to be provided to the user; and
logic configured to transmit the type of tactile feedback data to a client computer via the packet-switched network.

12. The non-transitory computer-readable medium of claim 11 , wherein the logic configured to receive data associated with movement of a haptic device comprises is configured to receive the using a user datagram protocol (UDP).

13. The non-transitory computer-readable medium of claim 11 , wherein the logic configured to model movement of the first virtual object within the virtual environment and to estimate the orientation and the position of the first virtual object is configured to model the movements as a stochastic process.

14. The non-transitory computer-readable medium of claim 13 , wherein the logic configured to model movement of the first virtual object within the virtual environment and to estimate the orientation and the position of the first virtual object is configured to estimate the orientation and the position using a recursive Bayesian filtering technique.

15. The non-transitory computer-readable medium of claim 14 , wherein the logic configured to model movement of the first virtual object within the virtual environment and to estimate the orientation and the position is configured to determine the orientation and the position using an optimal j-step look-ahead Kalman predictor.

16. A system for providing haptic feedback to a user over a packet-switched network, the system comprising:
a client computer;
a haptic device associated with the client computer, the haptic device being configured to control a first virtual object within a virtual environment; and
a remote server connected to the client computer via a packet-switched network, the server comprising a haptic communication and control system configured to:
receive via the packet-switched network data associated with movement of the haptic device by a user,
model movement of the first virtual object within the virtual environment and to estimate an orientation and a position of the first virtual object within the virtual environment based upon the received data,
determine a type of tactile feedback to be provided to the user, and
transmit the type of tactile feedback data to the client computer via the packet-switched network that can be used to control the haptic device to provide tactile feedback to the user.

17. The system of claim 16 , wherein the client computer is a thin client or a mobile client.

18. The system of claim 16 , wherein the haptic communication and control system of the server is configured to model the movements as a stochastic process.

19. The system of claim 18 , wherein the haptic communication and control system of the server is configured to estimate the orientation and the position of the first virtual object using a recursive Bayesian filtering technique.

20. The system of claim 19 , wherein the haptic communication and control system of the server is configured determine the orientation and the position of the first virtual object using an optimal j-step look-ahead Kalman predictor.

Assignees

• Kulkarni Parikshit Sudhir
• Lai-Yuen Susana Karina
• Univ South Florida

Inventors

• Kulkarni Parikshit Sudhir
• Lai-Yuen Susana Karina

Classifications

• G08B6/00

  Tactile signalling systems, e.g. tactile personal calling systems · CPC title

• G09B9/00 Primary

  Simulators for teaching or training purposes (for the use of weapons F41; computing aspects G06; {protocols for games, networked simulations or virtual reality H04L67/131}) · CPC title

• H04L29/06034

  Electricity · mapped topic

• H04L67/08 Primary

  specially adapted for terminal emulation, e.g. Telnet · CPC title

• G09B23/283

  for dentistry or oral hygiene (G09B23/286 takes precedence) · CPC title