Virtual reality system with inside-out six degrees of freedom function of the head and hands

Patent metadata

Field	Value
Publication number	US-12299191-B2
Application number	US-202017778781-A
Country	US
Kind code	B2
Filing date	Sep 15, 2020
Priority date	Nov 21, 2019
Publication date	May 13, 2025
Grant date	May 13, 2025

Abstract

The present disclosure provides a virtual reality system. The virtual reality system of the disclosure, comprising: a head-mounted display, an input device and a positioning module, wherein the head-mounted display comprises a central processing unit, a camera module connected with the central processing unit and a wireless connection module; the camera module comprises a binocular fisheye camera, an IR camera and a TOF camera, the positioning module comprises a first inertial measurement unit and an electromagnetic receiver provided on the head-mounted display, a second inertial measurement unit and an electromagnetic transmitter provided on the input device; the central processing unit, configured to implement data interaction and command control with the binocular fisheye camera, the IR camera, the TOF camera, the wireless connection module, the first inertial measurement unit and the electromagnetic receiver.

First claim

What is claimed is:

1. A virtual reality system, comprising:
a head-mounted display, an input device and a positioning module, wherein the head-mounted display comprises a central processing unit, a camera module connected with the central processing unit and a wireless connection module;
the camera module comprises a binocular fisheye camera, an IR camera and a TOF camera, the positioning module comprises a first inertial measurement unit provided on the head-mounted display, an electromagnetic receiver provided on the head-mounted display, a second inertial measurement unit provided on the input device and an electromagnetic transmitter provided on the input device; and
the central processing unit is further connected with the first inertial measurement unit and the electromagnetic receiver respectively, and configured to implement data interaction and command control with the binocular fisheye camera, the IR camera, the TOF camera, the wireless connection module, the first inertial measurement unit and the electromagnetic receiver, so as to identity 6-DoF spatial positioning of the head-mounted display, identify 6-DoF spatial positioning of the input device, and implement an obstacle avoidance function, a gesture recognition function, an eyeball tracking function and a wireless transmission function of the virtual reality system;
wherein to implement the obstacle avoidance function the central processing unit is further configured to:
receive a depth image from the TOF camera;
receive an image from the binocular fisheye camera, wherein the image includes a gray image;
virtualize an obstacle into a scene as a virtualized object based on the depth image and the gray image; and
perform obstacle avoidance based on the virtualized object.

2. The virtual reality system according to claim 1 , wherein
the binocular fisheye camera is configured to obtain an image and send the image to the central processing unit;
the first inertial measurement unit is configured to obtain spatial attitude data of the head-mounted display and send the spatial attitude data to the central processing unit; and
the central processing unit is configured to identity the 6-DoF spatial positioning of the head-mounted display according to the received image and the spatial attitude data.

3. The virtual reality system according to claim 1 , wherein the input device is further provided with a wireless transmission module;
the second inertial measurement unit is configured to obtain spatial attitude data of the input device and send the spatial attitude data to the central processing unit through the wireless transmission module and the wireless connection module;
the electromagnetic transmitter is configured to transmit electromagnetic data;
the electromagnetic receiver is configured to receive the electromagnetic data transmitted by the electromagnetic transmitter and send the electromagnetic data to the central processing unit; and
the central processing unit is configured to calculate a relative positional relationship between the input device and the head-mounted display according to the received electromagnetic data, thereby identify the 6-DoF spatial positioning of the input device based on the relative positional relationship and the received spatial attitude data.

4. The virtual reality system according to claim 3 , wherein the input device comprises an electromagnetic handle.

5. The virtual reality system according to claim 1 , wherein the wireless connection module comprises WiGig and Wifi protocols.

6. The virtual reality system according to claim 5 , wherein
the central processing unit is configured to send 6-DoF spatial positioning data to an opposite terminal device of the virtual reality system through the wireless connection module, so as to enable the opposite terminal device to render and compress a to-be-processed image according to the 6-DoF spatial positioning data and send processed image data to the central processing unit through the wireless connection module; and
the central processing unit is configured to receive the image data sent by the opposite terminal device through the wireless connection module and displays the image data.

7. The virtual reality system according to claim 1 , wherein:
the TOF camera is configured to obtain a depth image and send the depth image to the central processing unit, and
the central processing unit is configured to implement the gesture recognition function according to the depth image.

8. The virtual reality system according to claim 1 , wherein infrared transmitters are provided around two eye cameras of the head-mounted display respectively;
the infrared transmitters are configured to transmit infrared rays to eyes through the eye cameras;
the IR camera is configured to obtain an infrared image of the eyes and send the infrared image to the central processing unit; and
the central processing unit is configured to implement the eyeball tracking function according to the infrared image.

9. The virtual reality system according to claim 8 , wherein the head-mounted display comprises two groups of IR cameras, and the two groups of IR cameras are provided in corresponding positions of the two eye cameras respectively.

10. The virtual reality system according to claim 8 , further comprising color selective mirrors, wherein each color selective mirror is configured to reflect the infrared rays received from a respective eye to a respective IR camera.

Assignees

• Qingdao Pico Tech Co Ltd

Inventors

• Zhang Xiuzhi
• Zhou Hongwei
• Liu Guanghui
• Guo Hengjiang

Classifications

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G02B2027/0132

  comprising binocular systems · CPC title