Immersive virtual golf service with reality using XR device

Patent metadata

Field	Value
Publication number	US-12330039-B2
Application number	US-202418817694-A
Country	US
Kind code	B2
Filing date	Aug 28, 2024
Priority date	Aug 30, 2023
Publication date	Jun 17, 2025
Grant date	Jun 17, 2025

Abstract

One example relates to a method for providing an immersive virtual golf service using an eXtended Reality (XR). The method may comprise: capturing an image of an actual golf ball placed on a ground through a camera of an XR device; reproducing and displaying the image within a virtual golf environment displayed on a display unit of the XR device; capturing, in real-time, a golf swing of a user through the camera of the XR device; analyzing a golf swing motion of the user in real-time using an artificial neural network model; and reproducing and displaying the analyzed golf swing motion within the virtual golf environment displayed on the display unit of the XR device.

First claim

What is claimed is:

1. A method for providing an immersive virtual golf service using an eXtended Reality (XR), the method comprising:
capturing an image of an actual golf ball placed on a ground through a first plurality of cameras which are arranged to face forward of the XR device and a second plurality of cameras which are arranged to face downward of the XR device;
reproducing and displaying the image within a virtual golf environment displayed on a display unit of the XR device;
capturing, in real-time, a golf swing of a user through the first plurality of cameras and the second plurality of cameras;
analyzing a golf swing motion of the user in real-time using an artificial neural network model; and
reproducing and displaying the analyzed golf swing motion within the virtual golf environment displayed on the display unit of the XR device.

2. The method of claim 1 , wherein the artificial neural network model is stored in the XR device.

3. The method of claim 1 , wherein the artificial neural network model includes:
a first artificial neural network model configured for object detection, object segmentation, or object tracking.

4. The method of claim 1 , wherein the artificial neural network model includes:
a second artificial neural network model configured for a pose estimation.

5. The method of claim 1 , wherein the artificial neural network model includes:
a third artificial neural network model configured for scaling.

6. The method of claim 1 , wherein the artificial neural network model includes:
a fourth artificial neural network model configured for motion prediction.

7. The method of claim 1 , wherein the XR device includes a neural processing unit configured to perform computations for the artificial neural network model.

8. The method of claim 1 , further comprising:
displaying a virtual golfer's swing in caparison with a user's golf swing.

9. The method of claim 1 , further comprising:
scaling a body of a virtual golfer displayed on the XR device based on body dimensions of the user.

10. An eXtended Reality (XR) device comprising:
a first plurality of cameras which are arranged to face forward;
a second plurality of cameras which are arranged to face downward;
a display unit; and
a controller configured to control the camera and display and perform steps of:
capturing an image of an actual golf ball placed on a ground through the first plurality of cameras and the second plurality of cameras,
reproducing and displaying the image within a virtual golf environment displayed on the display unit,
capturing, in real-time, a golf swing of a user through the first plurality of cameras and the second plurality of cameras,
analyzing a golf swing motion of the user in real-time using an artificial neural network model, and
reproducing and displaying the analyzed golf swing motion within the virtual golf environment displayed on the display unit.

11. The XR device of claim 10 , wherein the artificial neural network model is stored in the XR device.

12. The XR device of claim 10 , wherein the artificial neural network model includes:
a first artificial neural network model configured for object detection, object segmentation, or object tracking.

13. The XR device of claim 10 , wherein the artificial neural network model includes:
a second artificial neural network model configured for a pose estimation.

14. The XR device of claim 10 wherein the artificial neural network model includes:
a third artificial neural network model configured for scaling.

15. The XR device of claim 10 , wherein the artificial neural network model includes:
a fourth artificial neural network model configured for motion prediction.

16. A non-volatile computer-readable storage medium storing instructions, the instructions, when executed by one or more processors, causing the one or more processors to perform steps comprising:
capturing an image of an actual golf ball placed on a ground through a camera of an eXtended Reality (XR) device,
wherein the camera of the XR device includes a first plurality of cameras which are arranged to face forward and a second plurality of cameras which are arranged to face downward;
reproducing and displaying the image within a virtual golf environment displayed on a display unit of the XR device;
capturing, in real-time, a golf swing of a user through the first plurality of cameras and the second plurality of cameras;
analyzing a golf swing motion of the user in real-time using an artificial neural network model; and
reproducing and displaying the analyzed golf swing motion within the virtual golf environment displayed on the display unit of the XR device.

Assignees

• Kineverse Inc
• Univ Konkuk Ind Coop Corp

Inventors

• Kim Gyoung Mo
• Yu Sang Geun
• Ryu Sungwon
• Yu In Gul

Classifications

• G06F3/012

  Head tracking input arrangements · CPC title

• G06N3/067 Primary

  using optical means · CPC title

• A63B2071/0666

  worn on the head or face, e.g. combined with goggles or glasses · CPC title

• A63B71/0619

  Displays, user interfaces and indicating devices, specially adapted for sport equipment, e.g. display mounted on treadmills · CPC title

• A63B2102/32

  Golf · CPC title