Interface system, control device, and operation assistance method

Patent metadata

Field	Value
Publication number	US-2025232522-A1
Application number	US-202519171355-A
Country	US
Kind code	A1
Filing date	Apr 7, 2025
Priority date	Oct 13, 2022
Publication date	Jul 17, 2025
Grant date	—

Abstract

An interface system includes a detecting unit to detect a three-dimensional position of a detection target in a virtual space divided into a plurality of operation spaces, a position acquiring unit to acquire a three-dimensional position of the detection target detected by the detecting unit, a projecting unit to project an aerial image indicating a boundary position of each of the operation spaces in the virtual space, an operation space determining unit to determine an operation space including the three-dimensional position of the detection target on the basis of the three-dimensional position of the detection target acquired by the position acquiring unit and the boundary position of each operation space in the virtual space, and an operation information output unit to output operation information for executing a predetermined operation on a display device by using at least a determination result by the operation space determining unit.

First claim

1 . An interface system comprising processing circuitry
to perform detection of a three-dimensional position of a detection target in a virtual space divided into a plurality of operation spaces, to acquire the three-dimensional position of the detection target detected by the detection, to project an aerial image indicating a boundary position of each of the plurality of operation spaces in the virtual space, to perform determination of an operation space including the three-dimensional position of the detection target on a basis of the three-dimensional position of the detection target and the boundary position of each of the plurality of operation spaces in the virtual space, and to output operation information for executing a predetermined pointer movement operation or a predetermined command execution operation to a display information of a display device, by using at least a result of the determination, wherein the predetermined pointer movement operation and the predetermined command execution operation which can be executed continuously are respectively associated with adjacent operation spaces among the plurality of operation spaces, and the aerial image is projected to a closed plane dividing the adjacent operation spaces.

2 . The interface system according to claim 1 , wherein the processing circuitry is further configured
to generate data representing the aerial image, and to project the aerial image based on the data generated.

3 . The interface system according to claim 2 , wherein the processing circuitry is further configured
to regenerate the data representing the aerial image to be projected in a projection mode depending on the result of the determination, and to project the aerial image based on the data regenerated.

4 . The interface system according to claim 2 , wherein
the processing circuitry is further configured to generate information for moving a pointer displayed on a screen of the display device corresponding to a motion of the detection target in a first operation space when the processing circuitry determines that the operation space including the three-dimensional position of the detection target is the first operation space, and output the operation information including the generated information.

5 . The interface system comprising processing circuitry
to perform detection of a three-dimensional position of a detection target in a virtual space divided into a plurality of operation spaces, to acquire the three-dimensional position of the detection target detected by the detection, to project an aerial image indicating a boundary position of each of the plurality of operation spaces in the virtual space, to perform determination of an operation space including the three-dimensional position of the detection target on a basis of the three-dimensional position of the detection target and the boundary position of each of the plurality of operation spaces in the virtual space, to output operation information for executing a predetermined operation, the predetermined operation being performed on a display device, by using at least a result of the determination, to generate data representing the aerial image, to project the aerial image based on the data generated, to generate information for moving a pointer displayed on a screen of the display device corresponding to a motion of the detection target in a first operation space when the processing circuitry determines that the operation space including the three-dimensional position of the detection target is the first operation space, and output the operation information including the generated information, and to output information indicating that a moving amount or a moving speed of the pointer displayed on the screen of the display device is variable depending on a distance between the three-dimensional position of the detection target included in the first operation space and a boundary plane of the virtual space represented by the aerial image in a direction orthogonal to the boundary plane, by including the information in the operation information.

6 . The interface system according to claim 4 , wherein
the processing circuitry is further configured to generate information indicating to output a sound corresponding to the three-dimensional position of the detection target in the first operation space or a sound corresponding to the motion of the detection target in the first operation space, and output the information by including the information in the operation information.

7 . The interface system comprising processing circuitry
to perform detection of a three-dimensional position of a detection target in a virtual space divided into a plurality of operation spaces, to acquire the three-dimensional position of the detection target detected by the detection, to project an aerial image indicating a boundary position of each of the plurality of operation spaces in the virtual space, to perform determination of an operation space including the three-dimensional position of the detection target on a basis of the three-dimensional position of the detection target and the boundary position of each of the plurality of operation spaces in the virtual space, to output operation information for executing a predetermined operation, the predetermined operation being performed on a display device, by using at least a result of the determination, to generate data representing the aerial image, to project the aerial image based on the data generated, to generate information for moving a pointer displayed on a screen of the display device corresponding to a motion of the detection target in a first operation space when the processing circuitry determines that the operation space including the three-dimensional position of the detection target is the first operation space, and output the operation information including the generated information, and to generate, when the processing circuitry determines that the operation space including the three-dimensional position of the detection target is a second operation space, information indicating to fix the pointer displayed on the screen of the display device, and outputs the generated information by including the information in the operation information.

8 . The interface system according to claim 7 , wherein
the processing circuitry is further configured to specify a motion of the detection target in a second operation space and specify a command corresponding to the specified motion of the detection target when the processing circuitry determines that the operation space including the three-dimensional position of the detection target is the second operation space; and to output the operation information including information indicating the command specified.

9 . The interface system according to claim 7 , wherein
the processing circuitry is further configured to generate, when the processing circuitry determines that the operation space including the three-dimensional position of the detection target is a second operation space, data indicating a lower limit position of a detectable range of the detection and representing an aerial image that divides the second operation space into left and right spaces, and to project the aerial image based on the data generated.

10 . The interface system according to claim 7 , wherein
the processing circuit is further configured to generate information indicating to output a sound corresponding to fixation of the pointer, and output the information by including the information in the operation information.

11 . The interface system according to claim 7 , wherein
the processing c

Assignees

• Mitsubishi Electric Corp

Inventors

• Higuchi Hirohiko
• Ito Maki
• Kayama Shodai
• KIKUTA Hayato
• TAKAGAWA Natsuki

Classifications

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G06F3/0484

  for the control of specific functions or operations, e.g. selecting or manipulating an object, an image or a displayed text element, setting a parameter value or selecting a range · CPC title

• G06F3/0487

  using specific features provided by the input device, e.g. functions controlled by the rotation of a mouse with dual sensing arrangements, or of the nature of the input device, e.g. tap gestures based on pressure sensed by a digitiser · CPC title