Interface device and interface system

Patent metadata

Field	Value
Publication number	US-2026072550-A1
Application number	US-202319107590-A
Country	US
Kind code	A1
Filing date	Aug 9, 2023
Priority date	Oct 13, 2022
Publication date	Mar 12, 2026
Grant date	—

Abstract

An interface device includes: a detection unit that detects the three-dimensional position of a detection target in a virtual space; and a projection unit that projects an aerial image onto the virtual space. The virtual space is divided into a plurality of operation spaces for each operation executable by a user in a case where the three-dimensional position of the detection target detected by the detection unit is included, the operation being defined in each of the operation spaces, and a boundary position of each operation space in the virtual space is indicated by an aerial image projected by the projection unit.

First claim

1 . An interface device comprising:
a detector to detect a three-dimensional position of a detection target in a virtual space; and a projector to project an aerial image onto the virtual space, wherein the virtual space comprises a plurality of operation spaces for each operation executable by a user when the three-dimensional position of the detection target detected by the detector is included and the operation being defined in each operation space among the plurality of operation spaces, a predetermined pointer movement operation to the display information of the display device in conjunction with the movement of the detection target in the operation space being defined in at least any one of the plurality of the operation spaces, and a boundary position of each of the operation spaces in the virtual space is indicated by the aerial image projected by the projector.

2 .- 25 . (canceled)

26 . The interface device according to claim 1 , wherein
the operation includes at least any one of an operation to perform movement operation of a predetermined pointer to the display information of a display device, execution operation of a predetermined command or operation performed by using a mouse or a touch panel.

27 . The interface device according to claim 1 , wherein
the aerial image projected onto the virtual space is formed at a position where a decrease in accuracy of detection of the three-dimensional position of the detection target by the detector is reduced.

28 . The interface device according to claim 1 , wherein
the projector forms the aerial image in the virtual space, to cause the aerial image to include an angle of view of the detector.

29 . The interface device according to claim 1 , wherein
the projector changes a projection mode of the aerial image projected onto the virtual space, depending on at least one of an operation space in which the three-dimensional position of the detection target detected by the detector is included, or movement of the detection target in the operation space in which the three-dimensional position of the detection target is included.

30 . The interface device according to claim 1 , wherein one or more of the aerial images are projected onto the virtual space, and at least one of the aerial images present one of an outer frame or an outer surface of the virtual space to the user.

31 . The interface device according to claim 1 , wherein the projector, is one or a plurality of imaging optical systems forming light emitted by the light source as a real image, the real image by the light source formed as the aerial image.

32 . The interface device according to claim 31 ,
wherein the projector includes:
two or more of the imaging optical systems forming light emitted from the light source as two or more of real images by different light paths, and
the two or more of the imaging optical systems forming the real image as each of different points, lines or planes as two or more of the aerial images indicating boundary positions of the operation spaces.

33 . The interface device according to claim 31 , wherein the projector includes:
two or more of the light sources for forming two or more of aerial image.

34 . The interface device according to claim 31 , wherein the imaging optical system includes:
a beam splitter splitting light emitted from the light source into a transmission light and a reflection light, having light beam bending plane constituting one plane on which a light path of the light emitted from the light source being bent; and a retroreflection member reflecting the reflected light to incident direction when the reflected light from the beam splitter is incident.

35 . The interface device according to claim 34 , wherein
the imaging optical system includes: two or more of pairs of the beam splitters and the retroreflection members, each pair of the beam splitters and the retroreflection members forms two or more of aerial images indicating the boundary of the operation space as different points, lines or planes of each of the light emitted from the light source.

36 . The interface device according to claim 31 , wherein
an imaging optical system has a light beam bending plane that forms one plane in which an optical path of light emitted from a light source is bent, the imaging optical system forming a real image with the light source disposed on one surface side of the light beam bending plane, the real image being formed as the aerial image on an opposite surface side of the light beam bending plane.

37 . The interface device according to claim 31 , wherein
an imaging optical system has a light beam bending plane that forms one plane in which an optical path of light emitted from a light source is bent,
the detector is disposed in an internal region of the imaging optical system, and on one surface side of the light beam bending plane of the imaging optical system.

38 . The interface device according to claim 1 , wherein
the aerial image projected by the projector is visually recognizable by the user, together with the video information displayed on a display device.

39 . The interface device according to claim 1 , further comprising a controller to change an angle at which the boundary plane as the plane onto which the aerial image is projected in the virtual space spatially intersects with a display plane of the display device or set an angle at which the boundary plane does not intersect with a display plane of the display device.

40 . The interface device according to claim 31 , further comprising two or more of the light sources, wherein
the light sources are arranged in such a manner that at least either axes or planes on spaces formed by the light sources are not parallel, real images are formed as the aerial images by pairs of the beam splitters and the retroreflective members, and the aerial images are formed in parallel to each other on any plane onto which the aerial images are projected in the virtual space.

41 . The interface device according to claim 31 , further comprising two or more of the light sources, wherein each of the light sources has a variable posture, and
changing the posture of each of the light sources changes the posture of each of the aerial images and changes an angle formed by a boundary plane onto which each of the aerial images is projected with respect to a horizontal plane.

42 . The interface device according to claim 41 , wherein two or more of the light sources have a variable posture, and
changing the posture of each of the light sources changes the position relations or directions of each of the aerial images formed.

43 . An interface device that allows an operation of an application displayed on a display device, the interface device comprising:
a detector to detect a three-dimensional position of a detection target in a virtual space comprising a plurality of operation spaces; at least one boundary definer to indicate a boundary position of each of the operation spaces, the at least one boundary definer including one of a line or a plane; and a boundary displayer to provide at least one visually recognizable boundary of each of the operation spaces, the boundary displayer including one of a point, a line, or a plane, wherein, when the three-dimensional position of the detection target detected by the detector is included in the virtual space, the detection target is enabled to perform a plurality of kinds of operations on the application, the operations b

Assignees

• Mitsubishi Electric Corp

Inventors

• KIKUTA Hayato
• Higuchi Hirohiko
• TAKAGAWA Natsuki
• Ito Maki
• Kayama Shodai

Classifications

• G06F3/0425

  using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386) · CPC title

• G06F3/017

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title

• G02B30/56

  by projecting aerial or floating images · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06F3/042 Primary

  by opto-electronic means · CPC title