Head-mounted display apparatus, inspection supporting display system, display method, and recording medium recording display program

Patent metadata

Field	Value
Publication number	US-10877269-B2
Application number	US-201916510443-A
Country	US
Kind code	B2
Filing date	Jul 12, 2019
Priority date	Jul 13, 2018
Publication date	Dec 29, 2020
Grant date	Dec 29, 2020

Abstract

A head-mounted display apparatus includes a display device configured to form a virtual display region by a virtual image of an electronic video in a visual field of a user, an image pickup device configured to pick up an image of an image pickup range including a part or a whole of visual field of the user, and a processor. The processor determines, with signal processing for a picked-up image by the image pickup device, an observation necessary range in the picked-up image and performs display control for displaying, in the virtual display region, an image portion of the observation necessary range in the picked-up image.

First claim

What is claimed is:

1. A head-mounted display apparatus comprising:
a display device configured to form a virtual display region by a virtual image of an electronic video in a visual field of a user;
an image pickup device configured to pick up an image of an image pickup range including a part or a whole of visual field of the user; and
a processor, wherein
the processor determines, with signal processing for a picked-up image by the image pickup device, an observation necessary range in the picked-up image and performs display control for displaying, in the virtual display region, an image portion of the observation necessary range in the picked-up image, and
the processor detects a fingertip of the user's left hand and a fingertip of the user's right hand and sets a range surrounded by the user's left hand fingertip and right hand fingertip as the observation necessary range.

2. A head-mounted display apparatus comprising:
a display device configured to form a virtual display region by a virtual image of an electronic video in a visual field of a user;
an image pickup device configured to pick up an image of an image pickup range including a part or a whole of visual field of the user;
at least one mechanism of a first mechanism configured to change a position of the virtual display region in the visual field of the user and a second mechanism configured to change an image pickup direction of the image pickup device, and
a processor, wherein
the processor determines, with signal processing for a picked-up image by the image pickup device, an observation necessary range in the picked-up image and performs display control for displaying, in the virtual display region, an image portion of the observation necessary range in the picked-up image,
the processor detects a fingertip of the user's left hand and a fingertip of the user's right hand and sets a range surrounded by the user's left hand fingertip and right hand fingertip as the observation necessary range, and
the processor determines an act of holding-up the user's hands as a determination condition for the observation necessary range.

3. The head-mounted display apparatus according to claim 2 , wherein the processor determines, as the observation necessary range, an image portion including a work object of the user.

4. The head-mounted display apparatus according to claim 2 , wherein the processor determines a work object of the user through an image analysis for the picked-up image and determines, as the observation necessary range, an image portion including the work object of the user.

5. The head-mounted display apparatus according to claim 2 , further comprising a work database including information concerning work of the user, wherein
the processor specifies a work object based on an image analysis for the picked-up image and information of the work database and determines, as the observation necessary range, an image portion including the specified work object.

6. The head-mounted display apparatus according to claim 2 , wherein the processor performs, using a predetermined inference model, inference of the observation necessary range based on the inputted picked-up image.

7. An inspection supporting display system comprising:
a head-mounted display apparatus including:
a display device configured to form a virtual display region by a virtual image of an electronic video in a visual field of a user; and
an image pickup device configured to pick up an image of an image pickup range including a work target of the user; and

a processor, wherein
the processor determines, with signal processing for a picked-up image by the image pickup device, an observation necessary range including the work target in the picked-up image and performs display control for enlarging and displaying an image portion of the observation necessary range in the picked-up image, and
the processor detects a fingertip of the user's left hand and a fingertip of the user's right hand and sets a range surrounded by the user's left hand fingertip and right hand fingertip as the observation necessary range.

8. A display method comprising:
forming a virtual display region by a virtual image of an electronic video in a visual field of a user;
picking up an image of an image pickup range including a part or a whole of visual field of the user;
determining, with signal processing for a picked-up image acquired by the image pickup, an observation necessary range in the picked-up image;
performing display control for displaying, in the virtual display region, an image portion of the observation necessary range in the picked-up image, and
detecting a fingertip of the user's left hand and a fingertip of the user's right hand and setting a range surrounded by the user's left hand fingertip and right hand fingertip as the observation necessary range.

9. A non-transitory computer-readable recording medium recording a display program for causing a computer to:
form a virtual display region by a virtual image of an electronic video in a visual field of a user;
pick up an image of an image pickup range including a part or a whole of visual field of the user; determine, with signal processing for a picked-up image acquired by the image pickup, an observation necessary range in the picked-up image;
execute display control for displaying, in the virtual display region, an image portion of the observation necessary range in the picked-up image, and
detect a fingertip of the user's left hand and a fingertip of the user's right hand and set a range surrounded by the left hand fingertip and right hand fingertip as the observation necessary range.

Assignees

• Olympus Corp

Inventors

• Uemura Tatsuyuki
• Endo Hirofumi
• Shibasaki Takao
• Osanai Yoji
• Tatsuta Seiji

Classifications

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G02B27/017 Primary

  Head mounted · CPC title

• G06T2207/30121

  CRT, LCD or plasma display · CPC title

• G06T7/0012

  Biomedical image inspection · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title