Movement assistance system and movement assistance method for controlling output of position estimation result

Patent metadata

Field	Value
Publication number	US-11812925-B2
Application number	US-202017079610-A
Country	US
Kind code	B2
Filing date	Oct 26, 2020
Priority date	Apr 26, 2018
Publication date	Nov 14, 2023
Grant date	Nov 14, 2023

Abstract

An movement assistance system includes a processor. The processor detects a position in picked-up images of an image of an image pickup target, causes a memory to store the position, estimates a change in the position of the image pickup target, estimates the position of the image pickup target and performs control for causing a display apparatus to display an estimation result of the position of the image pickup target, when detecting a disappearance state in which the image of the image pickup target is not detected, and performs control for causing the display apparatus not to display the estimation result of the position of the image pickup target when detecting, after the image pickup target is brought into the disappearance state, that a position indicated by the estimation result of the position of the image pickup target is a position inside an output determination region.

First claim

What is claimed is:

1. A movement assistance system comprising:
a memory; and
a processor configured to:
receive picked-up images acquired by an image pickup apparatus fixed to a mobile body, detect an image of an image pickup target included in each of the picked-up images, and detect a position in each of the picked-up images of the detected image of the image pickup target;
cause the memory to store the position in each of the picked-up images of the detected image of the image pickup target;
estimate a change in the position of the image pickup target relative to each of the picked-up images, by detecting a change in the received picked-up images;
estimate the position of the image pickup target relative to each of the picked-up images based on information on the position stored in the memory and an estimation result of the change in the position of the image pickup target, to output a position estimation result, and perform control for causing a display apparatus to display the position estimation result, when detecting a disappearance state in which the image of the image pickup target is not detected on the picked-up images;
perform control not to output the position estimation result, in a case where the image of the image pickup target is not detected, when the position stored in the memory immediately before the image of the image target is brought into the disappearance state indicates a position inside a target disappearance position determination region as a predetermined region in each of the picked-up images; and
perform control for causing the display apparatus not to display the position estimation result, when detecting, after the image of the image pickup target is brought into the disappearance state, that a position indicated by the position estimation result is a position inside an output determination region as a region for determining whether or not to output the position estimation result,
wherein the output determination region is a region inside with respect to a range of each of the pick-up images and the target disappearance position determination region is a region inside with respect to the output determination region.

2. The movement assistance system according to claim 1 ,
wherein when detecting the disappearance state, the processor is configured to:
start estimating the change in the position of the image pickup target relative to the each of the picked-up images; and
estimate the position of the image pickup target relative to each of the picked-up images, based on information on the position stored in the memory immediately before a start of the disappearance state and the estimation result of the change in the position of the image pickup target.

3. The movement assistance system according to claim 1 ,
wherein the mobile body comprises an insertion portion of an endoscope, and
wherein the image pickup apparatus is disposed in a distal end portion of the insertion portion of the endoscope.

4. The movement assistance system according to claim 1 ,
wherein the processor is configured to estimate the change in the position of the image pickup target by an optical flow estimation using the picked-up images.

5. The movement assistance system according to claim 4 ,
wherein the processor is configured to detect, on the picked-up images sequentially received by the processor, a moving amount and a moving direction of a predetermined tracking point in the picked-up images, and outputs a detection result as the estimation result of the change in the position of the image pickup target.

6. The movement assistance system according to claim 4 ,
wherein, when, on the picked-up images sequentially received by the processor, moving trajectories of a plurality of tracking points set in each of the picked-up images respectively form circular-arc shapes having a common rotation center, the processor is configured to determine that the position of the image pickup target changes in a rotation direction around the rotation center, to obtain the estimation result.

7. The movement assistance system according to claim 4 ,
wherein when, on the picked-up images sequentially received by the processor, moving trajectories of a plurality of tracking points set in each of the picked-up images respectively form linear lines extending radially from a common point, the processor is configured to determine that the position of the image pickup target changes in an optical axis direction of the image pickup apparatus, to obtain the estimation result.

8. The movement assistance system according to claim 1 ,
wherein the processor is configured to estimate the change in the position of the image pickup target by machine learning using the picked-up images.

9. The movement assistance system according to claim 1 ,
wherein the processor is configured to cause the display apparatus to display each of the picked-up images and a direction of the image pickup target relative to each of the picked-up images, based on the picked-up images and the position estimation result.

10. The movement assistance system according to claim 1 ,
wherein the processor is configured to cause the display apparatus to display each of the picked-up images and the position of the image pickup target relative to each of the picked-up images, based on the picked-up images and the position estimation result.

11. The movement assistance system according to claim 1 ,
wherein, when detecting that the disappearance state in which the image of the image pickup target is not detected on the picked-up images continues for a time period longer than a predetermined elapsed time, or a distance from each of the picked-up images to the estimated image pickup target is longer than a predetermined direct distance or a predetermined route length, the processor is configured to cause the display apparatus not to display the position estimation result and/or causes the display apparatus to display a sign indicating that a reliability of the position estimation result decreases.

12. The movement assistance system according to claim 1 , further comprising:
a driving control circuit configured to cause the mobile body to displace in a direction of an optical axis of the image pickup apparatus, a direction orthogonal to the direction of the optical axis, and in a rotation direction around the optical axis; and
a driving circuit configured to receive the position estimation result from the processor to generate a control signal for controlling the driving control circuit.

13. A movement assistance method comprising:
receiving picked-up images acquired by an image pickup apparatus fixed to a mobile body, detecting an image of an image pickup target included in each of the picked-up images, and detecting a position in each of the picked-up images of the detected image of the image pickup target;
causing a memory to store the position in each of the picked-up images of the detected image of the image pickup target;
estimating a change in the position of the image pickup target relative to each of the picked-up images, by detecting a change in the picked-up images;
estimating the position of the image pickup target relative to each of the picked-up images based on information on the position stored in the memory and an estimation result of the change in the position of the image pickup target, to output a position estimation result, and performing control for causing a display apparatus to display the position estimation result, when detecting a disappearance state in which the image of the image pickup target is not detected on the picked-up images;
performing control not to ou

Assignees

• Olympus Corp

Inventors

• Tojo Ryo

Classifications

• A61B1/000094 Primary

  extracting biological structures · CPC title

• A61B1/00045

  Display arrangement · CPC title

• A61B1/0051

  with controlled bending of insertion part · CPC title

• A61B1/000096

  using artificial intelligence · CPC title

• G06N20/00

  Machine learning · CPC title