Medical observation system, information processing apparatus, and information processing method

Patent metadata

Field	Value
Publication number	US-12573203-B2
Application number	US-202218681549-A
Country	US
Kind code	B2
Filing date	Mar 25, 2022
Priority date	Aug 13, 2021
Publication date	Mar 10, 2026
Grant date	Mar 10, 2026

Abstract

A medical observation system includes: an imaging unit that acquires an operative field image including an abdominal cavity environment; a display unit that displays the operative field image acquired by the imaging unit; a gaze information acquisition unit that acquires information regarding gaze of a user on the abdominal cavity environment of the operative field image displayed by the display unit; a target candidate recognition unit that recognizes a tracking target candidate from the operative field image acquired by the imaging unit; a reliability acquisition unit that acquires recognition reliability of the tracking target candidate recognized by the target candidate recognition unit; and a target determination unit that determines a tracking target in the abdominal cavity environment from the tracking target candidate on the basis of the information regarding the tracking target candidate, the recognition reliability, and the information regarding the gaze.

First claim

The invention claimed is:

1 . A medical observation system comprising:
an imaging unit that acquires an operative field image including an abdominal cavity environment; a display unit that displays the operative field image acquired by the imaging unit; a gaze information acquisition unit that acquires information regarding gaze of a user on the abdominal cavity environment of the operative field image displayed by the display unit; a target candidate recognition unit that recognizes a tracking target candidate from the operative field image acquired by the imaging unit; a reliability acquisition unit that acquires recognition reliability of the tracking target candidate recognized by the target candidate recognition unit; and a target determination unit that determines a tracking target in the abdominal cavity environment from the tracking target candidate on a basis of the information regarding the tracking target candidate recognized by the target candidate recognition unit, the recognition reliability acquired by the reliability acquisition unit, and the information regarding the gaze acquired by the gaze information acquisition unit.

2 . The medical observation system according to claim 1 , wherein
the target determination unit determines the tracking target from the tracking target candidate according to a predetermined threshold of the recognition reliability preset.

3 . The medical observation system according to claim 2 , wherein
the target determination unit uses the information regarding the gaze in a case where the recognition reliability acquired by the reliability acquisition unit is less than the predetermined threshold.

4 . The medical observation system according to claim 1 , wherein
the information regarding the tracking target candidate includes information regarding a recognition region of the tracking target candidate, the information regarding the gaze includes information regarding a gaze region of the user, and the target determination unit determines the tracking target according to an overlapping area of the recognition region and the gaze region.

5 . The medical observation system according to claim 4 , wherein
the target determination unit calculates an overlapping area between the recognition region and the gaze region for each of the tracking target candidate, and determines the tracking target candidate having the overlapping area being largest as the tracking target.

6 . The medical observation system according to claim 1 , further comprising
an input unit that receives information regarding a voice of the user and an operation of the tracking target candidate.

7 . The medical observation system according to claim 6 , wherein
the target determination unit determines the tracking target on a basis of at least one of the information regarding the voice of the user or the information regarding the operation of the tracking target candidate.

8 . The medical observation system according to claim 7 , wherein
the information regarding the voice of the user is a correction request, and the target determination unit determines the tracking target candidate having the recognition reliability being higher as the tracking target in response to the correction request.

9 . The medical observation system according to claim 1 , wherein
the target determination unit determines continuation and stop of processing of determining the tracking target according to whether or not the tracking target is at a predetermined expected position.

10 . The medical observation system according to claim 1 , wherein
the target determination unit determines the tracking target on a basis of the information regarding the gaze in a case where the target determination unit cannot acquire the information regarding the tracking target candidate.

11 . The medical observation system according to claim 1 , wherein
the target candidate recognition unit recognizes the tracking target candidate on a basis of the information regarding the gaze.

12 . The medical observation system according to claim 1 , wherein
the recognition reliability is a probability indicating whether or not the tracking target candidate is within a predetermined target region.

13 . The medical observation system according to claim 1 , wherein
the recognition reliability includes recognition reliability with respect to a position in a planar direction or a depth direction of the tracking target candidate.

14 . The medical observation system according to claim 1 , wherein
the recognition reliability includes recognition reliability for a type of the tracking target candidate.

15 . The medical observation system according to claim 1 , wherein
the display unit displays information for identifying a plurality of the tracking target candidate in superposition with the operative field image.

16 . The medical observation system according to claim 1 , wherein
the display unit displays a position adjustment image for adjusting a position in a planar direction or a depth direction of the tracking target determined by the target determination unit so as to be superimposed on the operative field image.

17 . The medical observation system according to claim 16 , wherein
the display unit displays the position adjustment image in accordance with the recognition reliability of the tracking target candidate recognized by the target candidate recognition unit.

18 . The medical observation system according to claim 1 , further comprising:
an input unit that receives information regarding a voice of the user or an input operation of the user; and a control unit that switches an automatic tracking state of tracking the tracking target, a voice input state of receiving a voice of the user, and an automatic tracking stop state in which tracking of the tracking target is stopped according to the information regarding the voice or the input operation.

19 . The medical observation system according to claim 1 , further comprising:
an arm portion that supports and moves the imaging unit; and an arm control unit that controls the arm portion such that the tracking target determined by the target determination unit is positioned in the operative field image.

20 . The medical observation system according to claim 19 , further comprising
an input unit that receives information regarding an operation of the arm portion, wherein the arm control unit controls the arm portion on a basis of the information regarding the operation of the arm portion.

21 . The medical observation system according to claim 1 , further comprising
an image processing unit that generates the operative field image such that the tracking target determined by the target determination unit is positioned in the operative field image.

22 . An information processing apparatus comprising:
a target candidate recognition unit that recognizes a tracking target candidate from an operative field image acquired by an imaging unit that acquires the operative field image including an abdominal cavity environment; a reliability acquisition unit that acquires recognition reliability of the tracking target candidate recognized by the target candidate recognition unit; a gaze information acquisition unit that acquires information regarding gaze of a user on the abdominal cavity environment of the operative field image displayed by a display unit that displays the operative field image acquired

Assignees

• Sony Group Corp

Inventors

• Taki Yuhei
• Kuroda Yohei
• Arai Jun
• Amano Kohei
• Inagaki Takeo
• Yasuda Ryouhei

Classifications

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• A61B1/3132

  for laparoscopy · CPC title

• A61B1/00045

  Display arrangement · CPC title

• A61B1/00009

  of image signals during a use of endoscope · CPC title

• H04N23/695

  Control of camera direction for changing a field of view, e.g. pan, tilt or based on tracking of objects · CPC title