Information processing apparatus and control method and control program thereof, and communication terminal and control method and control program thereof

Patent metadata

Field	Value
Publication number	US-9418314-B2
Application number	US-201314375452-A
Country	US
Kind code	B2
Filing date	Jan 25, 2013
Priority date	Jan 30, 2012
Publication date	Aug 16, 2016
Grant date	Aug 16, 2016

Abstract

A medical article and m-number of first local features which are respectively constituted by a feature vector of 1 dimension to i dimensions of m-number of feature points in an image of the medical article are stored in association with each other, n-number of feature points are extracted from an image in a captured video, n-number of second local features respectively constituted by a feature vector of 1 dimension to j dimensions are generated, a smaller number of dimensions among the number of dimensions i and the number of dimensions j is selected, and an existence of the medical article in the image in the video is recognized when it is determined that a prescribed ratio or more of the m-number of first local features up to the selected number of dimensions corresponds to the n-number of second local features up to the selected number of dimensions.

First claim

We claim:

1. An information processing system, comprising:
first local feature storing unit that stores, in association with each other, a medical article and m-number of first local features which are respectively constituted by a feature vector of 1 dimension to i dimensions and which are generated for each of m-number of local areas including each of m-number of feature points in an image of the medical article;
second local feature generating unit that extracts n-number of feature points from an image of a video captured by imaging unit, and that generates n-number of second local features respectively constituted by a feature vector of 1 dimension to j dimensions for n-number of local areas including each of the n-number of feature points; and
recognizing unit that selects a smaller number of dimensions among the number of dimensions i of the feature vector of the first local feature and the number of dimensions j of the feature vector of the second local feature, and that recognizes that the medical article exists in the image in the video when determining that a prescribed ratio or more of the m-number of first local features constituted by a feature vector up to the selected number of dimensions corresponds to the n-number of second local features constituted by a feature vector up to the selected number of dimensions.

2. The information processing system according to claim 1 , further comprising informing unit that informs a recognition result obtained from the recognizing unit.

3. The information processing system according to claim 2 , comprising a communication terminal carried by a user, and an information processing apparatus that communicates with the communication terminal, wherein
the communication terminal includes the imaging unit, the second local feature generating unit, and the informing unit, and the n-number of second local features are transmitted from the communication terminal to the information processing apparatus, and
the information processing apparatus includes the first local feature storing unit and the recognizing unit, and the recognition result obtained from the recognizing unit is transmitted from the information processing apparatus to the communication terminal.

4. The information processing system according to claim 1 , wherein
the first local feature storing unit stores the m-number of first local features generated from images of a plurality of medical articles in association with each of the medical articles, and
the recognizing unit recognizes a plurality of medical articles included in the image captured by the imaging unit, and includes arrangement determining unit that determines an arrangement of the plurality of medical articles in the image captured by the imaging unit based on an alignment of the n-number of second local features.

5. The information processing system according to claim 4 , wherein
the medical article is a medical device and an image captured by the imaging unit is of an examination room, a hospital room, or an operation room,
the arrangement determining unit recognizes an arrangement of the medical device in the examination room, the hospital room, or the operation room,
the second local feature generating unit includes accuracy adjusting unit that adjusts accuracy of the second local feature, and
the recognizing unit further recognizes a mistake, a defect, or a state of the medical device based on a second local feature generated by the second local feature generating unit by adjusting to a higher accuracy.

6. The information processing system according to claim 4 , wherein
the medical article is a medical instrument and an image captured by the imaging unit is of a tray on which the medical instrument is arranged,
the arrangement determining unit recognizes an arrangement of the medical instrument on the tray,
the second local feature generating unit includes accuracy adjusting unit that adjusts accuracy of the second local feature, and
the recognizing unit further recognizes a mistake, a defect, or a state of the medical instrument based on a second local feature generated by the second local feature generating unit by adjusting to a higher accuracy.

7. The information processing system according to claim 4 , wherein
the medical article is a pharmaceutical product and an image captured by the imaging unit is of a medicine shelf or a medicine tray,
the arrangement determining unit recognizes an arrangement of the pharmaceutical product on the medicine shelf or the medicine tray,
the second local feature generating unit includes accuracy adjusting unit that adjusts accuracy of the second local feature, and
the recognizing unit further recognizes a mistake, a defect, or a state of the pharmaceutical product based on a second local feature generated by the second local feature generating unit by adjusting to a higher accuracy.

8. The information processing system according to claim 7 , further comprising managing unit that performs inventory based on an arrangement of the plurality of pharmaceutical products recognized by the arrangement determining unit.

9. The information processing system according to claim 1 , wherein the first local feature and the second local feature are each generated by dividing a local area including a feature point extracted from an image into a plurality of sub-areas and generating a feature vector of a plurality of dimensions constituted by a histogram in a gradient direction in the plurality of sub-areas.

10. The information processing system according to claim 9 , wherein the first local feature and the second local feature are each generated by selecting a dimension at which a correlation between adjacent sub-areas is lower among the generated feature vector of a plurality of dimensions.

11. The information processing system according to claim 9 , wherein the plurality of dimensions of the feature vector are arranged to circle the local area once for every predetermined number of dimensions so that dimensions can be selected starting from a first dimension in a descending order of contributions to the feature point and in accordance with an increase in accuracy that is required with respect to the local feature.

12. The information processing system according to claim 11 , wherein the second local feature generating unit generates the second local feature corresponding to a correlation of the medical articles so that the second local feature with a larger number of dimensions is generated for a medical article that has a higher correlation with another medical article.

13. The information processing system according to claim 11 , wherein the first local feature storing unit stores the first local feature corresponding to a correlation of the medical articles so that the first local feature with a larger number of dimensions is stored for a medical article that has a higher correlation with another medical article.

14. An information processing method in an information processing system including first local feature storing unit that stores, in association with each other, a medical article and m-number of first local features which are respectively constituted by a feature vector of 1 dimension to i dimensions and which are generated for each of m-number of local areas including each of m-number of feature points in an image of the medical article, the method comprising the steps of:
extracting n-number of feature points from an image in a captured video and generating n-number of second local features respectively constituted by a feature vector of 1 dimension to j dimensions for n-number of local areas including each of th

Assignees

• Nec Corp

Inventors

• Nomura Toshiyuki
• Yamada Akio
• Iwamoto Kota
• Mase Ryota

Classifications

• G06V20/66

  Trinkets, e.g. shirt buttons or jewellery items (recognising microscopic objects G06V20/69) · CPC title

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• G16H40/20

  for the management or administration of healthcare resources or facilities, e.g. managing hospital staff or surgery rooms · CPC title

• G06V10/42

  Global feature extraction by analysis of the whole pattern, e.g. using frequency domain transformations or autocorrelation · CPC title

• G06V10/462

  Salient features, e.g. scale invariant feature transforms [SIFT] · CPC title