Medicine identification system, medicine identification device, medicine identification method, and program

Patent metadata

Field	Value
Publication number	US-2021312219-A1
Application number	US-202117349927-A
Country	US
Kind code	A1
Filing date	Jun 17, 2021
Priority date	Feb 8, 2019
Publication date	Oct 7, 2021
Grant date	—

Abstract

A medicine identification system and a medicine identification method can favorable identification of a medicine using a camera-equipped mobile terminal. A smartphone including a camera changes a light-emitting region on a display, changes an illuminating direction with respect to a medicine placed on a tray having an indicator used for standardizing size and shape of the medicine, and images the medicine with different illuminating directions with the camera. The medicine identification device standardizes the size and shape of first images acquired by imaging based on the indicators captured in the first images, and extracts a medicine region image from each of the standardized first images. The medicine identification device then generates, based on the extracted medicine region images, a second image having undergone an emphasis process for emphasizing an embossed mark or a printed mark provided on the medicine, and identifies the medicine based on the generated second image.

First claim

What is claimed is:

1 . A medicine identification system comprising:
a tray configured to have an indicator used for standardizing size and shape of a medicine, on which a medicine to be identified is to be placed; a camera-equipped mobile terminal configured to include a camera on the same face as a display; and a medicine identification device configured to identify the medicine based on an image taken by the camera-equipped mobile terminal, wherein the camera-equipped mobile terminal is provided with: a display control unit configured to, upon reception of an instruction input for medicine identification, change a light-emitting region on the display and change for a plurality of times an illuminating direction with respect to the medicine placed on the tray; and an image acquiring unit configured to cause the camera to image for a plurality of times the medicine with different illuminating directions through changing of the light-emitting region on the display, to thereby acquire a plurality of first images of the medicine, and the medicine identification device is provided with: a standardizing unit configured to standardize each of the plurality of first images based on the indicator captured in the plurality of first images; an image extracting unit configured to extract a medicine region image from each of the plurality of first images thus standardized; an image processing unit configured to generate, based on a plurality of medicine region images thus extracted, a second image having undergone an emphasis process for emphasizing an embossed mark or a printed mark provided on the medicine; and a medicine identifying unit configured to identify the medicine based on the second image thus generated.

2 . The medicine identification system according to claim 1 , wherein the image processing unit acquires a plurality of edge images from the plurality of medicine region images thus extracted by using edge extraction filters of respective directions corresponding to the illuminating directions, and generates the second image by using the plurality of edge images.

3 . The medicine identification system according to claim 1 , wherein:
the tray has a grid-shaped partition of which size and shape are known; the medicine is placed on a placement face partitioned by the grid-shaped partition; and the grid-shaped partition serves also as the indicator.

4 . The medicine identification system according to claim 3 , wherein color of the placement face of the tray for the medicine partitioned by the grid-shaped partition is different in at least one of hue, brightness, and saturation, for every placement face or for every plural placement faces.

5 . The medicine identification system according to claim 1 , wherein:
the image acquiring unit acquires, as the plurality of first images, three or more images with different illuminating directions and an image in a state in which the display is turned off; and the medicine identification device is provided with a generating unit configured to subtract the image in a state in which the display is turned off from each of the three or more images, to generate the plurality of first images with elimination of influence of ambient light.

6 . The medicine identification system according to claim 1 , wherein:
the display includes a polarizing plate; and the camera includes a polarizing plate installed in front of an imaging lens, a polarization direction of the polarizing plate being different by 90° from a polarization direction of light emitted from the display.

7 . The medicine identification system according to claim 1 , wherein the medicine identifying unit includes: a feature quantity calculating unit configured to calculate a feature quantity of the second image; and an inferring unit configured to infer which one of registered medicines resembles the medicine based on the feature quantity of the second image.

8 . The medicine identification system according to claim 7 , wherein the feature quantity calculating unit and the inferring unit comprise a learned convolutional neural network obtained through learning of each of the registered medicines by using as teacher data the second images corresponding to the registered medicines and medicine identification information for identifying the registered medicines.

9 . The medicine identification system according to claim 7 , wherein the medicine identifying unit identifies the medicine through template matching between images of the registered medicines narrowed down by inference by the inferring unit and the second image.

10 . The medicine identification system according to claim 1 , wherein the tray serves also as a protection cover for protecting the camera-equipped mobile terminal.

11 . A medicine identification device for identifying a medicine to be identified based on an image taken by a camera-equipped mobile terminal,
wherein the camera-equipped mobile terminal is configured to: include a camera on the same face as a display; when imaging the medicine to be identified placed on a tray having an indicator used for standardizing size and shape of a medicine, change a light-emitting region on the display; change for a plurality of times an illuminating direction with respect to the medicine placed on the tray; and image by means of the camera for a plurality of times the medicine with different illuminating directions through changing of the light-emitting region on the display, to acquire a plurality of first images of the medicine, the medicine identification device comprising: an image accepting unit configured to accept the plurality of first images of the medicine from the camera-equipped mobile terminal; a standardizing unit configured to standardize each of the plurality of first images based on the indicator captured in the plurality of first images; an image extracting unit configured to extract a medicine region image from each of the plurality of first images thus standardized; an image processing unit configured to generate, based on a plurality of medicine region images thus extracted, a second image having undergone an emphasis process for emphasizing an embossed mark or a printed mark provided on the medicine; and a medicine identifying unit configured to identify the medicine based on the second image thus generated.

12 . A medicine identification method for identifying a medicine to be identified placed on a tray having an indicator used for standardizing size and shape of a medicine, comprising:
a first step in which a display control unit changes a light-emitting region on a display of a camera-equipped mobile terminal including a camera on the same face as the display, to thereby change for a plurality of times an illuminating direction with respect to the medicine placed on the tray; a second step in which an image acquiring unit causes the camera to image for a plurality of times the medicine with different illuminating directions through changing of the light-emitting region on the display, to thereby acquire a plurality of first images of the medicine; a third step in which a standardizing unit standardizes each of the plurality of first images based on the indicator captured in the plurality of first images; a fourth step in which an image extracting unit extracts a medicine region image from each of the plurality of first images thus standardized; a fifth step in which an image processing unit generates, based on a plurality of medicine region images thus extracted, a second image having undergone an emphasis process for emphasizing an embossed mark or a printed mark provided on the medicine; and a si

Assignees

• Fujifilm Toyama Chemical Co Ltd

Inventors

• Iwami Kazuchika

Classifications

• G06V20/10 Primary

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06V20/66

  Trinkets, e.g. shirt buttons or jewellery items (recognising microscopic objects G06V20/69) · CPC title

• G16H40/63

  for local operation · CPC title

• G16H20/10

  relating to drugs or medications, e.g. for ensuring correct administration to patients · CPC title

• G06T7/174

  involving the use of two or more images · CPC title