Image processing system, optical sensor, and learning apparatus with irregular lens array

Patent metadata

Field	Value
Publication number	US-11087498-B2
Application number	US-201716478867-A
Country	US
Kind code	B2
Filing date	Mar 9, 2017
Priority date	Feb 1, 2017
Publication date	Aug 10, 2021
Grant date	Aug 10, 2021

Abstract

An image processing system according to an aspect of the invention includes: an optical sensor including: an optical member configured to refract light from a subject, the optical member including a plurality of transmissive units that have different optical properties, thus forming different refraction patterns of light that passes therethrough; and a plurality of image sensors configured to receive light that has passed through the transmissive units and to form captured images in which the subject appears; and an information processing apparatus configured to calculate attribute information representing an attribute of the subject by inputting the captured images obtained by the image sensors to a learning device that has learned the attribute of the subject.

First claim

The invention claimed is:

1. An image processing system comprising:
an optical sensor comprising: an optical member configured to refract light from a subject, the optical member comprising a plurality of transmissive units that have different optical properties, thus forming different refraction patterns of light that passes therethrough; and a plurality of image sensors configured to receive light that has passed through the transmissive units and to form captured images in which the subject appears, the plurality of image sensors being provided in correspondence with the transmissive units; and
a processor configured with a program to perform operations comprising operation as an information processing apparatus configured to acquire attribute information representing an attribute of the subject by inputting the captured images obtained by the image sensors to a trained learning device that has been trained to determine the attribute of the subject, wherein
the plurality of transmissive units in the optical member comprise a plurality of first transmissive units arranged at a center of the plurality of transmissive units in a rectangular shape and that have a predetermined first converging angle, and second transmissive units that are arranged forming edges of a hollow rectangular shape so as to surround a periphery of the plurality of first transmissive units and that have a second converging angle that is greater than the first converging angle.

2. The image processing system according to claim 1 , wherein the first converging angle is 2.0 degrees at most, and the second converging angle is at least 20 degrees.

3. The image processing system according to claim 1 , wherein the learning device is constituted by a learning device that is trained using a neural network, a support vector machine, a self-organizing map, or reinforcement learning.

4. The image processing system according to claim 1 ,
wherein the optical member comprises at least one of a lens array comprising a plurality of lenses, a diffraction grating, a scattering lens, and a holographic lens.

5. The image processing system according to claim 1 , wherein
the optical member comprises a lens array comprising a plurality of lenses as the plurality of transmissive units, and
the optical properties of the transmissive units are realized by at least one of a size, a material, a shape, and a color of the lenses.

6. The image processing system according to claim 1 ,
wherein the processor is configured with the program to perform operations such that operation as the information processing apparatus further comprises outputting, as the attribute information, one of, or a combination of, a position, a direction, an orientation, a size, a shape, a motion, a type, individual identification information, a color, a brightness, and environmental information of the subject.

7. The image processing system according to claim 1 ,
wherein the processor is configured with the program to perform operations such that operation as the information processing apparatus further comprises creating an image depicting the subject based on output from the learning device, and outputting the created image as the attribute information.

8. An image processing system comprising:
an optical sensor comprising: an optical member configured to refract light from a subject, the optical member comprising a plurality of transmissive units that have different optical properties, thus forming different refraction patterns of light that passes therethrough; and a plurality of image sensors configured to receive light that has passed through the transmissive units and to form captured images in which the subject appears, the plurality of image sensors being provided in correspondence with the transmissive units; and
an information processing apparatus configured to acquire attribute information representing an attribute of the subject by inputting the captured images obtained by the image sensors to a trained learning device that has been trained to determine the attribute of the subject,
wherein the optical properties of the plurality of transmissive units are irregular, and the plurality of transmissive units in the optical member comprises a plurality of first transmissive units arranged at a center of the plurality of transmissive units in a rectangular shape and that have a predetermined first converging angle, and second transmissive units that are arranged forming edges of a hollow rectangular shape so as to surround a periphery of the plurality of first transmissive units and that have a second converging angle that is greater than the first converging angle.

9. An optical sensor comprising:
an optical member configured to refract light from a subject, the optical member comprising a plurality of transmissive units that have different optical properties, thus forming different refraction patterns of light that passes therethrough; and
a plurality of image sensors configured to receive light that has passed through the transmissive units and to form captured images in which the subject appears, the plurality of image sensors being provided in correspondence with the transmissive units, wherein
the plurality of transmissive units in the optical member comprise a plurality of first transmissive units arranged at a center of the plurality of transmissive units in a rectangular shape and have a predetermined first converging angle, and second transmissive units that are arranged forming edges of a hollow rectangular shape so as to surround a periphery of the plurality of first transmissive units and have a second converging angle that is greater the first converging angle.

10. The optical sensor according to claim 9 ,
wherein the first converging angle is 2.0 degrees at most, and the second converging angle is at least 20 degrees.

11. The optical sensor according to claim 9 ,
wherein the optical member comprises at least one of a lens array comprising a plurality of lenses, a diffraction grating, a scattering lens, and a holographic lens.

12. The optical sensor according to claim 9 , wherein
the optical member comprises a lens array comprising a plurality of lenses as the plurality of transmissive units, and
the optical properties of the transmissive units are realized by at least one of a size, a material, a shape, and a color of the lenses.

13. A learning apparatus comprising a processor configured with a program to perform operations comprising:
operation as a learning image acquisition unit configured to acquire the captured images captured by the image sensors from the optical sensor according to claim 9 ; and
operation as a learning processing unit configured to train a learning device to output attribute information regarding the subject in response to the acquired captured images being input.

14. The learning apparatus according to claim 13 ,
wherein the processor is configured with the program such that operation as the learning processing unit further comprises training the learning device to output, as the attribute information, one of, or a combination of, a position, a direction, an orientation, a size, a shape, a motion, a type, individual identification information, a color, a brightness, environmental information of the subject, and an image depicting the subject.

15. An optical sensor comprising:
an optical member configured to refract light from a subject, the optical member comprising a plurality of transmissive units that have different optical properties, thus forming different refraction patterns of light that passes therethrough; and
a plurality

Assignees

• Omron Tateisi Electronics Co

Inventors

• Ando Tanichi

Classifications

• G06T7/97 Primary

  Determining parameters from multiple pictures (depth or shape recovery from multiple images G06T7/55; stereo camera calibration G06T7/85) · CPC title

• G06V20/64

  Three-dimensional [3D] objects · CPC title

• G06V10/454

  Integrating the filters into a hierarchical structure, e.g. convolutional neural networks [CNN] · CPC title

• G06V10/147 Primary

  Details of sensors, e.g. sensor lenses (fingerprint or palmprint sensors G06V40/13; vascular sensors G06V40/145; eye sensors G06V40/19) · CPC title

• G06V10/764

  using classification, e.g. of video objects · CPC title