Combined computer vision and human vision camera system

Patent metadata

Field	Value
Publication number	US-11936975-B2
Application number	US-202117318109-A
Country	US
Kind code	B2
Filing date	May 12, 2021
Priority date	May 12, 2021
Publication date	Mar 19, 2024
Grant date	Mar 19, 2024

Abstract

Embodiments include providing a computer vision video stream and a human vision video stream using a single camera. The camera can comprise a single, high-resolution senor and a wide-angle lens providing high resolution video having a wide aspect ratio field of view. The streams can be generated, wherein the computer vision video stream maintains the wide aspect ratio field of view and the human vision video stream comprises high resolution video. The computer vision video stream can be provided to a computer vision system, wherein the computer vision system uses the computer vision video stream as input for an automated process, and the human vision video stream can be provided to a system that displays the human vision video stream to a user.

First claim

What is claimed is:

1. A method for providing a computer vision video stream and a human vision video stream using a single camera, the method comprising:
receiving, by a video processing system, from the single camera, a video stream, wherein the single camera comprises a single, high-resolution senor and a wide-angle lens providing high resolution video having a wide aspect ratio field of view;
cropping, by the video processing system, each frame of the human vision video stream from a center of a corresponding frame of the video stream received from the single camera, wherein the single camera comprises one of a plurality of cameras of a vehicle, each camera installed in a different location on the vehicle, wherein video from each camera is cropped, and wherein an amount of the cropping, an aspect ratio of the cropping, and a center of the video stream received from each camera is different for each camera and are optimized for the human vision video stream;
generating, by the video processing system, the computer vision video stream and the human vision video stream from the cropping of each frame of the human vision video stream from the center of the corresponding frame of the video stream received from the single camera, wherein the computer vision video stream maintains the wide aspect ratio field of view and the human vision video stream comprises high resolution video;
providing, by the video processing system, the computer vision video stream to a computer vision system, wherein the computer vision system uses the computer vision video stream as input for an automated process; and
providing, by the video processing system, the human vision video stream to a system that displays the human vision video stream to a user.

2. The method of claim 1 , wherein generating the human vision video stream further comprises determining whether to generate the human vision video stream and wherein cropping each frame of the human vision video stream from a center of a corresponding frame of the video stream received from the single camera is performed only in response to determining to generate the human vision video stream.

3. The method of claim 1 , wherein generating the computer vision video stream further comprises post-processing the computer vision video stream based on requirements of the computer vision system.

4. The method of claim 1 , wherein generating the human vision video stream further comprises post-processing the human vision video stream based on requirements of the system that displays the human vision video stream.

5. The method of claim 1 , wherein the computer vision system comprises an Advanced Driver-Assistance System (ADAS) of an automobile.

6. The method of claim 1 , wherein the system that displays the human vision video stream comprises a driver assist system of an automobile.

7. The method of claim 1 , wherein the automated process comprises autonomous operation of a vehicle.

8. A video processing system comprising:
a camera;
a processor coupled with the camera; and
a memory coupled with and readable by the processor and storing therein a set of instructions which, when executed by the processor, causes the processor to provide a computer vision video stream and a human vision video stream using a single input video stream from the camera by:
receiving, from the camera, the single input video stream, wherein the camera comprises a single, high-resolution senor and a wide-angle lens providing high resolution video having a wide aspect ratio field of view;
cropping each frame of the human vision video stream from a center of a corresponding frame of the single input video stream received from a single camera, wherein the single camera comprises one of a plurality of cameras of a vehicle, each camera installed in a different location on the vehicle, wherein video from each camera is cropped, and wherein an amount of the cropping, an aspect ratio of the cropping, and a center of the single input video stream received from each camera is different for each camera and are optimized for the human vision video stream;
generating, from the received single input stream, the computer vision video stream and the human vision video stream from the cropping of each frame of the human vision video stream from the center of the corresponding frame of the single input video stream received from the single camera, wherein the computer vision video stream maintains the wide aspect ratio field of view and the human vision video stream comprises high resolution video,

providing the computer vision video stream to a computer vision system, wherein the computer vision system uses the computer vision video stream as input for an automated process; and
providing the human vision video stream to a system that displays the human vision video stream to a user.

9. The video processing system of claim 8 , wherein generating the human vision video stream further comprises determining whether to generate the human vision video stream and wherein cropping each frame of the human vision video stream from a center of a corresponding frame of the single input stream received from the camera is performed only in response to determining to generate the human vision video stream.

10. The video processing system of claim 9 , wherein generating the computer vision video stream further comprises post-processing the computer vision video stream based on requirements of the computer vision system.

11. The video processing system of claim 9 , wherein generating the human vision video stream further comprises post-processing the human vision video stream based on requirements of the system that displays the human vision video stream.

12. The video processing system of claim 9 , wherein the computer vision system comprises an Advanced Driver-Assistance System (ADAS) of an automobile.

13. The video processing system of claim 9 , wherein the system that displays the human vision video stream comprises a parking assist system of an automobile.

14. A vehicle comprising:
a camera;
an Advanced Driver-Assistance System (ADAS);
a parking assist system; and
a video processing system comprising a processor coupled with the camera, the ADAS, and the parking assist system and a memory coupled with and readable by the processor and storing therein a set of instructions which, when executed by the processor, causes the processor to provide a computer vision video stream and a human vision video stream using a single input video stream from the camera by:
receiving, from the camera, the single input video stream, wherein the camera comprises a single, high-resolution senor and a wide-angle lens providing high resolution video having a wide aspect ratio field of view;
cropping each frame of the human vision video stream from a center of a corresponding frame of the single input video stream received from a single camera, wherein the single camera comprises one of a plurality of cameras of a vehicle, each camera installed in a different location on the vehicle, wherein video from each camera is cropped, and wherein an amount of the cropping, an aspect ratio of the cropping, and a center of the single input video stream received from each camera is different for each camera and are optimized for the human vision video stream;
generating, from the received single input stream, the computer vision video stream and the human vision video stream from the cropping of each frame of the human vision video stream from the center of the corresponding frame of the single input video stream received from the single camera, wherein the computer vision video stream maintains the

Assignees

• Nio Technology Anhui Co Ltd

Inventors

• Samson Matthew L
• Angier Bradley J
• Agrawal Samir
• Chen Waylon Y
• Chen Luna

Classifications

• H04N23/58 Primary

  Means for changing the camera field of view without moving the camera body, e.g. nutating or panning of optics or image sensors · CPC title

• G06V20/56 Primary

  exterior to a vehicle by using sensors mounted on the vehicle · CPC title

• G01S13/931

  of land vehicles · CPC title

• G01S17/931

  of land vehicles · CPC title

• G06T5/00

  Image enhancement or restoration · CPC title