Methods and apparatus related to capturing and/or rendering images

Patent metadata

Field	Value
Publication number	US-2016373734-A1
Application number	US-201615255135-A
Country	US
Kind code	A1
Filing date	Sep 1, 2016
Priority date	Feb 24, 2015
Publication date	Dec 22, 2016
Grant date	—

Abstract

Camera and/or lens calibration information is generated as part of a calibration process in video systems including 3-dimensional (3D) immersive content systems. The calibration information can be used to correct for distortions associated with the source camera and/or lens. A calibration profile can include information sufficient to allow the system to correct for camera and/or lens distortion/variation. This can be accomplished by capturing a calibration image of a physical 3D object corresponding to the simulated 3D environment, and creating the calibration profile by processing the calibration image. The calibration profile can then be used to project the source content directly into the 3D viewing space while also accounting for distortion/variation, and without first translating into an intermediate space (e.g., a rectilinear space) to account for lens distortion.

First claim

What is claimed:

1 . A camera calibration system including:
a target apparatus having a known three dimensional (3D) shape and including markers on an interior surface of said target apparatus at known locations; a camera calibration apparatus including:
a camera interface for receiving an image of the surface of said target apparatus from a first camera being calibrated; and
a camera calibrator configured to compare actual locations of the markers in said image to expected locations of said markers and to generate a set of camera calibration information based on differences between the actual locations of the markers in said image and the expected locations of said markers.

2 . The camera calibration system of claim 1 , wherein said target apparatus includes four flat sides and a back, each of said flat sides and back including markers.

4 . The camera calibration system of claim 1 , wherein said target apparatus is an open box, the camera calibration system further comprising:
a camera mounting platform; and rails for support said camera mounting platform.

5 . The camera calibration system of claim 4 , further comprising:
a position drive motor; a drive shaft coupled to said camera mounting platform and to said position drive motor; and a camera position controller for controlling the position drive motor based on one or more images captured by the first camera while positioned on said camera mounting platform to position the first camera at a predetermined location in front of said target apparatus.

6 . The camera calibration system of claim 1 , wherein said predetermined location is a location positioned midway between a first side and a second side of said target apparatus.

7 . The camera calibration system of claim 6 , wherein said target apparatus is square, has four sides of the same length and wherein said predetermined location is at the center of said square, said back of said target apparatus including a center portion marker which can to facilitate camera alignment with said center portion marker.

8 . The camera calibration apparatus of claim 7 , wherein each of the sides of said target apparatus are at least two feet long.

9 . The camera calibration apparatus of claim 8 , wherein the sides and back of said target apparatus are made of a rigid sheet material and arranged perpendicular to each other.

10 . The camera calibration system of claim 1 , wherein each of the markers is part of a test pattern, each marker being an intersection of points in a grid formed by a pattern of squares.

11 . The camera calibration system of claim 1 , wherein the camera calibrator generates first UV map correction information indicating one or more adjustments to be made to the location of nodes in a UV map based on the differences between the actual locations of the markers in said first image and the expected locations of said markers.

12 . The camera calibration system of claim 1 , wherein the expected location of the markers is stored in information in memory and known based on properties of at least a lens included in the first camera.

13 . The camera calibration system of claim 11 , wherein the first UV map correction information includes information indicating a U offset and a V offset to be applied to a first node in the UV map, said UV map being a map to be used by a playback device to map an image captured by said first camera to a surface of a mesh model.

14 . The camera calibration system of claim 13 , wherein the calibration apparatus further includes:
a network interface for communicating the UV map, the first UV correction information, and the mesh model to a playback device for use in rendering images captured by the first camera.

15 . A method comprising:
operating a first camera to capture an image of an interior surface of a target apparatus having a known three dimensional (3D) shape and including markers on an interior surface of said target apparatus at known locations; operating a camera interface to receive the image of the interior surface of said target apparatus from the first camera being calibrated; and operating a camera calibrator configured to compare actual locations of the markers in said image to expected locations of said markers and to generate a first set of camera calibration information corresponding to the first camera based on differences between the actual locations of the markers in said image and the expected locations of said markers.

16 . The method of claim 15 , further comprising:
prior to performing said step of operating the first camera to capture the image of the interior surface; receiving, at a camera interface, images from the first camera while the first camera is mounted on a camera mounting platform positioned in front of the target apparatus; processing at least one of the received images to detect a position of the first camera to thereby determine the position of the first camera relative to the target apparatus; and operating a camera position controller to adjust the position of the camera mounting platform to position the first camera at a predetermined location in front of said target apparatus.

17 . The method of claim 16 , wherein operating the camera position controller includes:
operating the camera position controller to control a position drive motor based on one or more images captured by the first camera to move a drive shaft coupled to the camera mounting platform and to thereby adjust the position of the first camera until the first camera is positioned at a predetermined location in front of said target apparatus.

18 . The method of claim 17 , wherein said first set of camera calibration information includes first UV map correction information indicating one or more adjustments to be made to the location of nodes in a UV map based on the differences between the actual locations of the markers in said first image and the expected locations of said markers.

19 . The method of claim 18 , wherein generating said first set of camera calibration information ( 1536 ) includes generating for a first node in the UV map a U offset and a V offset to be applied to the position of the first node to adjust the location of the first node to compensate for a camera introduced distortion, said UV map being a map to be used by a playback device to map an image captured by said first camera to a surface of a mesh model.

20 . A non-transitory computer storage comprising instructions which, when executed, direct a computer system comprising one or more computing devices to perform the steps of:
operating a first camera to capture an image of an interior surface of a target apparatus having a known three dimensional (3D) shape and including markers on an interior surface of said target apparatus at known locations;
operating a camera interface to receive the image of the interior surface of said target apparatus from the first camera being calibrated; and
operating a camera calibrator configured to compare actual locations of the markers in said image to expected locations of said markers and to generate a first set of camera calibration information corresponding to the first camera based on differences between the actual locations of the markers in said image and the expected locations of said markers.

Assignees

• Nextvr Inc

Inventors

• Cole David
• Moss Alan Mckay
• Medina Hector M

Classifications

• G06T11/10

  Texturing; Colouring; Generation of textures or colours (retouching, inpainting or scratch removal G06T5/77) · CPC title

• H04N21/4402

  involving reformatting operations of video signals for household redistribution, storage or real-time display {(details of conversion of video standards at pixel level H04N7/01; video transcoding H04N19/40; adapting incoming signals to the display format of the display terminal G09G5/005; media handling at the source in data packet switching networks H04L65/764)} · CPC title

• G02B2027/014

  comprising information/image processing systems · CPC title

• H04N13/246

  Calibration of cameras · CPC title

• H04N13/275

  from three-dimensional [3D] object models, e.g. computer-generated stereoscopic image signals · CPC title