Systems and methods for motion capture

Patent metadata

Field	Value
Publication number	US-10812693-B2
Application number	US-201816102536-A
Country	US
Kind code	B2
Filing date	Aug 13, 2018
Priority date	Oct 20, 2017
Publication date	Oct 20, 2020
Grant date	Oct 20, 2020

Abstract

Embodiments of the disclosure provide systems and methods for motion capture to generate content (e.g., motion pictures, television programming, videos, etc.). An actor or other performing being can have multiple markers on his or her face that are essentially invisible to the human eye, but that can be clearly captured by camera systems of the present disclosure. Embodiments can capture the performance using two different camera systems, each of which can observe the same performance but capture different images of that performance. For instance, a first camera system can capture the performance within a first light wavelength spectrum (e.g., visible light spectrum), and a second camera system can simultaneously capture the performance in a second light wavelength spectrum different from the first spectrum (e.g., invisible light spectrum such as the IR light spectrum). The images captured by the first and second camera systems can be combined to generate content.

First claim

What is claimed is:

1. A computer-implemented method of capturing motion with a motion capture system, the method comprising:
emitting first light at wavelengths within a first spectrum towards an actor in a performance area;
capturing a first plurality of images of a performance using a first set of one or more cameras operable to capture the first light at wavelengths in the first spectrum, the first plurality of images representing captured emitted first light that has been reflected off of at least one marker applied to a face of the actor, wherein the first plurality of images is captured by two witness cameras of the first set of one or more cameras laterally positioned on opposite sides of a taking camera of a second set of one or more cameras attached to a moveable rig;
identifying locations of at least one marker applied to the face of the actor by analyzing at least some of the first plurality of images captured by at least one camera from the first set of one or more cameras;
generating position and motion data for a digital character based on the identified locations of the at least one marker applied to the face of the actor, wherein the at least one marker is a gel-based marker that reflects the first light at wavelengths in the first spectrum that is substantially invisible to a human eye and is detected by at least one camera of the first set of the one or more cameras;
capturing a second plurality of images of the performance using a second set of one or more cameras operable to capture second light at wavelengths in a second spectrum different from the first spectrum, the second plurality of images representing at least a portion of a body of the actor and the performance area; and
generating content based on the generated position and motion data for the digital character and the second plurality of images.

2. The computer-implemented method of claim 1 , wherein the at least one marker does not reflect the second light at wavelengths in the second spectrum.

3. The computer-implemented method of claim 1 , wherein the first plurality of images and the second plurality of images are simultaneously captured by the respective first set of one or more cameras and the second set of one or more cameras.

4. The computer-implemented method of claim 1 , wherein the first plurality of images are captured by receiving the reflected emitted first light propagating along a vector that is parallel but opposite in direction from a light source emitting the first light.

5. The computer-implemented method of claim 4 , wherein the first light is emitted from at least one light source configured as a ring of lights positioned around a lens of a respective camera of the first set of one or more cameras.

6. The computer-implemented method of claim 5 , wherein the first plurality of images is captured by two witness cameras of the first set of one or more cameras laterally positioned on opposite sides of a taking camera of the second set of one or more cameras.

7. The computer-implemented method of claim 6 , wherein the second plurality of images is captured by the taking camera.

8. The computer-implemented method of claim 1 , wherein the first spectrum is a spectrum of light that is visible to a naked human eye, and the second spectrum is a spectrum of light that is substantially invisible to the naked human eye.

9. The computer-implemented method of claim 8 , wherein the second spectrum is defined by the spectrum of infrared light having wavelengths between 701 nm to 1 mm.

10. A computer product comprising a non-transitory computer readable medium storing instructions that when executed control a device including one or more processors, the instructions comprising:
emitting light at wavelengths within a first spectrum towards an actor in a performance area;
capturing a first plurality of images of a performance using a first set of one or more cameras operable to capture first light at wavelengths in the first spectrum, the first plurality of images representing captured emitted first light that has been reflected off of at least one marker applied to a face of the actor, wherein the first plurality of images is captured by two witness cameras of the first set of one or more cameras laterally positioned on opposite sides of a taking camera of a second set of one or more cameras attached to a moveable rig;
identifying locations of at least one marker applied to the face of the actor by analyzing at least some of the first plurality of images captured by at least one camera from the first set of one or more cameras;
generating position and motion data for a digital character based on the identified locations of the at least one marker applied to the face of the actor, wherein the at least one marker is a gel-based marker that reflects the first light at wavelengths in the first spectrum that is substantially invisible to a human eye and is detected by at least one camera of the first set of the one or more cameras;
capturing a second plurality of images of the performance using a second set of one or more cameras operable to capture second light at wavelengths in a second spectrum different from the first spectrum, the second plurality of images representing at least a portion of a body of the actor and the performance area; and
generating content based on the generated position and motion data for the digital character and the second plurality of images.

11. The computer product of claim 10 , wherein the at least one marker does not reflect the second light at wavelengths in the second spectrum.

12. The computer product of claim 10 , wherein the first plurality of images and the second plurality of images are simultaneously captured by the respective first set of one or more cameras and the second set of one or more cameras.

13. The computer product of claim 10 , wherein first plurality of images are captured by receiving the reflected emitted first light propagating along a vector that is parallel but opposite in direction from a light source emitting the first light.

14. The computer product of claim 10 , wherein the first spectrum is a spectrum of light that is visible to a naked human eye, and the second spectrum is a spectrum of light that is substantially invisible to the naked human eye.

15. The computer product of claim 14 , wherein the first light is emitted from at least one light source configured as a ring of lights positioned around a lens of a respective camera of the first set of one or more cameras.

16. A computer system for a motion capture system, the computer system comprising:
one or more processors; and
memory containing code for instructing the one or more processors to:
emit light at wavelengths within a first spectrum towards an actor in a performance area;
capture a first plurality of images of a performance using a first set of one or more cameras operable to capture light at wavelengths in the first spectrum, the first plurality of images representing captured emitted light that has been reflected off of at least one marker applied to a face of the actor, wherein the first plurality of images is captured by two witness cameras of the first set of one or more cameras laterally positioned on opposite sides of a taking camera of a second set of one or more cameras attached to a moveable rig;
identify locations of at least one marker applied to the face of the actor by analyzing at least some of the first plurality of images captured by at least one camera from the first set of one or more cameras;
generate position and motion data for a digital character based on the ident

Assignees

• Lucasfilm Entertainment Co Ltd

Inventors

• Estebecorena Leandro
• Knoll John
• Grabli Stephane
• Karefelt Per
• Helman Pablo
• Levin John M

Classifications

• H04N23/90

  Arrangement of cameras or camera modules, e.g. multiple cameras in TV studios or sports stadiums · CPC title

• H04N23/56

  provided with illuminating means · CPC title

• H04N23/11

  for generating image signals from visible and infrared light wavelengths · CPC title

• H04N23/957 Primary

  Light-field or plenoptic cameras or camera modules · CPC title

• H04N23/54

  Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils · CPC title