Generating shadows from a segmentation mask

Patent metadata

Field	Value
Publication number	US-2026099993-A1
Application number	US-202418906416-A
Country	US
Kind code	A1
Filing date	Oct 4, 2024
Priority date	Oct 4, 2024
Publication date	Apr 9, 2026
Grant date	—

Abstract

Disclosed herein are system, method, and computer program product embodiments for generating realistic shadows for an image object, such as a vehicle, for computer browsing. The method includes generating a segmentation of an image object; generating a contour from the segmentation mask, wherein the contour comprises an array of the pixels along a boundary of the segmentation mask; calculating a plurality of regions of the contour, wherein each of the plurality of regions comprise a set of continuous points; identifying within the plurality of the regions of the contour, a plurality of key points that correspond to ground contact points of the image object; applying, based on the plurality of key points, a perspective warping to a first shadow template to generate the shadow for the image object, and rendering, on a selected background, the image segmentation as an overlay on the shadow.

First claim

What is claimed is:

1 . A computer-implemented method to generate a shadow for an image object, the computer-implemented method comprising:
generating, from received imagery, a segmentation of the image object, wherein the segmentation is generated based on a segmentation mask comprising an array of contiguous pixels of the image object; generating a contour from the segmentation mask, wherein the contour comprises an array of the pixels along a boundary of the segmentation mask; calculating a plurality of regions of the contour, wherein each of the plurality of regions comprise a set of continuous points; identifying within the plurality of the regions of the contour, a plurality of key points that correspond to ground contact points of the image object; applying, based on the plurality of key points, a perspective warping to a first shadow template to generate the shadow for the image object; and rendering, on a selected background, the image segmentation as an overlay on the shadow.

2 . The computer-implemented method of claim 1 , further comprising removing a background of the received imagery.

3 . The computer-implemented method of claim 1 , wherein the contour comprises at least a partially closed curve.

4 . The computer-implemented method of claim 1 , further comprising applying a one-dimensional Gaussian smoothing to the contour.

5 . The computer-implemented method of claim 4 , further comprising calculating a plurality of first derivatives along the contour.

6 . The computer-implemented method of claim 5 , wherein the identifying the plurality of key points comprises calculating the plurality of first derivatives along the contour in a selected direction to identify instantaneous rates of change of the contour, and wherein each of the plurality of key points correspond to a pixel that corresponds to a lowest absolute value of a first y-derivative, where y is a coordinate in an x-y coordinate system of the received imagery.

7 . The computer-implemented method of claim 6 , applying smoothing to the calculated plurality of first derivatives.

8 . The computer-implemented method of claim 7 , wherein the identifying a plurality of regions further comprises applying filters on the first derivatives.

9 . The computer-implemented method of claim 7 , wherein the identifying the plurality of key points further comprises calculating a plurality of second derivatives along the contour in the selected direction.

10 . The computer-implemented method of claim 1 , wherein the image object is a vehicle and the plurality of key points correspond to a plurality of ground contact points of tires of the vehicle.

11 . The computer-implemented method of claim 1 , further comprises estimating additional ones of the plurality of key points to assist in the perspective warping of the first shadow template.

12 . A system, comprising:
a memory; and one or more processors configured to:
generate, from received imagery, a segmentation of an image object, wherein the segmentation is based on a segmentation mask comprising an array of contiguous pixels of the image object;
generate a contour from the segmentation mask, wherein the contour comprises an array of the pixels along a boundary of the segmentation mask;
calculate a plurality of regions of the contour, wherein each of the plurality of regions comprise a set of continuous points;
identify within the plurality of the regions of the contour, a plurality of key points that correspond to ground contact points of the image object;
apply, based on the plurality of key points, a perspective warping to a first shadow template to generate a second shadow for the image object; and
render, on a selected background, the image segmentation as an overlay on the second shadow.

13 . The system of claim 12 , further configured to apply a one-dimensional Gaussian smoothing to the contour.

14 . The system of claim 12 , further configured to calculate a plurality of first derivatives and second derivatives along the contour to identify the plurality of key points, wherein each of the plurality of key points correspond to a pixel that corresponds to a lowest absolute value of a first y-derivative, where y is a coordinate in an x-y coordinate system of the received imagery.

15 . The system of claim 12 , wherein the image object is a vehicle and the plurality of key points correspond to a plurality of ground contact points of tires of the vehicle.

16 . The system of claim 12 , further configured to estimate additional ones of the plurality of key points to assist in the perspective warping of the first shadow template.

17 . A non-transitory computer-readable device having instructions stored thereon that, when executed by at least one computing device, causes the at least one computing device to perform operations comprising:
generating, from received imagery, a segmentation of an image object, wherein the segmentation mask comprises an array of contiguous pixels of the image object; generating a contour from the segmentation mask, wherein the contour comprises an array of the pixels along a boundary of the segmentation mask calculating a plurality of regions of the contour, wherein each of the plurality of regions comprises a set of continuous points; identifying within the plurality of the regions of the contour, a plurality of key points that correspond to ground contact points of the image object; applying, based on the plurality of key points, a perspective warping to a first shadow template to generate second shadow for the image object; and rendering, on a selected background, the image segmentation as an overlay on the second shadow.

18 . The non-transitory computer-readable device of claim 17 , comprising further operations to calculate a plurality of first derivatives and second derivatives along the contour to identify the plurality of key points, wherein each of the plurality of key points correspond to a pixel that corresponds to a lowest absolute value of a first y-derivative, where y is a coordinate in an x-y coordinate system of the received imagery.

19 . The non-transitory computer-readable device of claim 17 , wherein the image object is a vehicle and the plurality of key points correspond to a plurality of ground contact points of tires of the vehicle.

20 . The non-transitory computer-readable device of claim 17 , comprising further operations to estimate additional ones of the plurality of key points to assist in the perspective warping of the first shadow template.

Assignees

• Capital One Services Llc

Inventors

• Ramamohan Deepak
• Gampa Rama Teja

Classifications

• G06T7/12

  Edge-based segmentation · CPC title

• G06T7/194

  involving foreground-background segmentation · CPC title

• G06T2207/20021

  Dividing image into blocks, subimages or windows · CPC title

• G06T7/11

  Region-based segmentation · CPC title

• G06T15/60 Primary

  Shadow generation · CPC title