Method and system for sharing-oriented personalized route planning via a customizable multimedia approach

Patent metadata

Field	Value
Publication number	US-10217224-B2
Application number	US-201615388852-A
Country	US
Kind code	B2
Filing date	Dec 22, 2016
Priority date	Dec 22, 2016
Publication date	Feb 26, 2019
Grant date	Feb 26, 2019

Abstract

In accordance with some embodiments of the disclosed subject matter, a method and a system for recommending and sharing customized multimedia route planning are provided. The method includes: receiving a query image from a user, the query image containing an object-of-interest of the user, performing an integrative segmentation process to determine one or more contours of the object-of-interest in the query image; determining a route having a maximum area overlap with the one or more contours of object-of-interest on a map image; generating an output image including the object-of-interest and the route; and recommending the output image to the user, and sharing the output image on a social network platform.

First claim

What is claimed is:

1. A method for recommending and sharing customized multimedia route planning, comprising:
receiving a query image from a user, the query image containing an object-of-interest of the user;
performing an integrative segmentation process to determine one or more contours of the object-of-interest in the query image;
determining a route having a maximum area overlap with the one or more contours of object-of-interest on a map image;
generating an output image including the object-of-interest and the route; and
recommending the output image to the user, and sharing the output image on a social network platform.

2. The method of claim 1 , further comprising:
receiving additional user specified information with the query image;
wherein the additional user specified information includes an object bounding box, a user geographical location, a map ratio, and an expected route length.

3. The method of claim 2 , wherein:
the map image is obtained from an online map application based on the user geographical location and the map ratio; and
the route is determined based on the one or more contours of the object-of-interest in the query image, the map image, and the expected route length.

4. The method of claim 2 , wherein the integrative segmentation process comprises:
performing an initialization of the query image to obtain an initial trimap;
performing a first automatic segmentation process to identify an approximate border of the object-of-interest from the initial trimap;
performing a user interaction process to edit a segmentation result of the first automatic segmentation process to generate an updated trimap; and
performing a second automatic segmentation process to identify an accurate border of the object-of-interest from the updated trimap.

5. The method of claim 4 , wherein performing an initialization of the query image comprises:
determining an initial foreground region, an initial background region, and an initial remaining region of the query image based on the object bounding box; and
setting Gaussian Mixture Models to each pixel in the initial foreground region and the initial background region.

6. The method of claim 4 , wherein performing the first automatic segmentation process comprises:
assigning Gaussian Mixture Model components to each pixel in the initial remaining region;
learning Gaussian Mixture Model parameters from data of grey values of the initial trimap;
determining an estimate segmentation by using a minimum cut estimation algorithm to solve an equation of Gibbs energy, wherein the equation of Gibbs energy depends on the Gaussian Mixture Model components and the Gaussian Mixture Model parameters;
repeating the above three steps until a condition of convergence of iterative minimization is satisfied; and
performing a border matting to generate the approximate border of the object-of-interest from the initial trimap.

7. The method of claim 4 , wherein performing the user interaction process comprises:
receiving user input on the segmentation result of the first automatic segmentation process, wherein the user input includes at least one of a background brush and a foreground brush;
fixing pixels corresponding to areas indicated by the background brush and the foreground brush respectively; and
updating the segmentation result of the query image accordingly to generate the updated trimap.

8. The method of claim 4 , wherein performing the second automatic segmentation process comprises:
determining an updated segmentation of the update trimap by using a minimum cut estimation algorithm; and
performing a refine operation to identify the accurate border of the object-of-interest from the updated trimap.

9. The method of claim 8 , wherein the refine operation comprises:
assigning Gaussian Mixture Model components to each pixel in an updated remaining region of the update trimap;
learning Gaussian Mixture Model parameters from data of grey values of the updated trimap;
determining an estimate segmentation by using a minimum cut estimation algorithm to solve an equation of Gibbs energy, wherein the equation of Gibbs energy depends on the Gaussian Mixture Model components and the Gaussian Mixture Model parameters;
repeating the above three steps until a condition of convergence of iterative minimization is satisfied; and
performing a border matting to generate the accurate border of the object-of-interest from the updated trimap;
wherein the accurate border includes the one or more contours of object-of-interest.

10. The method of claim 1 , wherein determining the route having the maximum area overlap with the one or more contours of object-of-interest on the map image comprises:
employing a MorphSnakes algorithm to search a route candidate on the map image in eight different orientations;
using a morphological Geodesic Active Contour framework to evolve the route candidate by minimizing an energy functional; and
performing an evaluation method using distortion error to determine the route having the maximum area overlap with the one or more contours of object-of-interest on the map image.

11. The method of claim 10 , wherein the energy functional is represented as:
E ( C )=∫ 0 length(C) g ( I )( C ( s )) ds=∫ 0 1 g ( I )( C ( p )·| C p |) dp,

wherein ds=|C p |dp is an Euclidean arc-length parametrization of a curve C; and
g(I) is a mapping: d → + , x→g(I)(x), which allows selecting regions-of-interest in the image I.

12. The method of claim 10 , wherein employing the MorphSnakes algorithm comprises:
approximating numerical solutions of partial differential equations of curve evolution of the route candidate by successively applying a set of curvature morphological operators;
wherein the set of curvature morphological operators are defined on a binary level-set function.

13. The method of claim 12 , further comprising:
introducing a norm metric between two successive level sets of the binary level-set function to assess a convergence of the curve evolution of the route candidate.

14. A system for recommending and sharing customized multimedia route planning, comprising:
one or more hardware processors, and
a memory, wherein the one or more hardware processors are configured to:
receive a query image from a user, the query image containing an object-of-interest of the user;
perform an integrative segmentation process to determine one or more contours of the object-of-interest in the query image;
determine a route having a maximum area overlap with the one or more contours of object-of-interest on a map image;
generate an output image including the object-of-interest and the route; and
recommend the output image to the user, and share the output image on a social network platform.

15. The system of claim 14 , wherein the one or more hardware processors are further configured to:
receive additional user specified information with the query image;
wherein:
the additional user specified information includes an object bounding box, a user geographical location, a map ratio, and an expected route length;
the map image is obtained from an online map application based on the user geographical location and the map ratio; and
the route is determined based on the one or more contours of an object-of-interest in the query image, the map image, and the expected route length.

16. The system of claim 15 , wherein the one or more hardware processors are further configured to:
perform an initialization of the query image to obtain an initial trimap;
perform a first au

Assignees

• Tcl Res America Inc

Inventors

• Liao Xinpeng
• Ren Xiaobo
• Wang Haohong

Classifications

• G06Q10/40 Primary

  Business processes related to social networking or social networking services · CPC title

• G06F17/30259

  Physics · mapped topic

• G06T2210/12

  Bounding box · CPC title

• G06T2207/20116

  Active contour; Active surface; Snakes · CPC title

• H04L51/32

  Electricity · mapped topic