Determining and correlating visual context on a user device with user behavior using digital content on the user device

Patent metadata

Field	Value
Publication number	US-10631050-B2
Application number	US-201715810503-A
Country	US
Kind code	B2
Filing date	Nov 13, 2017
Priority date	Nov 13, 2017
Publication date	Apr 21, 2020
Grant date	Apr 21, 2020

Abstract

Certain embodiments involve determining visual context associated with user behavior and associating the visual context with the user behavior. For example, a system captures a portion of a user interface provided to a user at a time of a user action on the user interface. The captured portion can include digital content and the system can detect the digital content. The system can also generate a digital representation of the digital content. The digital representation can indicate the digital and exclude the digital content. The system can determine a visual context associated with the user action based on the generated digital representation. The visual context describes the digital content displayed via the user interface at the time of the user action. The system can also determine subsequent digital content to output to the user to create a subsequent visual context to encourage a particular user behavior.

First claim

The invention claimed is:

1. A method for determining visual context associated with user behavior and correlating the visual context with the user behavior, the method comprising operations performed by at least one processor, the operations comprising:
capturing, via a context determination application included in a user device and at a time of a user action on a user interface of the user device, a portion of the user interface provided to a user, the portion of the user interface including digital content displayed via the user interface;
detecting the digital content included in the portion of the user interface;
generating, via a digital representation generation engine, a digital representation of the digital content included in the captured portion, the digital representation indicating the digital content and wherein the digital representation excludes the digital content;
determining a visual context associated with the user action based on the generated digital representation, the visual context describing the digital content displayed via the user interface at the time of the user action; and
determining, based on the visual context, subsequent digital content to output to the user of the user device to create a subsequent visual context to encourage a particular user behavior.

2. The method of claim 1 , wherein detecting the digital content comprises:
analyzing, by the processor, the captured portion of the user interface; and
determining, by the processor, a position or type of the digital content within the captured portion of the user interface and wherein the digital representation of the digital content indicates the position or type of the digital content.

3. The method of claim 1 , wherein determining the visual context comprises:
determining, by the processor, an amount of digital content displayed via the portion of the user interface at the time of the user action by analyzing the captured portion of the user interface.

4. The method of claim 1 , wherein determining the visual context comprises:
determining, by the processor, a percentage of the captured portion of the user interface that includes the digital content at the time of the user action by analyzing the captured portion of the user interface.

5. The method of claim 1 , wherein determining the visual context comprises:
determining, by the processor, a first size of a first digital content displayed via the captured portion of the user interface at the time of the user action;
determining, by the processor, a second size of a second digital content displayed via the captured portion of the user interface at the time of the user action; and
comparing, by the processor, the first size of the first digital content and the second size of the second digital content to a size of the captured portion of the user interface to determine a ratio between the first digital content and the second digital content.

6. The method of claim 1 , wherein determining the visual context comprises:
determining, by the processor, a first size of a first digital content displayed via the captured portion of the user interface at the time of the user action;
determining, by the processor, a second size of a second digital content displayed via the captured portion of the user interface at the time of the user action;
determining, by the processor, a size of the captured portion of the user interface;
comparing, by the processor, the first size of the first digital content and the second size of the second digital content to the size of the captured portion of the user interface to determine a first relative size of the first digital content and a second relative size of the second digital content; and
comparing, by the processor, the first and second relative sizes to determine which of the first and second digital content is larger.

7. The method of claim 1 , wherein determining the visual context comprises:
detecting, by the processor, an amount of empty space in the captured portion of the user interface;
determining, by the processor, a size of the digital content; and
comparing, by the processor, the size of the digital content and the amount of empty space in the captured portion of the user interface to determine a ratio between the amount of empty space in the captured portion of the user interface and size of the digital content.

8. The method of claim 1 , wherein determining the visual context comprises:
detecting, by the processor, a first color in the captured portion of the user interface;
detecting, by the processor, a second color in the captured portion of the user interface;
determining, by the processor, a first amount of the first color and a second amount of the second color in the captured portion of the user interface; and
comparing, by the processor, the first and second amount to determine whether the first or second color occurs more frequently in the captured portion of the user interface.

9. The method of claim 1 , wherein determining the visual context comprises:
detecting, by the processor, a first color and a second color in the captured portion of the user interface;
determining, by the processor and using a Red, Green, Blue (RGB) color rounding model, a similarity between the first color or the second color and a color group of a plurality of color groups, the plurality of color group comprising a red color group, a green color group, or a blue color group;
grouping, by the processor, the first color or second color into a color group based on the determined similarity; and
determining, by the processor, which color group of the plurality of color groups that appears more frequently in the captured portion of the user interface.

10. The method of claim 1 , wherein determining the visual context comprises:
determining, by the processor and based on a pixel of the captured portion of the user interface, an average color of the captured portion of the user interface by recursively scaling down an image of the captured portion of the user interface to obtain the pixel.

11. The method of claim 1 , wherein determining the visual context comprises:
detecting, by the processor, a plurality of colors in the captured portion of the user interface; and
generating, by the processor, a color histogram of the plurality of colors, the color histogram representing a distribution of the plurality of colors.

12. A system comprising:
a processing device; and
a non-transitory computer-readable medium communicatively coupled to the processing device, wherein the processing device is configured to perform operations comprising:
capturing, via a context determination application, a portion of a user interface provided to a user at a time of a user action on the user interface, the portion of the user interface including digital content displayed via the user interface;
detecting the digital content included in the portion of the user interface;
generating, via a digital representation generation engine, a digital representation of the digital content included in the captured portion, the digital representation indicating the digital content and wherein the digital representation excludes the digital content;
determining a visual context associated with the user action based on the generated digital representation, the visual context describing the digital content displayed via the user interface at the time of the user action; and
determining, based on the visual context, subsequent digital content to output to the user to create a subsequent visual context to encourage a particular user behavior.

13. The system of claim 12 , wherein th

Assignees

• Adobe Inc

Inventors

• Fransen Peter Raymond
• Motayne Aaron

Classifications

• H04N21/25866

  Management of end-user data (customer care in data networks H04L41/5077) · CPC title

• H04N21/6582

  Data stored in the client, e.g. viewing habits, hardware capabilities, credit card number (arrangements where receivers interact with the broadcast H04H20/38) · CPC title

• G06T7/90

  Determination of colour characteristics · CPC title

• H04N21/4667

  Processing of monitored end-user data, e.g. trend analysis based on the log file of viewer selections · CPC title

• G06K9/00469

  Physics · mapped topic