Generating a response that depicts haptic characteristics

Patent metadata

Field	Value
Publication number	US-12429950-B2
Application number	US-202318462077-A
Country	US
Kind code	B2
Filing date	Sep 6, 2023
Priority date	Feb 13, 2017
Publication date	Sep 30, 2025
Grant date	Sep 30, 2025

Abstract

In various example embodiments, a system and method for generating a response that depicts haptic characteristics are presented. Haptic data is received from a client device and the haptic data indicates an interaction with a sensor included in the client device. Haptic characteristics are determined based on the haptic data. At least one image that depicts the determined haptic characteristics is generated. And the at least one image is caused to be displayed on the client device.

First claim

What is claimed is:

1. A method comprising:
modifying a portion of a first image displayed in a first region of a display, and captured by a camera, based on a plurality of haptic interactions, at least a portion of the plurality of haptic interactions being performed within the same first region of the display in which the first image is displayed; and
concurrently with displaying, in the first region of a display, the first image having the modified portion, causing display, in a second region of the display separate from the first image, of a visual representation of a history of haptic interactions, the visual representation of the history of haptic interactions comprising physical interaction with a device, the visual representation of the history of haptic interactions including the plurality of haptic interactions performed by user interactions with the device over a given time interval previous to a current time, at least two of the plurality of haptic interactions being detected by a different one of a plurality of types of sensors of the device, the visual representation comprising:
a first portion of the visual representation comprising a first visual indication of a first of the plurality of haptic interactions made during a first portion of the given time interval;
determining a type of component of the first of the plurality of haptic interactions performed by the user interactions with the device, the type of component corresponding to an individual one of the plurality of types of sensors of the device;
selecting a shape corresponding to the type of component of the first of the plurality of haptic interactions which has been performed by the user interactions with the device; and
the first visual indication having the selected shape with a size that is determined based on a duration of time corresponding to the first of the plurality of haptic interactions.

2. The method of claim 1 , the visual representation comprising a first portion with a first shape representing the first of the plurality of haptic interactions detected by a first of the plurality of types of sensors and comprising a second portion with a second shape representing a second of the plurality of haptic interactions detected by a second of the plurality of types of sensors, further comprising:
receiving haptic data from the device after capturing the first image, the haptic data indicating an interaction with a sensor included in the device; and
determining, based on the haptic data, the plurality of haptic interactions performed over the given time interval.

3. The method of claim 1 , wherein the plurality of haptic interactions is performed with respect to the first image displayed on a first portion of a touchscreen of a device and result in one or more modifications of the first image, further comprising:
determining first and second haptic characteristics;
determining that a combination of the first and second haptic characteristics match a predetermined combination of threshold characteristics; and
in response to determining that the combination of the first and second haptic characteristics match the predetermined combination of threshold characteristics, generating a second image that depicts the combination of the first and second haptic characteristics, the second image comprising the visual representation of the history of haptic interactions.

4. The method of claim 1 , further comprising:
causing display of a graph over a portion of the first image, wherein the visual representation includes a graph that corresponds to the plurality of haptic interactions, the graph indicating a length that corresponds to a duration of each of the plurality of interactions, the graph indicating an amplitude that corresponds to an amount of force of each of the plurality of interactions, the graph having a frequency that corresponds to a gesture performed by a user.

5. The method of claim 1 , further comprising:
receiving audio or video data;
determining characteristics of the audio or video data being played back;
determining that the characteristics of the audio or video data being played back trigger a haptic response; and
executing the haptic response during playback of the audio or video data that includes the plurality of haptic interactions.

6. The method of claim 1 , further comprising setting a color for the first visual indication based on a first type of haptic data included in a first of the plurality of haptic interactions and setting the shape for the first visual indication based on a second type of haptic data included in the first of the plurality of haptic interactions, and wherein an intensity of the color is determined based on an amount of force corresponding to the first of the plurality of haptic interactions.

7. The method of claim 1 , further comprising:
determining that a second device has a different download speed than a first device;
adjusting the plurality of haptic interactions based on a difference in download speed of the first device relative to the second device; and
triggering a response on the second device based on the adjusted plurality of haptic interactions.

8. The method of claim 1 , further comprising:
modifying a graphical element that assists with navigation of a user interface of a device based on the plurality of haptic interactions, wherein the visual representation of the history of haptic interactions is generated in response to determining that haptic characteristics match predetermined threshold characteristics.

9. The method of claim 1 , further comprising:
determining capabilities of a second device;
translating the plurality of haptic interactions into further haptic characteristics based on the capabilities of the second device;
generating an image that depicts the further haptic characteristics; and
causing display of the image that depicts the further haptic characteristics on the second device.

10. The method of claim 9 , wherein determining the capabilities of the second device includes receiving, from the second device, a signal that indicates the capabilities of the second device, and wherein translating comprises removing data from the plurality of haptic interactions in response to determining that the second device includes circuitry that results in slower processing speeds than a first device.

11. A system comprising:
one or more processors; and
a memory storing executable instructions that, when executed by the one or more processors, cause the one or more processors to perform operations comprising:
modifying a portion of a first image displayed in a first region of a display, and captured by a camera, based on a plurality of haptic interactions, at least a portion of the plurality of haptic interactions being performed within the same first region of the display in which the first image is displayed; and
concurrently with displaying, in the first region of a display, the first image having the modified portion, causing display, in a second region of the display separate from the first image, of a visual representation of a history of haptic interactions, the visual representation of the history of haptic interactions comprising physical interaction with a device, the visual representation of the history of haptic interactions including the plurality of haptic interactions performed by user interactions with the device over a given time interval previous to a current time, at least two of the plurality of haptic interactions being detected by a different one of a plurality of types of sensors of the device, the visual representation comprising:
a first portion of the visual representation comprising a first visual indica

Assignees

• Snap Inc

Inventors

• Chand Jesse
• Jayaram Krish

Classifications

• G06F2203/04803

  Split screen, i.e. subdividing the display area or the window area into separate subareas · CPC title

• G06F3/0488 Primary

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F2203/014

  Force feedback applied to GUI · CPC title

• G06F3/04883

  for inputting data by handwriting, e.g. gesture or text · CPC title

• G06F3/0425

  using a single imaging device like a video camera for tracking the absolute position of a single or a plurality of objects with respect to an imaged reference surface, e.g. video camera imaging a display or a projection screen, a table or a wall surface, on which a computer generated image is displayed or projected (tracking a projected light spot to determine a position on a display surface G06F3/0386) · CPC title