Interaction method, electronic device, storage medium and program product

Patent metadata

Field	Value
Publication number	US-12487724-B2
Application number	US-202218567602-A
Country	US
Kind code	B2
Filing date	Jul 27, 2022
Priority date	Aug 4, 2021
Publication date	Dec 2, 2025
Grant date	Dec 2, 2025

Abstract

Provided are an interaction method and system, and a device, a storage medium and a program product. A first device receives a first interaction on a first interface and generates a first graphic object presented on the first interface, wherein the first interface presents a first image that is captured by a first camera of the first device. The first device generates first interaction information for indicating the first interaction based on a first camera parameter of the first camera and sends the first interaction information to a second device, which presents on a second interface a second graphic object corresponding to the first interaction based on the first interaction information and a second camera parameter of a second camera of the second device, wherein the second interface presents a second image captured by the second camera.

First claim

I claim:

1 . An interaction method comprising:
receiving, by a first device, a first interaction on a first interface; generating, based on the first interaction, a first graphic object displayed on the first interface, wherein the first interface further displays at least a first image captured by a first camera of the first device; generating, based on a first camera parameter of the first camera, first interaction information indicating the first interaction; and transmitting, to a second device, the first interaction information to cause the second device to display on a second interface a second graphic object corresponding to the first interaction based on the first interaction information and a second camera parameter of a second camera of the second device, wherein the second interface further displays at least a second image captured by the second camera.

2 . The method of claim 1 , wherein the first interaction is received via a screen of the first device, and generating, based on the first interaction, the first graphic object displayed on the first interface comprises:
obtaining screen coordinates corresponding to the first interaction; determining, based on the screen coordinates and the first camera parameter, a display position of the first graphic object; and displaying, based on the display position, the first graphic object on the first interface.

3 . The method of claim 1 , further comprising:
receiving from the second device second interaction information, the second interaction information being generated by the second device based on the second camera parameter and a second interaction on the second interface; and displaying on the first interface a third graphic object corresponding to the second interaction based on the second interaction information and the first camera parameter.

4 . The method of claim 1 , wherein the first interaction information includes:
position information of the first camera; and orientation information of the first camera.

5 . The method of claim 4 , wherein the first interaction information excludes screen coordinate information corresponding to the first interaction.

6 . The method of claim 1 , further comprising:
creating a session; transmitting to the second device a session invitation for joining the session; and in response to receiving a confirmation for the session invitation from the second device, generating the first interface corresponding to the session.

7 . The method of claim 6 , wherein the session invitation includes an encrypted link generated by the first device.

8 . The method of claim 6 , wherein generating the first interaction information indicating the first interaction comprises: generating, by a lightweight plug-in module in the first device, the first interaction information based on the first camera parameter.

9 . The method of claim 8 , wherein the lightweight plug-in module further includes an event manager, wherein the event manager includes:
an event generator configured to, in response to a first operation associated with the session executed by a first engine, generate an event message corresponding to the first operation, wherein the event message includes an event identification corresponding to the first operation; and an event monitor configured to monitor the event message generated by the event generator for triggering a second engine to execute a second operation corresponding to the first operation, wherein the first engine and the second engine are configured to respectively implement different functions associated with the session.

10 . The method of claim 8 , wherein the lightweight plug-in module further includes a network module configured to:
obtain communication data associated with the session; and encapsulate the communication data into a data packet, wherein the data packet includes type information, the type information indicating a type of the encapsulated communication data.

11 . The method of claim 10 , wherein the type information includes at least one of:
first type information indicating that the communication data include network data for maintaining a session connection; second type information indicating that the communication data include identification information of users participating the session; third type information indicating that the communication data include control data of the session; fourth type information indicating that the communication data include snapshot data of the session; fifth type information indicating that the communication data include time data; or sixth type information indicating that the communication data include interaction data generated in the session.

12 . A first device, comprising:
a memory; and a processor; wherein the memory stores one or more computer instructions which, when executed by the processor, cause the first device to perform acts comprising: receiving a first interaction on a first interface of the first device; generating, based on the first interaction, a first graphic object displayed on the first interface, wherein the first interface further displays at least a first image captured by a first camera of the first device; generating, based on a first camera parameter of the first camera, first interaction information indicating the first interaction; and transmitting, to a second device, the first interaction information to cause the second device to display on a second interface a second graphic object corresponding to the first interaction based on the first interaction information and a second camera parameter of a second camera of the second device, wherein the second interface further displays at least a second image captured by the second camera.

13 . A non-transitory computer readable storage medium having one or more computer instructions stored thereon, wherein the one or more computer instructions, when executed by a processor, implement acts comprising:
receiving, by a first device, a first interaction on a first interface; generating, based on the first interaction, a first graphic object displayed on the first interface, wherein the first interface further displays at least a first image captured by a first camera of the first device; generating, based on a first camera parameter of the first camera, first interaction information indicating the first interaction; and transmitting, to a second device, the first interaction information to cause the second device to display on a second interface a second graphic object corresponding to the first interaction based on the first interaction information and a second camera parameter of a second camera of the second device, wherein the second interface further displays at least a second image captured by the second camera.

14 . The non-transitory computer readable storage medium of claim 13 , wherein the first interaction is received via a screen of the first device, and generating, based on the first interaction, the first graphic object displayed on the first interface comprises:
obtaining screen coordinates corresponding to the first interaction; determining, based on the screen coordinates and the first camera parameter, a display position of the first graphic object; and displaying, based on the display position, the first graphic object on the first interface.

15 . The non-transitory computer readable storage medium of claim 13 , the acts further comprising:
receiving from the second device second interaction information, the second interaction information being generated by the second device based on

Assignees

• Beijing Zitiao Network Technology Co Ltd

Inventors

• Pan Keting

Classifications

• G06F3/01

  Input arrangements or combined input and output arrangements for interaction between user and computer (G06F3/16 takes precedence) · CPC title

• H04L69/164

  Adaptation or special uses of UDP protocol · CPC title

• G06F3/0481 Primary

  based on specific properties of the displayed interaction object or a metaphor-based environment, e.g. interaction with desktop elements like windows or icons, or assisted by a cursor's changing behaviour or appearance · CPC title