Remote touch detection enabled by peripheral device

Patent metadata

Field	Value
Publication number	US-11714540-B2
Application number	US-202017028298-A
Country	US
Kind code	B2
Filing date	Sep 22, 2020
Priority date	Sep 28, 2018
Publication date	Aug 1, 2023
Grant date	Aug 1, 2023

Abstract

The present disclosure relates generally to remote touch detection. In some examples, a first electronic device obtains first image data and second image data about an input, and performs an operation based on the input in accordance with a determination that a set of one or more criteria is met based on the first image data and the second image data. In some examples, a first electronic device causes emission of infrared light by an infrared source of a second electronic device, obtains image data about an input, and performs an operation based on the input in accordance with a determination that a set of one or more criteria is met based on the image data.

First claim

What is claimed is:

1. A first electronic device, comprising:
one or more processors; and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
obtaining first image data about an input, wherein the first image data is obtained using one or more camera sensors of the first electronic device external to a second electronic device;
obtaining second image data about the input, wherein the second image data is obtained using one or more camera sensors of the second electronic device, the first electronic device different from the second electronic device; and
in accordance with a determination that a set of one or more criteria is met based on the first image data and the second image data, performing an operation based on the input, wherein the set of one or more criteria includes a criterion that is met when the input is a touch input.

2. The first electronic device of claim 1 , the one or more programs further including instructions for:
in accordance with a determination that the set of one or more criteria is not met based on the first image data and the second image data, forgoing performing the operation based on the input.

3. The first electronic device of claim 1 , the one or more programs further including instructions for:
prior to obtaining the first image data and the second image data about the input, determining, based on orientation data from at least one of the first electronic device or the second electronic device, whether a suitable environment exists for receiving touch inputs.

4. The first electronic device of claim 1 , wherein the input occurs at a location other than the location of the first electronic device and the location of the second electronic device.

5. The first electronic device of claim 1 , wherein the criteria includes a criterion that is met when the input is a tap gesture, a pinch or spread gesture, a rotate gesture, or corresponds to one or more digits of a user.

6. The first electronic device of claim 1 , wherein the determination that a set of one or more criteria is met is made by the first electronic device.

7. The first electronic device of claim 1 , wherein the first image data and the second image data correspond to a same point in time or period of time.

8. The first electronic device of claim 1 , the one or more programs further including instructions for:
in accordance with a determination that the one or more camera sensors of the second electronic device are occluded, prompting to move the second electronic device to a first new location; and
in accordance with a determination that the one or more camera sensors of the first electronic device are occluded, prompting to move the first electronic device to a second new location.

9. The first electronic device of claim 1 , the one or more programs further including instructions for:
obtaining image data about the second electronic device using the one or more camera sensors of the first electronic device;
determining, based on the image data about the second electronic device, a location of the second electronic device; and
displaying a virtual keyboard based on the location of the second electronic device.

10. The first electronic device of claim 1 , wherein:
the input includes movement of a first object and a second object,
at least one of the first image data and the second image data does not include information about both the first object and the second object, and
the first image data and the second image data, in combination, provides sufficient data for determining whether the set of one or more criteria is met.

11. The first electronic device of claim 1 , the one or more programs further including instructions for:
obtaining image data about the first electronic device using the one or more camera sensors of the second electronic device;
determining, based on the image data about the first electronic device, a location of the first electronic device; and
displaying a virtual keyboard based on the location of the first electronic device.

12. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of a first electronic device, the one or more programs including instructions for:
obtaining first image data about an input, wherein the first image data is obtained using one or more camera sensors of the first electronic device external to a second electronic device;
obtaining second image data about the input, wherein the second image data is obtained using one or more camera sensors of the second electronic device, the first electronic device different from the second electronic device; and
in accordance with a determination that a set of one or more criteria is met based on the first image data and the second image data, performing an operation based on the input, wherein the set of one or more criteria includes a criterion that is met when the input is a touch input.

13. The computer-readable storage medium of claim 12 , the one or more programs further including instructions for:
in accordance with a determination that the set of one or more criteria is not met based on the first image data and the second image data, forgoing performing the operation based on the input.

14. The computer-readable storage medium of claim 12 , the one or more programs further including instructions for:
prior to obtaining the first image data and the second image data about the input, determining, based on orientation data from at least one of the first electronic device or the second electronic device, whether a suitable environment exists for receiving touch inputs.

15. The computer-readable storage medium of claim 12 , wherein the input occurs at a location other than the location of the first electronic device and the location of the second electronic device.

16. The computer-readable storage medium of claim 12 , wherein the first image data and the second image data correspond to a same point in time or period of time.

17. The computer-readable storage medium of claim 12 , the one or more programs further including instructions for:
in accordance with a determination that the one or more camera sensors of the second electronic device are occluded, prompting to move the second electronic device to a first new location; and
in accordance with a determination that the one or more camera sensors of the first electronic device are occluded, prompting to move the first electronic device to a second new location.

18. The computer-readable storage medium of claim 12 , wherein:
the input includes movement of a first object and a second object, and
at least one of the first image data and the second image data does not include information about both the first object and the second object.

19. A method, comprising:
at a first electronic device:
obtaining first image data about an input, wherein the first image data is obtained using one or more camera sensors of the first electronic device external to a second electronic device;
obtaining second image data about the input, wherein the second image data is obtained using one or more camera sensors of the second electronic device, the first electronic device different from the second electronic device; and
in accordance with a determination that a set of one or more criteria is met based on the first image data and the second image data, performing an operation based on the input, wherein the set of one or

Assignees

• Apple Inc

Inventors

• Iglesias Samuel L
• Chalmers Devin W
• SETHI Rohit
• Wang Lejing

Classifications

• G06F3/0488 Primary

  using a touch-screen or digitiser, e.g. input of commands through traced gestures · CPC title

• G06F3/044

  by capacitive means · CPC title

• G06F3/0426

  tracking fingers with respect to a virtual keyboard projected or printed on the surface (virtual keyboards on touch screens G06F3/04886) · CPC title

• G06F3/04845 Primary

  for image manipulation, e.g. dragging, rotation, expansion or change of colour · CPC title

• G06F3/017 Primary

  Gesture based interaction, e.g. based on a set of recognized hand gestures (interaction based on gestures traced on a digitiser G06F3/04883) · CPC title