Discovery of and connection to remote devices

Patent metadata

Field	Value
Publication number	US-12039859-B2
Application number	US-202217980057-A
Country	US
Kind code	B2
Filing date	Nov 3, 2022
Priority date	May 21, 2019
Publication date	Jul 16, 2024
Grant date	Jul 16, 2024

Abstract

A method includes receiving one or more signals that each indicate a device type for a respective remote device, identifying one or more visible devices in one or more images, matching a first device from the one or more visible devices with a first signal from the one or more signals based on a device type of the first device matching a device type for the first signal and based on a visible output of the first device, pairing the first device with a second device, and controlling a function of the first device using the second device.

First claim

What is claimed is:

1. A method, comprising:
receiving one or more signals that each indicate a device type for a respective remote device;
identifying visible devices in one or more images;
identifying two or more visible devices from the visible devices as potential matches to a first signal from the one or more signals by comparing a device type of each of the two or more visible devices with a device type for the first signal;
matching a first visible device from the two or more visible devices with the first signal from the one or more signals based on content output for display on a screen of the first visible device;
pairing the first visible device with a second device; and
controlling a function of the first visible device using the second device.

2. The method of claim 1 , wherein the content output for display on the screen of the first visible device includes predetermined content requested by the second device.

3. The method of claim 1 , wherein the one or more signals each indicate a location for the respective remote device, and identifying the two or more visible devices as potential matches to the first signal further comprises comparing locations of the two or more visible devices with the location for the first signal.

4. The method of claim 3 , wherein the locations of the two or more visible devices are determined using the one or more images.

5. The method of claim 1 , wherein the device type of each of the two or more visible devices is determined based on the one or more images using an object detection model.

6. The method of claim 1 , wherein the one or more images are captured using an imaging device that is associated with at least one of the second device or a host device that is separate from the second device.

7. The method of claim 1 , wherein matching the first visible device with the first signal includes the first visible device transmitting a screenshot to the second device in response to a request from the second device, and wherein the screenshot includes arbitrary content that is content currently output for display on the screen of the first visible device when the second device requests the screenshot.

8. A non-transitory computer-readable storage device including program instructions executable by one or more processors that, when executed, cause the one or more processors to perform operations, the operations comprising:
receiving one or more signals that each indicate a device type for a respective remote device;
identifying visible devices in one or more images;
identifying two or more visible devices from the visible devices as potential matches to a first signal from the one or more signals by comparing a device type of each of the two or more visible devices with a device type for the first signal;
matching a first visible device from the two or more visible devices with the first signal from the one or more signals based on content output for display on a screen of the first visible device;
pairing the first visible device with a second device; and
controlling a function of the first visible device using the second device.

9. The non-transitory computer-readable storage device of claim 8 , wherein the content output for display on the screen of the first visible device includes predetermined content requested by the second device.

10. The non-transitory computer-readable storage device of claim 8 , wherein the one or more signals each indicate a location for the respective remote device, and identifying the two or more visible devices as potential matches to the first signal further comprises comparing locations of the two or more visible devices with the location for the first signal.

11. The non-transitory computer-readable storage device of claim 10 , wherein the locations of the two or more visible devices are determined using the one or more images.

12. The non-transitory computer-readable storage device of claim 8 , wherein the device type of each of the two or more visible devices is determined based on the one or more images using an object detection model.

13. The non-transitory computer-readable storage device of claim 8 , wherein the one or more images are captured using an imaging device that is associated with at least one of the second device or a host device that is separate from the second device.

14. The non-transitory computer-readable storage device of claim 8 , wherein matching the first visible device with the first signal includes the first visible device transmitting a screenshot to the second device in response to a request from the second device, and wherein the screenshot includes arbitrary content that is content currently output for display on the screen of the first visible device when the second device requests the screenshot.

15. A system, comprising:
a memory; and
one or more processors that are configured to execute instructions that are stored in the memory, wherein the instructions, when executed, cause the one or more processors to:
receive one or more signals that each indicate a device type for a respective remote device;
identify visible devices in one or more images;
identify two or more visible devices from the visible devices as potential matches to a first signal from the one or more signals by comparing a device type of each of the two or more visible devices with a device type for the first signal;
match a first visible device from the two or more visible devices with the first signal from the one or more signals based on content output for display on a screen of the first visible device;
pair the first visible device with a second device; and
control a function of the first visible device using the second device.

16. The system of claim 15 , wherein the content output for display on the screen of the first visible device includes predetermined content requested by the second device.

17. The system of claim 15 , wherein the one or more signals each indicate a location for the respective remote device, and identifying the two or more visible devices as potential matches to the first signal further comprises comparing locations of the two or more visible devices with the location for the first signal.

18. The system of claim 17 , wherein the locations of the two or more visible devices are determined using the one or more images.

19. The system of claim 15 , wherein the device type of each of the two or more visible devices is determined based on the one or more images using an object detection model.

20. The system of claim 15 , wherein the one or more images are captured using an imaging device that is associated with at least one of the second device or a host device that is separate from the second device.

21. The system of claim 15 , wherein matching the first visible device with the first signal includes the first visible device transmitting a screenshot to the second device in response to a request from the second device, and wherein the screenshot includes arbitrary content that is content currently output for display on the screen of the first visible device when the second device requests the screenshot.

Assignees

• Apple Inc

Inventors

• Norris Jeffrey S
• Sommer Bruno M
• Da Veiga Alexandre

Classifications

• G06F18/22

  Matching criteria, e.g. proximity measures · CPC title

• H04W8/005

  Discovery of network devices, e.g. terminals · CPC title

• G08C2201/31

  Voice input · CPC title

• G06V10/22

  by selection of a specific region containing or referencing a pattern; Locating or processing of specific regions to guide the detection or recognition · CPC title

• H04W76/10

  Connection setup · CPC title