Automatic network device identification and presentation of network information via augmented reality

Patent metadata

Field	Value
Publication number	US-11575584-B2
Application number	US-202117357260-A
Country	US
Kind code	B2
Filing date	Jun 24, 2021
Priority date	Jun 24, 2021
Publication date	Feb 7, 2023
Grant date	Feb 7, 2023

Abstract

Automatic network device identification and presentation of network information via augmented reality is disclosed. Real time imagery of a scene within a field of view of a camera is received. It is determined that the imagery depicts a first object. It is determined that the first object comprises a first network-attached device attached to a network. Information that corresponds to the first network-attached device is obtained, and the information is presented in conjunction with a real time view of the scene.

First claim

What is claimed is:

1. A method comprising:
receiving, by a computing device comprising a processor device, real time imagery of a scene within a field of view of a camera;
determining, by the computing device, that the real time imagery depicts a first object;
generating a first image of the first object from the real time imagery;
causing, by the computing device, the first image of the first object to be provided to a machine-learned model (MLM) that has been trained to recognize one or more devices that are connected to a particular network of a plurality of different networks;
receiving, by the computing device based on output from the MLM, information that indicates that the first object depicted in the first image is a first network-attached device that is connected to the particular network;
obtaining, by the computing device, information that corresponds to the first network-attached device; and
presenting, by the computing device on a display device, the information that corresponds to the first network-attached device in conjunction with a real time view of the scene.

2. The method of claim 1 further comprising:
obtaining an identifier of the first network-attached device;
and wherein obtaining the information that corresponds to the first network-attached device comprises sending the identifier of the first network-attached device to a remote computing device that maintains information corresponding to a plurality of different network-attached devices; and
receiving, from the remote computing device, the information that corresponds to the first network-attached device.

3. The method of claim 1 further comprising:
wherein presenting the information that corresponds to the first network-attached device further comprises presenting a user interface (UI) control that, if selected, causes one of: pausing traffic to the first network-attached device, increasing a download speed of the first network-attached device, executing a speed test of the first network-attached device, disconnecting the first network-attached device from the particular network, and assigning a traffic restriction rule.

4. The method of claim 1 further comprising:
prior to causing the first image of the first object to be provided to the MLM, receiving, by the computing device, the MLM.

5. The method of claim 1 further comprising:
determining, by the computing device, that the scene includes a second object; and
determining, by the computing device, that the second object is not a network-attached device.

6. The method of claim 5 wherein determining, by the computing device, that the second object is not a network-attached device further comprises:
generating, from the real time imagery, a second image that depicts the second object;
providing the second image to the MLM; and
receiving, from the MLM, output that identifies the second image as an image that depicts a device that is not a network-attached device.

7. The method of claim 1 wherein presenting the information that corresponds to the first network-attached device in conjunction with the real time view of the scene further comprises:
generating information imagery that depicts the information, determining a location of the first object in a real time video stream generated by the camera, overlaying the information imagery on top of a portion of the real time video stream at a location in proximity to the location of the first object in the real time video stream to form a composite real time video stream, and sending the composite real time video stream to the display.

8. The method of claim 1 further comprising:
determining, by the computing device, that the scene includes a second object;
determining, by the computing device, that the second object comprises a second network-attached device attached to the particular network;
obtaining, by the computing device, information that corresponds to the second network-attached device; and
presenting, by the computing device on the display device, the information that corresponds to the second network-attached device concurrently with the information that corresponds to the first network-attached device in conjunction with a real time view of the scene.

9. The method of claim 1 wherein the first network-attached device comprises one of a media streaming device, a security camera, a smart speaker, and a game console.

10. The method of claim 1 wherein the information that corresponds to the first network-attached device comprises a name of the first network-attached device.

11. The method of claim 1 wherein the information that corresponds to the first network-attached device identifies one or more of: a Wi-Fi access point to which the first network-attached device is connected, a network utilization of the first network-attached device, a connectivity status of the first network-attached device, a device type of the first network-attached device, a media access control (MAC) address of the first network-attached device, an internet protocol (IP) address of the first network-attached device, and a current activity of the first network-attached device.

12. The method of claim 1 wherein the computing device comprises a see-through augmented reality (AR) device comprising a transparent lens having a reflective interior surface, and wherein presenting the information that corresponds to the first network-attached device in conjunction with the real time view of the scene comprises projecting the information that corresponds to the first network-attached device onto the reflective interior surface.

13. The method of claim 1 wherein the computing device comprises a head-mounted AR device comprising a display, and wherein presenting the information that corresponds to the first network-attached device in conjunction with the real time view of the scene comprises generating imagery that depicts the information that corresponds to the first network-attached device, overlaying a real time video stream generated by the camera with the imagery that depicts the information that corresponds to the first network-attached device to form a composite real time video stream, and sending the composite real time video stream to the display.

14. The method of claim 1 wherein the computing device comprises a mobile device comprising a display, and wherein presenting the information that corresponds to the first network-attached device in conjunction with the real time view of the scene comprises generating imagery that depicts the information that corresponds to the first network-attached device, overlaying a real time video stream generated by the camera with the imagery that depicts the information that corresponds to the first network-attached device to form a composite real time video stream, and sending the composite real time video stream to the display.

15. The method of claim 1 further comprising:
prior to receiving the real time imagery of the scene, detecting that a device has connected to the particular network;
determining a type of device of the device;
obtaining imagery of the device based on the type of the device; and
training the MLM using the imagery of the device to identify the device as the first network-attached device in response to an input comprising an image of the first network-attached device.

16. A computing device, comprising:
a memory; and
a processor device coupled to the memory and configured to:
receive real time imagery of a scene within a field of view of a camera;
determine that the real time imagery depicts a first object;
generate a first image of the

Assignees

• Charter Communications Operating Llc

Inventors

• Wright Ethan
• Alami Omar

Classifications

• H04L41/0896

  Bandwidth or capacity management, i.e. automatically increasing or decreasing capacities (flow or congestion control using dynamic resource allocation, e.g. in-call renegotiation, H04L47/76) · CPC title

• H04L41/12

  Discovery or management of network topologies · CPC title

• G06N20/00

  Machine learning · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• H04L41/16 Primary

  using machine learning or artificial intelligence · CPC title