Device state interface

Patent metadata

Field	Value
Publication number	US-11711495-B2
Application number	US-202217653432-A
Country	US
Kind code	B2
Filing date	Mar 3, 2022
Priority date	Jul 9, 2016
Publication date	Jul 25, 2023
Grant date	Jul 25, 2023

Abstract

A system and method for visually automated interface integration that includes collecting image data; detecting a device interface source in the image data; processing the image data associated with the device interface source into an extracted interface representation; and exposing at least one access interface to the extracted interface representation.

First claim

We claim:

1. A method for remote device monitoring comprising:
collecting image data of a device interface source;
extracting a parameterized representation of the device interface source;
detecting device state;
characterizing, based on the device state, an extracted interface representation from the parameterized representation, the characterization being a process based on the device state; and
exposing at least one access interface to the extracted interface representation.

2. The method of claim 1 , further comprising applying image data transformations to a region of the image data and thereby generating formatted image data of the device interface source.

3. The method of claim 2 , further comprising streaming the formatted image data to a remote monitoring system, wherein the remote monitoring system is configured to manage a set of media streams of formatted image data.

4. The method of claim 1 , wherein characterizing information data from the parameterized representation comprises characterizing at least two sets of information data from the parameterized representation.

5. The method of claim 1 , wherein extracting the parameterized representation comprises performing character recognition in the image data.

6. The method of claim 1 , wherein characterizing the extracted interface representation from the parameterized representation comprises selecting a type of information data for characterization based on the detected device state.

7. The method of claim 1 , wherein collecting image data comprises collecting audio-visual data; and further comprising detecting audio output of the device interface source and characterizing informational data based in part on the detected audio output.

8. The method of claim 1 , wherein collecting image data of a device interface source further comprises collecting image data of a second device interface source; and further comprising:
processing the image data associated with the second device interface source into a second extracted interface representation; and
wherein the second extracted interface representation is exposed through the at least one access interface.

9. The method of claim 1 , further comprising monitoring the capture status of a device interface source and generating feedback in response to a change of capture status of the device interface source.

10. The method of claim 9 , further comprising tracking user presence in the field of view of the image data and wherein generating feedback in response to a change of capture status comprises announcing a change in capture status during user exit of the field of view.

11. The method of claim 1 , wherein the device interface source is a scale, and the extracted interface representation includes measured weight.

12. The method of claim 1 , wherein device state is conditional on detected user interactions with the device interface source.

13. The method of claim 1 , wherein the device interface source is a digital thermometer, and the extracted interface representation includes temperature.

14. The method of claim 1 , further comprising detecting a displayed mode indicator on the device interface source and updating the device state based on the displayed mode indicator.

15. The method of claim 1 , wherein detecting the device state further comprises detecting state of physical elements of the device interface source.

16. The method of claim 1 , wherein detecting device state comprises detecting an active application of the device interface source.

17. The method of claim 1 , further comprising detecting a user-object in the image data and tracking changes in the device state in association with the user-object.

18. The method of claim 1 , further comprising analyzing patterns of the information data.

19. The method of claim 1 , wherein exposing at least one access interface to the extracted interface representation comprises streaming a rendered interface representation to a remote monitoring system, and further comprising selectively elevating priority of the rendered interface representation according to the informational data.

20. The method of claim 1 , exposing at least one access interface to the extracted interface representation comprises exposing an application programming interface to a historical data record of the information data.

21. A non-transitory computer-readable medium storing instructions that, when executed by one or more computer processors of a computing platform, cause the computing platform to perform operations comprising:
collecting image data of a device interface source;
extracting a parameterized representation of the device interface source;
detecting device state;
characterizing, based on the device state, an extracted interface representation from the parameterized representation, the characterization being a process based on the device state; and
exposing at least one access interface to the extracted interface representation.

22. A system comprising:
one or more computer-readable mediums storing instructions that, when executed by the one or more computer processors, cause a computing platform to perform operations comprising:
collecting image data of a device interface source;
extracting a parameterized representation of the device interface source;
detecting device state;
characterizing, based on the device state, an extracted interface representation from the parameterized representation, the characterization being a process based on the device state; and
exposing at least one access interface to the extracted interface representation.

Assignees

• Grabango Co

Inventors

• Glaser William
• Van Osdol Brian

Classifications

• H05B47/1975

  Gesture control · CPC title

• H05B47/19

  via wireless transmission · CPC title

• H04N7/18 Primary

  Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast · CPC title

• G06F18/213

  Feature extraction, e.g. by transforming the feature space; Summarisation; Mappings, e.g. subspace methods · CPC title

• G06V10/245

  by locating a pattern; Special marks for positioning · CPC title