Enhanced printer and printer operator interactions

Patent metadata

Field	Value
Publication number	US-11055037-B2
Application number	US-201916584492-A
Country	US
Kind code	B2
Filing date	Sep 26, 2019
Priority date	Sep 26, 2019
Publication date	Jul 6, 2021
Grant date	Jul 6, 2021

Abstract

System and method for enhanced printer and printer operator interactions. One embodiment includes a maintenance controller in communication with a camera having a field of view overlapping with a field of view of a human operator, and in communication with a display to display images for the human operator. The maintenance controller obtains a first image of the printer components, analyzes the first image to determine locations of the printer components in the first image based on the image data of the printer components stored in the memory, and modifies the first image to include highlighted areas corresponding with the locations of the printer components. The maintenance controller receives an instruction from the human operator indicating one of the highlighted areas as a selected printer component, retrieves the maintenance information of the selected printer component from memory, and sends the maintenance information to the display.

First claim

What is claimed is:

1. A system comprising:
memory configured to store image data of printer components, and maintenance information of the printer components; and
a maintenance controller in communication with a camera having a field of view overlapping with a field of view of a human operator, and in communication with a display to display images for the human operator,
the maintenance controller configured to obtain from the camera a first image of the printer components, to analyze the first image to determine locations of the printer components in the first image based on the image data of the printer components stored in the memory, to modify the first image to include highlighted areas corresponding with the locations of the printer components, and to send the modified first image to the display,
the maintenance controller configured to receive an instruction from the human operator indicating one of the highlighted areas as a selected printer component, to retrieve the maintenance information of the selected printer component from the memory based on the instruction, and to send the maintenance information to the display, and
the maintenance controller is configured to obtain from the camera a second image of the printer components, wherein the instruction from the human operator is included in the second image.

2. The system of claim 1 wherein:
the maintenance controller is configured to analyze the second image to identify a gesture in the second image that selects the one of the highlighted areas.

3. The system of claim 1 wherein:
the maintenance controller is configured to analyze the second image to identify a pointing device in the second image that selects the one of the highlighted areas.

4. The system of claim 1 wherein:
the instruction includes a request for additional maintenance information of the selected printer component, and
the maintenance controller is configured to retrieve the additional maintenance information of the selected printer component from memory, and to send the additional maintenance information to the display.

5. The system of claim 1 wherein:
the instruction includes a request for removal information of the selected printer component, and
the maintenance controller is configured to retrieve the removal information of the selected printer component from memory, and to send the removal information to the display.

6. The system of claim 1 wherein:
the highlighted areas include bounding boxes that accentuate a perimeter of the locations of the printer components in the first image, the bounding boxes having different colors to visually differentiate the printer components.

7. The system of claim 1 wherein:
at least one of the highlighted areas corresponds with a printer component that is not present in the first image.

8. A method comprising:
storing, in memory, image data of printer components and maintenance information of the printer components;
obtaining, with a maintenance controller in communication with a camera having a field of view overlapping with a field of view of a human operator, a first image of the printer components from the camera;
analyzing the first image to determine locations of the printer components in the first image based on the image data of the printer components stored in the memory;
modifying the first image to include highlighted areas corresponding with the locations of the printer components;
sending the modified first image to a display for the human operator;
receiving an instruction from the human operator indicating one of the highlighted areas as a selected printer component;
obtaining, from the camera, a second image of the printer components, wherein the instruction from the human operator is included in the second image;
retrieving the maintenance information of the selected printer component from the memory based on the instruction; and
sending the maintenance information to the display.

9. The method of claim 8 further comprising:
analyzing the second image to identify a gesture in the second image that selects the one of the highlighted areas.

10. The method of claim 8 further comprising:
analyzing the second image to identify a pointing device in the second image that selects the one of the highlighted areas.

11. A non-transitory computer readable medium including programmed instructions which, when executed by a processor, are operable for performing a method, the method comprising:
storing, in memory, image data of printer components and maintenance information of the printer components;
obtaining, with a maintenance controller in communication with a camera having a field of view overlapping with a field of view of a human operator, a first image of the printer components from the camera;
analyzing the first image to determine locations of the printer components in the first image based on the image data of the printer components stored in the memory;
modifying the first image to include highlighted areas corresponding with the locations of the printer components;
sending the modified first image to a display for the human operator;
receiving an instruction from the human operator indicating one of the highlighted areas as a selected printer component;
obtaining, from the camera, a second image of the printer components, wherein the instruction from the human operator is included in the second image;
retrieving the maintenance information of the selected printer component from the memory based on the instruction; and
sending the maintenance information to the display.

12. The non-transitory computer readable medium of claim 11 wherein the method further comprises:
analyzing the second image to identify a gesture in the second image that selects the one of the highlighted areas.

13. The non-transitory computer readable medium claim 11 wherein the method further comprises:
analyzing the second image to identify a pointing device in the second image that selects the one of the highlighted areas.

14. The non-transitory computer readable medium claim 11 wherein:
the highlighted areas include bounding boxes that accentuate a perimeter of the locations of the printer components in the first image, the bounding boxes having different colors to visually differentiate the printer components.

15. A system comprising:
memory configured to store image data of printer components, and maintenance information of the printer components; and
a maintenance controller in communication with a camera having a field of view overlapping with a field of view of a human operator, and in communication with a display to display images for the human operator,
the maintenance controller configured to obtain from the camera a first image of the printer components, to analyze the first image to determine locations of the printer components in the first image based on the image data of the printer components stored in the memory, to modify the first image to include highlighted areas corresponding with the locations of the printer components, and to send the modified first image to the display,
the maintenance controller configured to obtain audio from the camera that includes an instruction from the human operator indicating one of the highlighted areas as a selected printer component, to retrieve the maintenance information of the selected printer component from the memory based on the instruction, and to send the maintenance information to the display.

16. A method comprising:
storing, in memory, image data of printer components and maintenance information of the print

Assignees

• Ernst Larry M
• Munson Mike
• Ricoh Co Ltd

Inventors

• Ernst Larry M
• Munson Mike

Classifications

• G06V40/20

  Movements or behaviour, e.g. gesture recognition (recognition of facial expressions G06V40/16) · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06F3/1229

  Printer resources management or printer maintenance, e.g. device status, power levels · CPC title

• G06F3/121 Primary

  Facilitating exception or error detection and recovery, e.g. fault, media or consumables depleted · CPC title

• H04N1/00251

  with an apparatus for taking photographic images, e.g. a camera · CPC title