Identifying and configuring controls on a control panel

Patent metadata

Field	Value
Publication number	US-9612131-B2
Application number	US-201213442564-A
Country	US
Kind code	B2
Filing date	Apr 9, 2012
Priority date	Apr 9, 2012
Publication date	Apr 4, 2017
Grant date	Apr 4, 2017

Abstract

A method includes receiving, at a processor from a camera, image data associated with a control panel. The method also includes presenting one or more images at a display. The one or more images are generated based on the image data. The method includes determining a location of a first control of the control panel based on the image data and based on control settings data. The method includes providing an indication of the location of the first control at the display, and providing an indication of a desired configuration of the first control at the display.

First claim

What is claimed is:

1. A method comprising:
receiving, at a processor from a movable camera, image data associated with a control panel;
presenting one or more images at a display available to a pilot, wherein the one or more images are generated based on the image data;
retrieving control settings data from an electronic checklist stored in a memory accessible by the processor;
comparing the image data to the control settings data to determine a first location of a first control of the control panel relative to a particular location of the control panel corresponding to the image data;
determining whether the particular location corresponds to the first location;
in response to the particular location not corresponding to the first location, providing a first indication of the first location of the first control at the display; and
in response to the particular location corresponding to the first location:
determining a desired configuration of the first control based on the control settings data;
providing a second indication of the desired configuration of the first control at the display;
after providing the second indication, receiving input indicating that a configuration of the first control has been modified;
after receiving the input, receiving second image data associated with the control panel, wherein the second image data represents a modified configuration of the first control;
determining whether the modified configuration of the first control corresponds to the desired configuration of the first control based on the second image data and based on the control settings data; and
in response to a determination that the modified configuration of the first control corresponds to the desired configuration of the first control, presenting an icon at the display, wherein the icon indicates that the modified configuration of the first control corresponds to the desired configuration.

2. The method of claim 1 , further comprising:
in response to the pilot selecting the icon, determining that a second control is to be configured;
presenting one or more additional images at the display, wherein the one or more additional images are generated based on the second image data;
comparing the second image data to the control settings data to determine a second location of the second control of the control panel relative to a second particular location of the control panel corresponding to the second image data;
determining whether the second particular location corresponds to the second location;
in response to the second particular location not corresponding to the second location, providing a third indication of the second location of the second control at the display, wherein the third indication indicates a direction to move the movable camera so that the movable camera is pointed at the second control; and
in response to the second particular location corresponding to the second location, providing a fourth indication of a second desired configuration of the second control at the display, wherein the second desired configuration of the second control is determined based on the control settings data.

3. The method of claim 2 , wherein the control settings data of the electronic checklist comprises a plurality of steps, wherein the first location and the desired configuration correspond to a first step of the plurality of steps, wherein the second location and the second desired configuration correspond to a second step of the plurality of steps, and wherein the second step is ordered subsequent to the first step in the control settings data.

4. The method of claim 1 , further comprising determining calibration data based on a second location of the movable camera relative to a calibration feature, wherein the first location is determined in part based on the calibration data.

5. The method of claim 1 , wherein the movable camera is movable relative to the control panel.

6. The method of claim 1 , further comprising:
determining whether the first location is in the field of view of the movable camera, and
if the first location is not in the field of view of the movable camera, the first indication indicating a direction to move the movable camera.

7. An apparatus comprising:
a movable camera;
a processor; and
a memory comprising:
an electronic checklist comprising control settings data; and
instructions that, when executed by the processor, cause the processor to:
receive image data associated with a control panel from the movable camera;
present one or more images at a display that is available to a pilot, wherein the one or more images are generated based on the image data;
compare the image data to the control settings data to determine a first location of a first control of the control panel relative to a particular location of the control panel corresponding to the image data;
determine whether the particular location corresponds to the first location;
in response to the particular location not corresponding to the first location, provide a first indication of the first location of the first control at the display; and
in response to the particular location corresponding to the first location:
determine a desired configuration of the first control based on the control settings data;
provide a second indication of the desired configuration of the first control at the display;
determine whether a configuration of the first control has been modified;
receive second image data associated with the control panel, wherein the second image data represents a modified configuration of the first control; and
determine whether the modified configuration of the first control corresponds to the desired configuration of the first control based on the second image data and based on the control settings data.

8. The apparatus of claim 7 , wherein the instructions, when executed by the processor, cause the processor to:
in response to the pilot selecting the icon, determine that a second control is to be configured;
present one or more additional images at the display, wherein the one or more additional images are generated based on the second image data;
compare the second image data to the control settings data to determine a second location of the second control of the control panel relative to a second particular location of the control panel corresponding to the second image data;
determine whether the second particular location corresponds to the second location;
in response to the second particular location not corresponding to the second location, provide a third indication of the second location of the second control at the display; and
in response to the second particular location corresponding to the second location, provide a fourth indication of a second desired configuration of the second control at the display, wherein the second desired configuration is determined based on the control settings data.

9. The apparatus of claim 8 , wherein the second location is identified relative to the first location, a third location of a reference control, a fourth location of a reference marking, or a combination thereof.

10. The apparatus of claim 7 , further comprising at least one sensor, wherein the first location is further determined based on sensor data received from the at least one sensor, and wherein the at least one sensor comprises an inertial sensor, a motion sensor, an orientation sensor, or a combination thereof.

11. The apparatus of claim 7 , wherein the control settings data of the electronic checklist comprises control data and directional data, wherein the control data identifies a layout of a plurality of controls of the contr

Assignees

• Smailus Thomas O
• Rosman Lafever Monica C
• Boeing Co

Inventors

• Smailus Thomas O
• Rosman Lafever Monica C

Classifications

• B64D31/04

  actuated personally · CPC title

• B64D43/00

  Arrangements or adaptations of instruments · CPC title

• G01C23/00 Primary

  Combined instruments indicating more than one navigational value, e.g. for aircraft; Combined measuring devices for measuring two or more variables of movement, e.g. distance, speed or acceleration · CPC title