Methods and systems for determining image processing operations relevant to particular imagery

Patent metadata

Field	Value
Publication number	US-9256806-B2
Application number	US-201414189236-A
Country	US
Kind code	B2
Filing date	Feb 25, 2014
Priority date	Mar 19, 2010
Publication date	Feb 9, 2016
Grant date	Feb 9, 2016

Abstract

Image data, such as from a mobile phone camera, is analyzed to determine a colorfulness metric (e.g., saturation) or a contrast metric (e.g., Weber contrast). This metric is then used in deciding which of, or in which order, plural different image recognition processes should be invoked in order to present responsive information to a user. A great number of other features and arrangements are also detailed.

First claim

The invention claimed is:

1. A method of improving the functioning of a cell phone, the cell phone including a battery, a camera, and a processing system, the cell phone further being equipped to perform plural different types of image recognition operations including at least barcode recognition and digital watermark decoding on frames of imagery captured by the camera, the method comprising the acts:
performing a plurality of preliminary image processing operations on each of 15 or more frames of imagery captured per second, said performing requiring application of less than 10% of computational capabilities of the cell phone processing system, said preliminary image processing operations including analyzing each of said frames of imagery to determine at least one of: a colorfulness metric, a color saturation metric, or a contrast metric, said preliminary image processing operations yielding intermediate data, including said determined metric, for each of said frames;
(a) based at least in part on said determined metric, performing a barcode recognition operation and not a digital watermark decoding operation on a first of said frames of imagery captured by the camera;
(b) based at least in part on said determined metric, performing a digital watermark decoding operation and not a barcode recognition operation on a second of said frames of imagery captured by the camera;
wherein said barcode recognition and watermark decoding operations are performed using said cell phone processing system, without off-loading to a processor remote from the cell phone; and
wherein a type of image recognition process performed on a frame of captured imagery, as well as a type of image recognition process not-performed, are based on said determined metric;
wherein the cell phone does not perform all of said plural types of image recognition operations on said first or second frames of captured imagery, but rather selects among said plural types of image recognition operations based on imagery characteristics, thereby conserving battery power and processing resources, despite performing said barcode recognition and watermark decoding operations using said cell phone processing system without reliance on a processor remote from the cell phone.

2. The method of claim 1 wherein selection of a particular image recognition operation to apply to a particular frame of captured imagery proceeds without input from a user, wherein the phone thereby seems to respond intuitively to content depicted in the captured imagery.

3. The method of claim 1 wherein said plural different types of image recognition operations additionally includes optical character recognition, wherein:
act (a) includes performing both said barcode recognition operation and said optical character recognition operation, and not said digital watermark decoding operation; and
act (b) includes performing said digital watermark decoding operation, and not said barcode recognition operation and not said optical character recognition operation.

4. The method of claim 1 wherein said plural different types of image recognition operations additionally includes facial recognition, wherein:
act (a) includes performing said barcode recognition operation, and not said digital watermark decoding operation and not said facial recognition operation; and
act (b) includes performing said digital watermark decoding operation and said facial recognition operation, and not said barcode recognition operation.

5. The method of claim 4 wherein said plural different types of image recognition operations additionally includes optical character recognition, wherein:
act (a) includes performing both said barcode recognition operation and said optical character recognition operation, and not said digital watermark decoding operation and not said facial recognition operation; and
act (b) includes performing both said digital watermark decoding operation and said facial recognition operation, and not said barcode recognition operation and not said optical character recognition operation.

6. The method of claim 1 in which said plurality of preliminary image processing operations includes analyzing to determine a colorfulness metric.

7. The method of claim 1 in which said plurality of preliminary image processing operations includes analyzing to determine a color saturation metric.

8. The method of claim 1 in which said plurality of preliminary image processing operations includes analyzing to determine a contrast metric.

9. The method of claim 1 in which said plurality of preliminary image processing operations includes converting red/green/blue signals of frames of captured image data into another representation in which color is represented separately than luminance.

10. The method of claim 1 in which the intermediate data indicates more than 50% of said first frame of captured image data has a color saturation of less than 50%.

11. The method of claim 1 in which the first intermediate data indicates more than 50% of said first frame of captured image data has a color saturation of less than 15%.

12. A cell phone including a battery, a camera, a memory containing software instructions, and a processing system, said instructions configuring the cell phone to define plural recognition agents, said agents including a barcode recognition agent that extracts barcode data from camera-captured imagery, and a watermark decoding agent that extracts watermark data from camera-captured imagery, said barcode recognition agent and watermark decoding agent not off-loading processing to a processor remote from the cell phone, said instructions in the memory further configuring the cell phone to perform acts including:
performing a plurality of preliminary image processing operations on each of 15 or more frames of imagery captured per second, said performing requiring application of less than 10% of computational capabilities of the cell phone processing system, said preliminary image processing operations including analyzing each of said frames of imagery to determine at least one of: a colorfulness metric, a color saturation metric, or a contrast metric, said preliminary image processing operations yielding intermediate data, including said determined metric, for each of said frames;
based at least in part on said determined metric, selecting less than all of said plural recognition agents to invoke on one of said frames of imagery; and
invoking the selected recognition agent(s) to operate on said frame of imagery, and not invoking an unselected recognition agent to operate on said frame of imagery;
wherein the recognition agent(s) invoked to operate on said frame of imagery, as well as the recognition agent not invoked to operate on said frame of imagery, are identified based on said determined metric, wherein the cell phone does not invoke all of said plural recognition agents to operate on said frame of imagery, but rather selects among said plural recognition agents based on image characteristics, thereby conserving battery power and processing resources, despite performing said barcode recognition and watermark decoding operations using said cell phone processing system without reliance on a processor remote from the cell phone.

13. The cell phone of claim 12 wherein said plural recognition agents additionally include a facial recognition agent that extracts facial identification information from camera-captured imagery.

14. The cell phone of claim 13 wherein said plural recognition agents additionally include an optical character recognition agent that extracts text information from camera-captured imagery.

Assignees

• Digimarc Corp

Inventors

• Aller Joshua V
• Rodriguez Tony F

Classifications

• G06F18/21

  Design or setup of recognition systems or techniques; Extraction of features in feature space; Blind source separation · CPC title

• G06Q30/0281 Primary

  Customer communication at a business location, e.g. providing product or service information, consulting · CPC title

• G06K9/00973

  Physics · mapped topic

• G06K9/00268

  Physics · mapped topic

• G06Q30/06

  Buying, selling or leasing transactions · CPC title