Portable handheld device with multi-core image processor

Patent metadata

Field	Value
Publication number	US-9338312-B2
Application number	US-201213620936-A
Country	US
Kind code	B2
Filing date	Sep 15, 2012
Priority date	Jul 12, 1997
Publication date	May 10, 2016
Grant date	May 10, 2016

Abstract

A portable handheld device includes an image sensor for capturing an image; an image sensor interface for receiving data from the image sensor; a DRAM for storing the data received by the image sensor interface; an image processor for storing the image the DRAM, the image processor including a plurality of micro-coded processing units; a central processor for instructing the image processor; and an orientation sensor for sensing a rotation of the protable handheld device at a time of sensing the image. The central processor is configured to load the plurality of micro-coded processing units of the image processor with micro-code for performing an affine transform of the data in the DRAM to rotate the data in the DRAM by a rotation corresponding to the rotation sensed by the orientation sensor.

First claim

I claim:

1. A device comprising:
an image sensor configured for capturing an image;
a second image sensor configured for capturing a second image;
an image sensor interface configured for receiving data associated with the image from the image sensor;
a second image sensor interface separate from the image sensor interface and configured for receiving data associated with the second image from the second image sensor;
a memory configured for storing the data associated with the image;
an image processor configured for processing the data associated with the image stored at the memory, the image processor including the image sensor interface and a plurality of processing units;
a central processor configured for instructing the plurality of processing units; and
an orientation sensor configured for sensing a rotation of the device at a time of sensing the image, wherein:
the central processor is configured to load the plurality of processing units of the image processor with code for performing image processing of the data associated with the image;
the image sensor interface, the second image sensor interface and the image processor are integrated as a system-on-chip device; and
the memory is provided external to the system-on-chip device.

2. A device as claimed in claim 1 , wherein the image sensor interface is a state machine, and the state machine is configured to send frame sync pulses to the image sensor to enable reading of the image captured by the image sensor.

3. A device as claimed in claim 1 , wherein the central processor is integrated as part of the system-on-chip device, and the image sensor interface, the image processor and the central processor all share a common wafer.

4. A device as claimed in claim 3 , further comprising a data cache integrated in the system-on-chip device, the data cache being shared by the plurality of processing units via a data bus.

5. A device as claimed in claim 4 , wherein each of the plurality of processing units includes an individual input buffer and an individual output buffer, each individual input buffer and each individual output buffer being connected to the data bus to thereby achieve connection of each processing unit with the data cache.

6. A device as claimed in claim 1 , wherein each processing unit includes an ALU.

7. A device as claimed in claim 6 , further comprising a crossbar connecting each ALU in a ring topology, the crossbar effecting parallel connection of the plurality of processing units.

8. A device as claimed in claim 7 , further comprising a common synchronization register shared by the plurality of processing units, the common synchronization register for synchronizing one or more of the processing units to function as a single process.

9. A device as claimed in claim 1 , wherein the plurality of processing units is a plurality of micro-coded processing units and the central processor is configured to load the plurality of micro-coded processing units of the image processor with micro-code.

10. A device as claimed in claim 1 , wherein the memory is a DRAM.

11. A device as claimed in claim 1 , wherein the image processing rotates the data associated with the image in the memory by a rotation corresponding to the rotation sensed by the orientation sensor.

12. A device as claimed in claim 11 , wherein the image processing is an affine transformation of the data associated with the image in the memory.

13. A device as claimed in claim 1 , wherein the image sensor interface provides control information to the image sensor.

14. A device as claimed in claim 1 , wherein each individual processing unit of the plurality of processing units includes an address generator, the address generator is configured to transfer data to and from the memory.

15. A device as claimed in claim 1 , wherein the plurality of processing units interfaces with the image sensor interface.

16. A device as claimed in claim 8 , wherein the common synchronization register contains therein synchronization bits from each of the plurality of processing units, and the central processor is configured to write the synchronization bits from each of the multiple processing units to the common synchronization register for performing image processing of the data associated with the image.

Assignees

• Silverbrook Kia
• Google Inc

Inventors

• Silverbrook Kia

Classifications

• H10W42/405

  using active circuits · CPC title

• H04N25/00

  Circuitry of solid-state image sensors [SSIS]; Control thereof · CPC title

• B41J2/14 Primary

  Structure thereof {only for on-demand ink jet heads} · CPC title

• H04N23/843

  Demosaicing, e.g. interpolating colour pixel values · CPC title

• B42D2035/34

  Operations & Transport · mapped topic