Apparatus, video processing unit and method for clustering events in a content addressable memory

Patent metadata

Field	Value
Publication number	US-10892012-B2
Application number	US-201816110990-A
Country	US
Kind code	B2
Filing date	Aug 23, 2018
Priority date	Aug 23, 2018
Publication date	Jan 12, 2021
Grant date	Jan 12, 2021

Abstract

An apparatus, vision processing unit, and method are provided for clustering motion events in a content addressable memory. A motion event is received including coordinates in an image frame that have experienced a change and a timestamp of the change. A determination is made as to whether determine whether there is a valid entry in the memory having coordinates within a predefined range of coordinates included in the motion event. In response to a determination that there is the valid entry having the coordinates within the predefined range of coordinates included in the motion event, write to the valid entry the coordinates and the timestamp in the motion event.

First claim

What is claimed:

1. An apparatus for storing motion events, comprising:
a memory comprising a memory array; and
control logic coupled to the memory and configured to:
receive a motion event including coordinates in an image frame that have experienced a change and a timestamp of the change;
determine whether there is a valid entry in the memory array having coordinates within a predefined range of coordinates included in the motion event; and
in response to a determination that there is the valid entry having the coordinates within the predefined range of coordinates included in the motion event, write to the valid entry the coordinates and the timestamp in the motion event.

2. The apparatus of claim 1 , wherein the control logic is further configured to:
in response to a determination that there is no valid entry having coordinates within the predefined range of coordinates included in the motion event, determine whether there is an invalid entry in the memory array; and
write, to the invalid entry, the coordinates and the timestamp included in the motion event, in response to a determination that there is the invalid entry in the memory array.

3. The apparatus of claim 2 , wherein the control logic is further configured to:
in response to a determination that there is no invalid entry in the memory array, determine whether there is a valid entry that has been in the memory array longer than a threshold time; and
write to the valid entry, having been in the memory array longer than the threshold time, the coordinates and the timestamp included in the motion event, in response to a determination that there is the valid entry that has been in the memory array longer than the threshold time.

4. The apparatus of claim 1 , wherein the coordinates include a first coordinate value and a second coordinate value indicating a location in the image frame and an image related value at the first coordinate value and the second coordinate value, wherein the determining whether there is the valid entry in the memory array, having the coordinates within the predefined range of coordinates, determines whether the first coordinate value and the second coordinate value are each within the predefined range of values.

5. The apparatus of claim 1 , wherein the memory array comprises a content addressable memory, wherein the coordinates include a first coordinate value and a second coordinate value indicating a location in the image frame of a video having a polarity change of a pixel at the first and the second coordinate values, and wherein entries in the memory array include polarity change values for pixels at the first and the second coordinate values that have changed to detect movement in the video.

6. The apparatus of claim 5 , wherein each entry of the entries in the content addressable memory stores a polarity change value for one cluster of motion events of a plurality of clusters of pixels in the image frame, wherein each cluster of pixels has a cluster size of C×C, where C=2k, wherein the predefined range of values comprises k, further comprising:
a valid bit comparator coupled to the entry to determine whether a valid bit in the entry indicates that the entry is valid and to output a signal to cause a coordinate comparator to determine whether the first and second coordinates in the entry and event differ by the predefined range of values in response to determining that the entry is valid; and
an invalid bit comparator coupled to the entry to determine whether the valid bit in the entry indicates that the entry is invalid and to output a signal to a priority encoder to select one of indicated the entries that are invalid to send to an evict decoder to use the selected invalid entry to write the first and second coordinates, the polarity change, and the timestamp in the event to the entry when no entry has first and second coordinates within the predefined range of values.

7. The apparatus of claim 5 , wherein there are a plurality of content addressable memories, one content addressable memory for each sub-frame of the image frame, wherein each sub-frame includes a region of non-overlapping coordinates of the image frame with respect to other sub-frames, and wherein each content addressable memory of the content addressable memories, associated with a sub-frame, has entries having first and second coordinates within the region of the sub-frame associated with the content addressable memory, wherein the control logic is further configured to:
parallel process events having first and second coordinates for different sub-frames at the content addressable memories.

8. The apparatus of claim 5 , wherein the control logic includes:
a first decoder;
a second decoder;
for each entry of entries in the memory, entry logic comprising:
a first comparator, coupled to the entry and the first decoder, to determine whether the first and second coordinates from the entry differ from the first and second coordinates from the event, respectively, within the predefined range of values, and send a signal to the first decoder to write the first and second coordinates, the polarity change, and the timestamp in the event to the entry in response to the first and second coordinates from the entry and the event differing by the predefined range of values;
a second comparator coupled to the entry to determine whether a value for a valid bit in the entry indicates that the entry is valid and to output a signal to cause the first comparator to determine whether the first and second coordinates in the entry and event differ by the predefined range of values in response to determining that the entry is valid; and
a third comparator coupled to the entry to determine whether a value for a valid bit in the entry indicates that the entry is invalid and to output a signal to a priority encoder to select one of indicated the entries that are invalid to send to the second decoder to use the selected invalid entry to write the first and second coordinates, the polarity change, and the timestamp in the event to the entry when no entry has first and second coordinates within the predefined range of values.

9. The apparatus of claim 8 , wherein each valid entry in the memory array includes a cluster tag indicating a first coordinate spread and a second coordinate spread comprising spatial spreads for the first and the second coordinates of motion event information clustered in the entry, wherein the first decoder, in response to an output signal sent by the first comparator in response to the first and second coordinates from the entry and the event differing within the predefined range of values, is further configured to:
read the first coordinate spread and the second coordinate spread from the cluster tag in the entry;
increment the first coordinate spread and increment the second coordinate spread read from the entry; and
write the first coordinate spread and the second coordinate spread, the polarity change, and the timestamp to the valid entry in the memory array.

10. The apparatus of claim 9 , wherein the first comparator, second comparator, and the third comparator perform comparison logic on a first clock cycle, wherein the first decoder reads the first and the second coordinate spreads in the cluster tag and performs the increment of the first coordinate spread and/or the second coordinate spread on a second clock cycle, wherein the second decoder writes to an invalid entry on a second clock cycle, and wherein the first decoder writes the first and the second coordinate spreads for the cluster tag to the valid entry on a third clock cycle.

11. The apparatus of claim 8 , wherein the control logic further includes:
a t

Assignees

• Intel Corp

Inventors

• Majumder Turbo
• Paul Somnath
• Augustine Charles
• Khellah Muhammad M

Classifications

• G11C15/04 Primary

  using semiconductor elements · CPC title

• G06T1/60

  Memory management · CPC title

• G11C15/00

  Digital stores in which information comprising one or more characteristic parts is written into the store and in which information is read-out by searching for one or more of these characteristic parts, i.e. associative or content-addressed stores · CPC title

• G11C8/10

  Decoders · CPC title

• G06T2207/10016

  Video; Image sequence · CPC title