Note recognition for overlapping physical notes

Patent metadata

Field	Value
Publication number	US-9412174-B2
Application number	US-201414514473-A
Country	US
Kind code	B2
Filing date	Oct 15, 2014
Priority date	Oct 16, 2013
Publication date	Aug 9, 2016
Grant date	Aug 9, 2016

Abstract

Techniques for creating and manipulating software notes representative of physical notes are described. A computing device includes a processor, an image collection module executable by the processor and configured to receive an input image of an environment having a plurality of overlapping physical notes, and an image processing engine executable by the processor and configured to process the input image with the computing device to identify the plurality of overlapping physical notes in the input image. The image processing engine determines a boundary of each note in the plurality of overlapping physical notes in the input image, and generates a plurality of digital notes corresponding to the determined boundary of each of the overlapping physical notes identified in the input image.

First claim

The invention claimed is:

1. A method comprising:
receiving, by a computing device, an input image of an environment having a plurality of overlapping physical notes, each the plurality of physical notes having a background color and a border having a border color different from the background color;
processing the input image with the computing device to apply a color segmenting process to identify, based on the background colors and the border colors of the overlapping physical notes, borders of each of the overlapping physical notes in the input image;
processing the identified borders of each of the overlapping physical notes as fiducial markers for the respective physical note to extract, based on the fiducial markers, content for each of the respective overlapping physical notes, wherein the extracted content for a respective physical note includes portions of the border of the respective overlapping physical note not covered by another physical note of the plurality of overlapping physical notes;
creating, by the computing device, a plurality of digital notes, each of the digital notes representing a particular one of the identified overlapping physical notes; and
associating, by the computing device, the extracted content for each respective physical note of the identified overlapping physical notes with a corresponding digital note of the plurality of digital notes that represents the respective physical note.

2. The method of claim 1 ,
wherein the color segmenting process is applied in response to determining that the background colors of the plurality of overlapping physical notes are different.

3. The method of claim 1 , further comprising:
displaying, by the computing device, the input image on a first portion of a graphical user interface; and
displaying, by the computing device, the digital notes on a second portion of the graphical user interface.

4. The method of claim 1 , further comprising:
detecting one or more fiducial markers in the input image that are different than the identified borders of each of the overlapping physical notes,
wherein extracting the content from at least one of the overlapping physical notes comprises extracting, by the computing device and based on the identified fiducial markers that are different than the identified borders, the content from the at least one overlapping physical note.

5. The method of claim 1 , wherein extracting the content from each of the plurality of overlapping physical notes includes identifying, by the computing device and based on the fiducial markers for the respective physical note, a size and a shape for each of the respective overlapping physical notes.

6. The method of claim 1 , wherein extracting the content from each of the plurality of overlapping physical notes includes identifying the plurality of notes according to a pattern recognition algorithm.

7. The method of claim 6 , wherein the pattern recognition algorithm includes at least one of a group consisting of:
a support vector machine;
a cross-correlation; and
a template matching.

8. A non-transitory computer-readable medium comprising instructions for causing a programmable processor to:
receive, by a computing device, an input image of an environment having a plurality of overlapping physical notes, each the plurality of physical notes having a background color and a border having a border color different from the background color;
process the input image with the computing device to apply a color segmenting process to identify, based on the background colors and the border colors of the overlapping physical notes, borders of each of the overlapping physical notes in the input image;
process the identified borders of each of the overlapping physical notes as fiducial markers for the respective physical note to extract, based on the fiducial markers, content for each of the respective overlapping physical notes, wherein the extracted content for a particular physical note includes portions of the border of the particular overlapping physical note not covered by another physical note of the plurality of overlapping physical notes;
create a plurality of digital notes, each of the digital notes representing a particular one of the identified overlapping physical notes; and
associate the extracted content for each respective physical note of the identified overlapping physical notes with a corresponding digital note of the plurality of digital notes that represents the respective physical note.

9. A computing device comprising:
one or more processors;
an image collection module executable by the one or more processors to receive an input image of an environment having a plurality of overlapping physical notes, each the plurality of physical notes having a background color and a border having a border color different from the background color; and
an image processing engine executable by the one or more processors to:
process the input image to apply a color segmenting process to identify, based on the background colors and the border colors of the overlapping physical notes, borders of each of the overlapping physical notes in the input image having the border color;
process the identified borders of each of the overlapping physical notes as fiducial markers for the respective physical note to extract, based on the fiducial markers, content for each of the respective overlapping physical notes, wherein the extracted content for a respective physical note includes portions of the border of the respective overlapping physical note not covered by another physical note of the plurality of overlapping physical notes;
create a plurality of digital notes, each of the digital notes representing a particular one of the identified overlapping physical notes; and
associate the extracted content for each respective physical note of the identified overlapping physical notes with a corresponding digital note of the plurality of digital notes that represents the respective physical note.

10. The device of claim 9 , wherein the image collection module includes an image capture device configured to capture the input image.

11. The device of claim 9 , wherein the image collection module includes a communication interface configured to receive the input image.

12. The device of claim 11 , wherein the communication interface includes a wireless communication device configured to receive the input image.

13. The device of claim 11 , wherein the communication interface includes a wired communication device configured to receive the input image.

14. The device of claim 9 , wherein the image processing engine is executable by the one or more processors to:
apply the color segmenting process in response to determining that the background colors of the plurality of overlapping physical notes are different.

15. The device of claim 9 , where the image processing engine further configured to:
display the input image on a first portion of a graphical user interface; and
display the digital notes on a second portion of the graphical user interface.

16. The device of claim 9 , wherein the image processing engine is further executable by the one or more processors to:
detect one or more fiducial markers in the input image that are different than the identified borders of each of the overlapping physical notes,
wherein, to extract the content from at least one of the overlapping physical notes, the image processing engine is executable by the one or more processors to determine, based on the identified fiducial markers that are different than the identified bord

Assignees

• 3M Innovative Properties Co

Inventors

• Somasundaram Guruprasad
• Sivalingam Ravishankar
• Mahli David M
• Moran Cristin E
• Moore Richard J
• Axelsson Pontus
• Rotstein Michael
• Persson Roland

Classifications

• H04L51/08 Primary

  Annexed information, e.g. attachments · CPC title

• G06V30/414

  Extracting the geometrical structure, e.g. layout tree; Block segmentation, e.g. bounding boxes for graphics or text · CPC title

• G06F40/171

  by use of digital ink · CPC title

• G06F40/169

  Annotation, e.g. comment data or footnotes · CPC title

• G06V10/245

  by locating a pattern; Special marks for positioning · CPC title