Smart marker

Patent metadata

Field	Value
Publication number	US-10239345-B2
Application number	US-201715605025-A
Country	US
Kind code	B2
Filing date	May 25, 2017
Priority date	May 25, 2017
Publication date	Mar 26, 2019
Grant date	Mar 26, 2019

Abstract

Conventional whiteboards or chalkboards on which writing is effected using Dry-Erase or chalk markers are enabled for digital conversion of the writing into the digital domain. An indoor positioning subsystem determines the absolute position of a marker with a dry-erase or another conventional writing tip within a local coordinate system to track the location of the marker on the board. Ultra wide band (UWB) is an example technology that may be used for this purpose. An inertial sensor in the marker captures sensor data generated by handwriting by sensing the movement of the marker, essentially tracking the relative positioning and motion of the handwriting. The absolute position and relative positions are then processed to generate digital handwriting which may be refined using deep learning subsystem(s) for handwriting recognition and classification for the data captured by the inertial sensor to determine the most appropriate character or shape of the writing.

First claim

What is claimed is:

1. A system comprising:
at least one hand-holdable marker with a substance-dispensing end;
at least one substrate a person can mark on by moving the substance-dispensing end of the marker against the substrate to cause substance from the substance-dispensing end to be deposited on the substrate;
at least one motion sensor operably associated with the marker to sense motion of the marker; and
at least one processor configured for receiving signals from the motion sensor for conversion of the signals to a digital domain, the processor being configured to access instructions executable by the processor to:
implement at least one learning algorithm for conversion of the signals from the motion sensor to a digital domain, the learning algorithm receiving and classifying the signals and outputting a result prompting refinement of writing using the marker, refinements of writing using the marker being used to adjust the learning algorithm.

2. The system of claim 1 , wherein the motion sensor comprises at least one gyroscope on the marker.

3. The system of claim 1 , wherein the motion sensor comprises at least one accelerometer on the marker.

4. The system of claim 1 , comprising:
at least one location system configured for generating signals representative of a location of the marker relative to the substrate.

5. The system of claim 4 , wherein the location system comprises at least one ultra wide band (UWB) transceiver.

6. The system of claim 5 , wherein the UWB transceiver is a first UWB transceiver on the marker, and the location system comprises at least a second UWB transceiver not on the marker and configured for communicating with the first UWB transceiver.

7. The system of claim 4 , wherein the location system comprises at least one global positioning system (GPS) transceiver on the marker.

8. The system of claim 1 , wherein the instructions configure the processor to:
output and indication of what the learning algorithm believes was written using the marker;
receiving input indicating whether the indication is correct; and
refining the learning algorithm based at least in part on the input.

9. A system, comprising:
a first subsystem configured to determine an absolute position of a marker;
a second subsystem on the marker configured to capture sensor data generated by handwriting on the marker board using the marker; and
a third subsystem to generate digital handwriting refined using a learning algorithm for handwriting recognition based on data from the first and second subsystems, the learning algorithm receiving and classifying signals from the second subsystem and outputting a result prompting refinement of writing using the marker.

10. The system of claim 9 , wherein the receivers comprise at least one ultra wide band (UWB) transceiver.

11. The system of claim 9 , wherein the second subsystem comprises at least one gyroscope.

12. The system of claim 9 , wherein the second subsystem comprises at least one accelerometer.

13. A method comprising:
providing a marker for writing on a marker board;
capturing signals from a motion sensor in the marker as the marker is moved relative to the marker board; and
implementing at least one learning algorithm for conversion of the signals from the motion sensor to a digital domain, the learning algorithm receiving and classifying the signals and outputting a result useful for refining use of the marker, refinements of use being useful to adjust the learning algorithm.

14. The method of claim 13 , wherein capturing signals from a motion sensor in the marker comprises capturing signals from at least one accelerometer.

15. The method of claim 13 , wherein capturing signals from a motion sensor in the marker comprises capturing signals from at least one gyroscope.

16. The method of claim 13 , comprising determining a location of the marker with respect to the marker board using ultra wide band (UWB) signaling.

17. The method of claim 13 , comprising determining a location of the marker with respect to the marker board using global positioning satellite (GNSS) information from the marker.

18. The method of claim 13 , comprising using the signals from the motion sensor and a location of the marker with respect to the marker board to generate a digitized version of marks on the marker board.

19. The method of claim 13 , wherein capturing signals from a motion sensor in the marker comprises capturing signals from at least one magnetometer.

Assignees

• Sony Corp

Inventors

• Youssef Mohamed

Classifications

• G01S2201/01

  adapted for specific applications or environments · CPC title

• G06V30/19133

  Interactive pattern learning with a human teacher · CPC title

• G06V10/82

  using neural networks · CPC title

• G06F18/41

  Interactive pattern learning with a human teacher · CPC title

• G06F3/0346

  with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors · CPC title