Action detection and activity classification

Patent metadata

Field	Value
Publication number	US-9694241-B2
Application number	US-201615132499-A
Country	US
Kind code	B2
Filing date	Apr 19, 2016
Priority date	Jan 19, 2012
Publication date	Jul 4, 2017
Grant date	Jul 4, 2017

Abstract

Activities, actions and events during user performance of physical activity may be detected using various algorithms and templates. Templates may include an arrangement of one or more states that may identify particular event types and timing between events. Templates may be specific to a particular type of activity (e.g., types of sports, drills, events, etc.), user, terrain, time of day and the like.

First claim

What is claimed is:

1. A system, comprising:
an activity processing system having at least a first processor and a first memory, the first memory storing computer-executable instructions that, when executed by the at least a first processor, cause the activity processing system to:
receive identification of an athletic activity to be performed;
determine, based on the received identification of the athletic activity to be performed, a sensor subscription;
transmit, to a sensor system, the determined sensor subscription;

a sensor system including a plurality of sensors, at least a second processor and at least a second memory storing computer-executable instructions that, when executed, cause the sensor system to:
receive, from the plurality of sensors, raw athletic activity data associated with a potential corresponding event during performance of the athletic activity;
process the raw athletic activity data;
transmit, from the sensor system to the activity processing system, the processed athletic activity data, transmitting the processed athletic activity data to the activity processing system including controlling, by the sensor system, at least one of a type of data and an amount of data transmitted based on the determined sensor subscription;

the activity processing system further including instructions that, when executed, cause the activity processing system to:
compare the received processed athletic activity data to a plurality of physical action templates to determine whether a physical activity of at least one physical action template of the plurality of physical action templates has occurred, the comparing including:
determining a plurality of required states associated with the physical activity of the at least one physical action template and forming a portion of the at least one physical action template;
for each of the plurality of required states, determine a state-specific set of one or more constraints defining match characteristics of a corresponding required event,

determine, based on the comparison, a match score for the one or more constraints;
determine, based on the match score, whether the physical activity has occurred; and
transmit a result of the determining whether the physical activity has occurred to one or more devices.

2. The system of claim 1 , wherein a state-specific set of one or more constraints for a first state of the plurality of required states includes a relative constraint defining a required relationship between the first state and a second state of the plurality of required states, and
wherein the state-specific set of one or more constraints further includes a scoring distribution for determining the match score for the one or more constraints.

3. The system of claim 2 , wherein the state-specific set of one or more constraints further includes a first parameter indicating whether a first characteristic of the corresponding event is to be scored.

4. The system of claim 1 , wherein the state-specific set of one or more constraints further includes a second parameter indicating whether a second characteristic of the corresponding event is to be scored.

5. The system of claim 1 , wherein the state-specific set of one or more constraints includes at least one of: an event occurrence time stamp constraint, an event generation time stamp constraint, an event value constraint and an event duration constraint.

6. The system of claim 1 , wherein the one or more devices include a remote activity tracking system.

7. The system of claim 1 , further including instructions that, when executed, cause the first processor to:
responsive to determining, based on the match score, that the physical activity has occurred, displaying that a user has performed the physical activity corresponding to the at least one physical action template on a display of the one or more devices.

8. The system of claim 1 , wherein the plurality of sensors include at least one of: an accelerometer, a gyroscope, a location determining sensor, a light sensor, a temperature sensor, a heart rate monitor, an image capturing sensor, and a moisture sensor.

9. The system of claim 1 , wherein the plurality of sensors are arranged in at least one of: an article of apparel, an article of footwear, and a wrist-borne device.

10. One or more non-transitory computer readable media storing computer readable instructions that, when executed, cause a processor to:
receive, by an activity processing system, identification of an athletic activity to be performed;
determine, by the activity processing system and based on the received identification of the athletic activity to be performed, a sensor subscription;
transmit, to a sensor system having a plurality of sensors, the determined sensor subscription;
receive, by the sensor system and from the plurality of sensors, raw athletic activity data associated with a potential corresponding event during performance of the athletic activity;
process the raw athletic activity data;
transmit, from the sensor system to the activity processing system, the processed athletic activity data, transmitting the processed athletic activity data to the activity processing system including controlling, by the sensor system, at least one of a type of data and an amount of data transmitted based on the determined sensor subscription;
compare, by the activity processing system, the received, processed athletic activity data to a plurality of physical action templates to determine whether a physical activity of at least one physical action template of the plurality of physical action templates has occurred, the comparing including:
determining a plurality of required states associated with the physical activity of the at least one physical action template and forming a portion of the at least one physical action template;
for each of the plurality of required states, determine a state-specific set of one or more constraints defining match characteristics of a corresponding required event;

determine, based on the comparison, a match score for the one or more constraints;
determine, based on the match score, whether the physical activity has occurred; and
transmit a result of the determining whether the physical activity has occurred to one or more devices.

11. The one or more non-transitory computer readable media of claim 10 , wherein a state-specific set of one or more constraints for a first state of the plurality of required states includes a relative constraint defining a required relationship between the first state and a second state of the plurality of required states,
wherein the state-specific set of one or more constraints includes at least one of: an event occurrence time stamp constraint, an event generation time stamp constraint, an event value constraint and an event duration constraint, and
wherein the state-specific set of one or more constraints includes different scoring distributions for each of the at least one of: the event occurrence time stamp constraint, the event value constraint, the event generation time stamp constraint and the event duration constraint.

12. The one or more non-transitory computer readable media of claim 10 , wherein the physical activity action template further stores one or more exclusion state constraints configured to exclude the potential corresponding event from being identified as potential matches with one or more of the plurality of required states when the potential corresponding event is determined to match the one or more exclusion state constraints.

13. The one or more non-transitory computer readable media of claim 10 , wherein the one or more

Assignees

• Nike Inc

Inventors

• Balakrishnan Santoshkumar
• Rice Jordan M
• Walker Steven H
• Carroll Adam S
• Dow-Hygelund Corey C
• Goodwin Aaron K
• Mullin James M
• Rattenbury Tye L
• Rooke-Ley Joshua M
• Schmitt John M

Classifications

• A61B2562/0252

  Load cells · CPC title

• A61B5/7264

  Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems · CPC title

• G09B19/0038

  Sports · CPC title

• A61B5/7257

  using Fourier transforms · CPC title

• A61B5/7246

  using correlation, e.g. template matching or determination of similarity · CPC title