Action detection and activity classification

Patent metadata

Field	Value
Publication number	US-10384113-B2
Application number	US-201615254548-A
Country	US
Kind code	B2
Filing date	Sep 1, 2016
Priority date	Jan 19, 2012
Publication date	Aug 20, 2019
Grant date	Aug 20, 2019

Abstract

Activities, actions and events during user performance of physical activity may be detected using various algorithms and templates. Templates may include an arrangement of one or more states that may identify particular event types and timing between events. Templates may be specific to a particular type of activity (e.g., types of sports, drills, events, etc.), user, terrain, time of day and the like.

First claim

What is claimed is:

1. A computer-implemented method comprising:
receiving, from one or more sensors and by a sensor system, raw sensor data;
identifying, by the sensor system and from the raw sensor data, a plurality of events detected from one or more sensor signal streams during performance of athletic activity by a user, one or more events including at least physical environment characteristics and user physiology characteristics;
responsive to identifying the plurality of events, transmitting from the sensor system to an activity processing system, the sensor data;
analyzing, by the activity processing system, the physical environment characteristics and user physiology characteristics in the received sensor data to evaluate the plurality of events against one or more constraints of a first action template to determine whether the user performed a first type of action corresponding to the first action template, wherein evaluating the plurality of events against the one or more constraints includes:
selecting a first state from the first action template to evaluate; and
determining whether a first candidate event of the plurality of events meets one or more non-relative constraints of the first state;
determining whether the first candidate event meets one or more relative constraints of the first state, wherein the one or more relative constraints defines a required relationship between the first candidate event and a second candidate event,
wherein the one or more relative constraints further define at least one of: a required relative timing between a current template matching analysis and one or more past template matching analyses of one or more other events, and a required relationship with statistics regarding prior occurrences of events of a type corresponding to the first state;

determining, by the activity processing system, whether the plurality of events matches the first action template based on the evaluation of the plurality of events against the one or more constraints of the first action template; and
in response to determining that the plurality of events matches the one or more constraints of the first action template, registering, by the activity processing system, user performance of the first type of action corresponding to the first action template.

2. The computer-implemented method of claim 1 , further comprising:
determining whether one or more events of the plurality of events matches an exclusion state constraint defined in the first action template; and
in response to determining that the one or more events matches the exclusion state constraint, determining that the one or more events is not a match with the first action template or that the plurality of events are not a match with the first action template.

3. The computer-implemented method of claim 1 , wherein determining whether the first candidate event meets the one or more relative constraints of the first state includes:
determining whether the second candidate event of the plurality of events meets the one or more relative constraints of the first event; and
in response to determining that the second candidate event meets the one or more relative constraints of the first event, determining whether the second candidate event meets one or more constraints of the second event specified in the first action template.

4. The computer-implemented method of claim 1 , wherein upon determining that the first candidate event meets the non-relative and relative constraints of the first state, analyzing each of one or more remaining template states in a sequential order such that an analysis for each of the one or more remaining template state includes:
determining whether a candidate event from the plurality of events meets the one or more non-relative constraints of the template state being analyzed;
determining whether the candidate event from the plurality of events that satisfied the non-relative constraints of the template state being analyzed also satisfies one or more relative constraints of the template state being analyzed, wherein the one or more relative constraints depend on one or more template states already analyzed; and
in response to determining that the candidate event meets the one or more non-relative and relative constraints of the template state being analyzed, proceeding to analyze another remaining template state.

5. The computer-implemented method of claim 1 , further comprising:
register user performance of a second type of action based on the plurality of events matching a second action template, wherein each of the first and second action templates corresponds to a different type of action.

6. The computer-implemented method of claim 1 , further comprising:
non-dimensionalizing one or more characteristics of the plurality of events.

7. The computer-implemented method of claim 1 , further comprising:
determining a level of match between the plurality of events and the first action template; and
generating one or more instructions or recommendations for improving user performance of the first type of action based on one or more of: the determined level of match, a location of the user performance of the first type of action and a time of the user performance of the first type of action.

8. The computer-implemented method of claim 1 , further comprising:
determining that the plurality of events matches the first action template and a second action template;
selecting one of the first and second templates; and
defining the first type of action based on the selected one of the first and second templates.

9. The computer-implemented method of claim 8 , wherein selecting the one of the first and second action templates includes:
determining a first level of match between the plurality of events and the first action template;
determining a second level of match between the plurality of events and the second action template; and
selecting the one of the first and second templates having a greater level of match.

10. The computer-implemented method of claim 1 , wherein the first action template includes a match tolerance.

11. A system comprising:
a sensor system including one or more sensors, the sensor system further including:
at least a first processor; and
at least a first memory storing computer readable instructions that, when executed, cause the at least a first processor to:
receive, from one or more sensors, raw sensor data;
identify, from the raw sensor data, a plurality of events detected from one or more sensor signal streams during performance of an athletic activity by a user, the plurality of events including at least physical environment characteristics and user physiology characteristics;
responsive to identifying a plurality of events, transmit to an activity processing system, the sensor data;

the activity processing system, including:
at least a second processor; and
at least a second memory storing computer readable instructions that, when executed, cause the at least a second processor to:
determine a first type of action to be performed by the user;
determine, based on the first type of action, a sensor subscription for the one or more sensors;
receive, based on the sensor subscription, the transmitted sensor data;
obtain, based on the first type of action, a plurality of action templates;
analyze the physical environment characteristics and user physiology characteristics in the received sensor data to evaluate the plurality of events against one or more constraints of a first action template to determine whether the user performed the first type of action corresponding to the first action template;
determine whether

Assignees

• Nike Inc

Inventors

• Balakrishnan Santoshkumar
• Rice Jordan M
• Walker Steven H
• Carroll Adam S
• Dow-Hygelund Corey C
• Goodwin Aaron K
• Mullin James M
• Rattenbury Tye L
• Rooke-Ley Joshua M
• Schmitt John M

Classifications

• G16H20/40

  relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture · CPC title

• A61B2562/0252

  Load cells · CPC title

• G09B5/125

  the stations being mobile · CPC title

• A61B5/1118

  Determining activity level · CPC title

• G09B19/003

  Repetitive work cycles; Sequence of movements · CPC title