Encoding input for machine learning

Patent metadata

Field	Value
Publication number	US-12382147-B2
Application number	US-202318222605-A
Country	US
Kind code	B2
Filing date	Jul 17, 2023
Priority date	Oct 31, 2018
Publication date	Aug 5, 2025
Grant date	Aug 5, 2025

Abstract

A method and system for providing synchronized input feedback, comprising receiving an input event, encoding the input event in an output stream wherein the encoding of the input event is synchronized to a specific event and reproducing the output stream through an output device whereby the encoded input event in the reproduced output stream is imperceptible to the user.

First claim

What is claimed is:

1. A method for providing synchronized input feedback:
a) encoding one or more input events in an output stream, wherein:
the output stream includes a video stream;
one or more of the one or more input events are encoded as a watermark in an alpha channel over the video stream;
the encoding of the one or more input events is synchronized to one or more subsequent specific events occurring within a software environment;
the one or more input events include a control input that precedes one or more of the one or more subsequent specific events;
the encoding of the one or more input events is synchronized to the one or more subsequent specific events occurring on a remote device over a network; and
the one or more input events are encoded synchronously with the video stream or an audio stream having the one or more subsequent specific events occurring therein; and

b) storing the output stream or reproducing the output stream through an output device, wherein:
the one or more encoded input events in the output stream is imperceptible to a user when reproduced through the output device; and
the one or more encoded input events in the reproduced output stream are decodable for use by one or more neural networks.

2. The method of claim 1 , wherein the output stream includes the audio stream.

3. The method of claim 2 wherein the one or more input events is encoded into the audio stream as an infrasonic tone.

4. The method of claim 2 wherein one or more of the one or more input events are encoded into the audio stream as an ultrasonic tone.

5. The method of claim 1 wherein one or more of the one or more input events are encoded as metadata in the video stream.

6. The method of claim 5 wherein the metadata is supplemental enhancement information for each video frame.

7. The method of claim 1 wherein the output stream comprises a series of output events and the encoding of the one or more input events is synchronized with the series of output events.

8. The method of claim 1 wherein the encoding of the one or more input events are synchronized to one or more events occurring within a videogame.

9. The method of claim 1 , wherein the control input is a key press.

10. The method of claim 1 , wherein the control input is a joystick control input.

11. The method of claim 1 , wherein the control input is a steering wheel control input.

12. The method of claim 1 , wherein the control input is an analog joy pad input.

13. The method of claim 1 , wherein the control input is a potentiometer input.

14. The method of claim 1 , further comprising, filtering the output stream of the one or more encoded input events to separate the one or more encoded input events from the output stream.

15. The method of claim 14 , further comprising providing the output stream and the one or more encoded input events as inputs to the one or more neural networks trained to associate the one or more encoded input events with the one or more subsequent specific events.

16. The method of claim 1 wherein the one or more input events are configured to cause the one or more subsequent specific events.

17. A non-transitory computer readable medium computer-readable instructions embedded thereon that when executed by a computer cause the computer to enact a method comprising:
a) encoding one or more input events in an output stream, wherein:
the output stream includes a video stream;
one or more of the one or more input events are encoded as a watermark in an alpha channel over the video stream;
the encoding of the one or more input events is synchronized to one or more subsequent specific events occurring within a software environment on a remote device over a network;
the one or more input events include a control input that precedes one or more of the one or more subsequent specific events;
the encoding of the one or more input events is synchronized to the one or more subsequent specific events; and
the one or more input events are encoded synchronously with the video stream or an audio stream having the one or more subsequent specific events occurring therein; and

b) storing the output stream or reproducing the output stream through an output device whereby the one or more encoded input events in the output stream are undetectable to a user when reproduced through the output device, wherein the one or more encoded input events in the reproduced output stream are decodable for use by one or more neural networks.

18. A system comprising:
a processor;
memory;
non-transitory instruction in the memory that when executed cause the processor to enact a method comprising:
a) encoding one or more input events in an output stream, wherein:
the output stream includes a video stream;
one or more of the one or more input events are encoded as a watermark in an alpha channel over the video stream;
the encoding of the one or more input events are synchronized to one or more subsequent specific events occurring within a software environment on a remote device over a network;
the one or more input events include a control input that precedes one or more of the one or more subsequent specific events;
the encoding of the one or more input events is synchronized to the one or more subsequent specific events; and
the one or more input events are encoded synchronously with the video stream or an audio stream having the one or more subsequent specific events occurring therein; and

b) storing the output stream or reproducing the output stream through an output device whereby the one or more encoded input events in the output stream are undetectable to a user when reproduced through the output device, wherein the one or more encoded input events in the reproduced output stream are decodable for use by one or more neural networks.

19. A method for improved machine learning training comprising;
a) receiving an output stream comprising one or more encoded input events that include a control input that precedes one or more subsequent specific events occurring within a software environment on a remote device over a network, wherein:
the output stream includes a video stream;
one or more of the one or more input events are encoded as a watermark in an alpha channel over the video stream;
the one or more encoded input events are undetectable to a user;
the one or more encoded input events in the output stream are decodable for use by one or more neural networks;
the encoding of the one or more input events is synchronized to the one or more subsequent specific events; and
the one or more input events are encoded synchronously with the video stream or an audio stream having the one or more subsequent specific events occurring therein;

b) filtering the audio stream or video stream to recover the one or more encoded input events, wherein the one or more encoded input events are synchronized to one or more of the one or more subsequent specific events occurring in the software environment; and
c) training a neural network to associate the one or more encoded input events with the one or more subsequent specific events.

20. A non-transitory computer-readable medium having an output stream embedded thereon, the output stream comprising:
one or more input events encoded into the output stream, wherein:
the output stream includes a video stream;
one or more of the one or more input events are encoded as a watermark in an alpha channel over the video stream;
the one or more input events include a control input tha

Assignees

• Sony Interactive Entertainment Inc

Inventors

• Bennett Matthew

Classifications

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06N3/0442

  characterised by memory or gating, e.g. long short-term memory [LSTM] or gated recurrent units [GRU] · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06V10/776

  Validation; Performance evaluation · CPC title

• G06F18/214

  Generating training patterns; Bootstrap methods, e.g. bagging or boosting · CPC title