Visual experience modulation based on stroboscopic effect

Patent metadata

Field	Value
Publication number	US-11910120-B2
Application number	US-202117524088-A
Country	US
Kind code	B2
Filing date	Nov 11, 2021
Priority date	Nov 11, 2021
Publication date	Feb 20, 2024
Grant date	Feb 20, 2024

Abstract

An approach for modifying in real-time by removing or reinforcing stroboscopic effect from images associated with a viewing experience is disclosed. The approach includes identifying video clips, detecting environmental parameters and calculating display setting. The approach also analyzes display setting using recommendation from GAN, output displaying setting on an AR display and receiving feedback from user.

First claim

What is claimed is:

1. A computer-implemented method for modifying in real-time stroboscopic effect, by leveraging machine learning, from one or more video clips, the computer-method comprising:
identifying the one or more video clips from a user;
receiving IoT (Internet of Things) data from one or more IoT devices;
determining an optical data based on the IoT data and the user;
determining a stroboscopic effect setting based on the optical data and factors and wherein the factors further comprise of, environmental condition, user medical profile, lighting condition, frequency of light, rotating speed of the object and spatio-temporal object visibility score;
executing the stroboscopic effect setting on the one or more video clips;
outputting the video clips based on the updated the stroboscopic effecting setting; and
receiving one or more feedbacks from the user.

2. The computer-implemented method of claim 1 , wherein determining optical data further comprising:
collecting the optical data, wherein optical data includes environmental data and user data; and
encoding optical data into latent space.

3. The computer-implemented method of claim 1 , wherein stroboscopic effect setting further comprises, enabling/disabling stroboscopic effect, enhancing stereoscopic effect and negating stereoscopic effect.

4. The computer-implemented method of claim 3 wherein enabling/disabling stroboscopic effect, enhancing stereoscopic effect or negating stereoscopic effect further comprises, (i) modulating visibility of moving objects and background simultaneously, (ii) augmenting the background and adding transparency, (iii) making the moving object opaque and distinguishing it with respect to the background, (iv) controlling of the stroboscopic effect using light source, (v) making the moving object opaque and distinguishing it with respect to the background, (vi) augmenting the background and adding transparency, (vii) changing the movement of the objects in the video.

5. The computer-implemented method of claim 1 , wherein executing stroboscopic effecting setting on the one or more video clips further comprises:
generating an augmented video by learning discriminator and generators using various loss functions.

6. The computer-implemented method of claim 1 , wherein outputting the video clips based on the updated the stroboscopic effecting setting further comprises:
superimposing a GAN (generative adversarial network) generated video as an additional layer with an (AR) augmented reality system.

7. The computer-implemented method of claim 1 , wherein receiving feedback from user further comprises:
updating model parameters; and
fine-tuning the model parameters based on the reward score provided by the user using reinforcement learning.

8. A computer program product for modifying in real-time stroboscopic effect from one or more video clips, the computer program product comprising:
one or more computer readable storage media and program instructions stored on the one or more computer readable storage media, the program instructions, executable by one or more computer processors, comprising:
program instructions to the one or more video clips from a user;
program instructions to receive IoT (Internet of Things) data from one or more IoT devices;
program instructions to determine an optical data based on the IoT data and the user;
program instructions to determine a stroboscopic effect setting based on the optical data and factors and wherein the factors further comprises of, environmental condition, user medical profile, lighting condition, frequency of light, rotating speed of the object and spatio-temporal object visibility score;
program instructions to execute stroboscopic effect setting on the one or more video clips;
program instructions to output the video clips based on the updated the stroboscopic effecting setting; and
program instructions to receive one or more feedbacks from the user.

9. The computer program product of claim 8 , wherein program instructions to determine optical data further comprising:
program instructions to collect the optical data, wherein optical data includes environmental data and user data; and
program instructions to encode optical data into latent space.

10. The computer program product of claim 8 , wherein stroboscopic effect setting further comprises, enabling/disabling stroboscopic effect, enhancing stereoscopic effect and negating stereoscopic effect.

11. The computer program product of claim 10 , wherein enabling/disabling stroboscopic effect, enhancing stereoscopic effect or negating stereoscopic effect further comprises, (i) modulating visibility of moving objects and background simultaneously, (ii) augmenting the background and adding transparency, (iii) making the moving object opaque and distinguishing it with respect to the background, (iv) controlling of the stroboscopic effect using light source, (v) making the moving object opaque and distinguishing it with respect to the background, (vi) augmenting the background and adding transparency, (vii) changing the movement of the objects in the video.

12. The computer program product of claim 8 , wherein program instructions to execute stroboscopic effecting setting on the one or more video clips further comprises:
program instructions to generate an augmented video by learning discriminator and generators using various loss functions.

13. The computer program product of claim 8 , wherein program instructions to output the video clips based on the updated the stroboscopic effecting setting further comprises:
program instructions to superimpose a GAN (generative adversarial network) generated video as an additional layer with an (AR) augmented reality system.

14. The computer program product of claim 8 , wherein program instructions to receiving feedback from user further comprises:
program instructions to update model parameters; and
program instructions to fine-tune the model parameters based on the reward score provided by the user using reinforcement learning.

15. A computer system for modifying in real-time stroboscopic effect from one or more video clips, the computer system comprising:
one or more computer processors;
one or more computer readable storage media; and
program instructions stored on the one or more computer readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising:
program instructions to the one or more video clips from a user;
program instructions to receive IoT (Internet of Things) data from one or more IoT devices;
program instructions to determine an optical data based on the IoT data and the user;
program instructions to determine a stroboscopic effect setting based on the optical data and factors and wherein the factors further comprises of, environmental condition, user medical profile, lighting condition, frequency of light, rotating speed of the object and spatio-temporal object visibility score;
program instructions to execute stroboscopic effect setting on the one or more video clips;
program instructions to output the video clips based on the updated the stroboscopic effecting setting; and
program instructions to receive one or more feedbacks from the user.

16. The computer system of claim 15 , wherein program instructions to determine optical data further comprising:
program instructions to collect the optical data, wherein optical data includes environmental data and user data; and
program instructions to encode optical data into latent space.

Assignees

• IBM

Inventors

• Pandey Diwesh
• Bose Tathagato
• MARVANIYA Smitkumar Narotambhai
• Joseph Arun
• Rakshit Sarbajit K

Classifications

• H04N5/2625 Primary

  for obtaining an image which is composed of images from a temporal image sequence, e.g. for a stroboscopic effect (sequence generated by event triggered capturing H04N7/188) · CPC title

• G06N20/20

  Ensemble learning · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• H04N5/76

  Television signal recording · CPC title

• H04N9/79

  Processing of colour television signals in connection with recording · CPC title