Dynamic targeting of preferred objects in video stream of smartphone camera

Patent metadata

Field	Value
Publication number	US-2022269396-A1
Application number	US-202217741303-A
Country	US
Kind code	A1
Filing date	May 10, 2022
Priority date	Mar 30, 2015
Publication date	Aug 25, 2022
Grant date	—

Abstract

Selecting objects in a video stream of a smart phone includes detecting quiescence of frame content in the video stream, detecting objects in a scene corresponding to the frame content, presenting at least one of the objects to a user of the smart phone, and selecting at least one of the objects in a group of objects in response to input by the user. Detecting quiescence of frame content in the video stream may include using motion sensors in the smart phone to determine an amount of movement of the smart phone. Detecting quiescence of frame content in the video stream may include detecting changes in view angles and distances of the smart phone with respect to the scene. Detecting objects in a scene may use heuristics, custom user preferences, and/or specifics of scene layout. At least one of the objects may be a person or a document.

First claim

What is claimed is:

1 . A method of capturing a subset of objects within a video stream captured by an electronic device, the method comprising:
receiving a video stream captured by an electronic device; detecting within a frame of the video stream one or more objects for capture; determining a plurality of scenarios based on the one or more objects, wherein each scenario of the plurality of scenarios is a distinct subset of the one or more objects; displaying, via the electronic device, the frame of the video stream in conjunction with a first scenario of the plurality of scenarios; responsive to a first user input rejecting the first scenario of the plurality of scenarios, displaying, via the electronic device, the frame of the video stream in conjunction with a second scenario of the plurality of scenarios; and responsive to a second user input selecting the second scenario of the plurality of scenarios, extracting a respective subset of the one or more objects corresponding to the second scenario from the frame of the video stream.

2 . The method of claim 1 , further comprising:
after determining the plurality of scenarios, pre-selecting the first scenario of the plurality of scenarios to be displayed, via the electronic device, based on a third user input.

3 . The method of claim 2 , wherein the third user input includes one or more of a change in a view angle and a change in a distance of the electronic device with respect to the one or more objects.

4 . The method of claim 1 , wherein:
displaying the frame of the video stream in conjunction with the first scenario of the plurality of scenarios includes displaying the first scenario with an overlay highlighting a respective subset of the one or more objects corresponding to the first scenario; and displaying the frame of the video stream in conjunction with the second scenario of the plurality of scenarios includes displaying the second scenario with an overlay highlighting a respective subset of the one or more objects corresponding to the second scenario.

5 . The method of claim 1 , wherein the one or more objects include one or more of a person and a document.

6 . The method of claim 1 , further comprising:
after extracting the respective subset of the one or more objects corresponding to the second scenario from the frame of the video stream, displaying, via the electronic device, the respective subset of the one or more objects corresponding to the second scenario.

7 . The method of claim 1 , further comprising:
after extracting the respective subset of the one or more objects corresponding to the second scenario from the frame of the video stream, displaying, via the electronic device, one or more affordances including:
a first affordance that allows the user to store the respective subset of the one or more objects corresponding to the second scenario, and
a second affordance that allows the user to share the respective subset of the one or more objects corresponding to the second scenario.

8 . The method of claim 1 , wherein detecting within the frame of the video stream the one or more objects for capture includes one or more of determining one or more objects in focus, determining one or more objects with a predetermined distance relative to the electronic device, and determining one or more unobstructed objects.

9 . The method of claim 1 , wherein the first user input includes one or more of selection of an rejection affordance displayed on the electronic device and a rejection gesture including shaking the electronic device left-and-right.

10 . The method of claim 1 , wherein the second user input includes one or more of selection of an approval affordance displayed on the electronic device, allowing a predetermined amount of time to elapse without moving the electronic device, eye-tracking, spatial gestures, and facial expressions.

11 . An electronic device, comprising:
one or more processors; and memory storing one or more programs for execution by the one or more processors, the one or more programs including instructions for:
receiving a video stream captured by an electronic device;
detecting within a frame of the video stream one or more objects for capture;
determining a plurality of scenarios based on the one or more objects, wherein each scenario of the plurality of scenarios is a distinct subset of the one or more objects;
displaying, via the electronic device, the frame of the video stream in conjunction with a first scenario of the plurality of scenarios;
responsive to a first user input rejecting the first scenario of the plurality of scenarios, displaying, via the electronic device, the frame of the video stream in conjunction with a second scenario of the plurality of scenarios; and
responsive to a second user input selecting the second scenario of the plurality of scenarios, extracting a respective subset of the one or more objects corresponding to the second scenario from the frame of the video stream.

12 . The electronic device of claim 11 , wherein the one or more programs further include instructions for:
after determining the plurality of scenarios, pre-selecting the first scenario of the plurality of scenarios to be displayed, via the electronic device, based on a third user input.

13 . The electronic device of claim 12 , wherein the third user input includes one or more of a change in a view angle and a change in a distance of the electronic device with respect to the one or more objects.

14 . The electronic device of claim 11 , wherein:
displaying the frame of the video stream in conjunction with the first scenario of the plurality of scenarios includes displaying the first scenario with an overlay highlighting a respective subset of the one or more objects corresponding to the first scenario; and displaying the frame of the video stream in conjunction with the second scenario of the plurality of scenarios includes displaying the second scenario with an overlay highlighting a respective subset of the one or more objects corresponding to the second scenario.

15 . The electronic device of claim 11 , wherein the one or more objects include one or more of a person and a document.

16 . A non-transitory computer-readable storage medium storing one or more programs for execution by one or more processors of an electronic device, the one or more programs comprising instructions for:
receiving a video stream captured by the electronic device; detecting within a frame of the video stream one or more objects for capture; determining a plurality of scenarios based on the one or more objects, wherein each scenario of the plurality of scenarios is a distinct subset of the one or more objects; displaying, via the electronic device, the frame of the video stream in conjunction with a first scenario of the plurality of scenarios; responsive to a first user input rejecting the first scenario of the plurality of scenarios, displaying, via the electronic device, the frame of the video stream in conjunction with a second scenario of the plurality of scenarios; and responsive to a second user input selecting the second scenario of the plurality of scenarios, extracting a respective subset of the one or more objects corresponding to the second scenario from the frame of the video stream.

17 . The non-transitory computer-readable storage medium of claim 16 , wherein the one or more programs further include instructions for:
after determining the plurality of scenarios, pre-selecting the first scenario of the plurality of scenarios to b

Assignees

• Evernote Corp

Inventors

• Pashintsev Alexander
• Gorbatov Boris
• Livshitz Eugene
• Glazkov Vitaly

Classifications

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title

• G06F3/002

  Specific input/output arrangements not covered by G06F3/01 - G06F3/16 (other optical apparatus G02B27/00) · CPC title

• G06F1/1694

  the I/O peripheral being a single or a set of motion sensors for pointer control or gesture input obtained by sensing movements of the portable computer · CPC title

• G06F1/1686

  the I/O peripheral being an integrated camera · CPC title