Identity-based inclusion/exclusion in a computer-generated reality experience

Patent metadata

Field	Value
Publication number	US-11295529-B2
Application number	US-202117149949-A
Country	US
Kind code	B2
Filing date	Jan 15, 2021
Priority date	Sep 27, 2018
Publication date	Apr 5, 2022
Grant date	Apr 5, 2022

Abstract

In one implementation, a method of including a person in a CGR experience or excluding the person from the CGR experience is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes, while presenting a CGR experience, capturing an image of scene; detecting, in the image of the scene, a person; and determining an identity of the person. The method includes determining, based on the identity of the person, whether to include the person in the CGR experience or exclude the person from the CGR experience. The method includes presenting the CGR experience based on the determination.

First claim

What is claimed is:

1. A method comprising:
at an electronic device including one or more processors, non-transitory memory, a scene camera, and a display:
while presenting a computer-generated reality (CGR) experience via the display:
capturing, via the scene camera, an image of a physical scene; and
detecting a person in the image of the physical scene;

presenting, via the display, an avatar of the person within the CGR experience according to a determination that an identity of the person is included among a list of identities, wherein the avatar for the person that is different from an appearance of the person detected within the image of the physical scene; and
presenting, via the display, an unaltered version of the person within the CGR experience according to a determination that the identity of the person is not included among the list of identities, wherein the unaltered version of the person corresponds to the appearance of the person detected within the image of the physical scene.

2. The method of claim 1 , further comprising:
presenting, via the display, an inclusion indicia adjacent to the avatar according to the determination that the identity of the person is included among the list of identities.

3. The method of claim 1 , further comprising:
presenting, via the display, data associated with the person or the avatar of the person according to the determination that an identity of the person is included among the list of identities.

4. The method of claim 1 , further comprising:
while presenting the CGR experience, capturing, via a microphone, audio content from the person;
altering the audio content from the person and providing the altered audio content according to the determination that the identity of the person is included among the list of identities; and
providing the captured audio content from the person according to the determination that the identity of the person is not included among the list of identities.

5. The method of claim 1 , wherein detecting the person includes applying an object detecting neural network to the image of the physical scene to generate a pixel set corresponding to a representation of the person and wherein determining the identity of the person is based on the pixel set.

6. The method of claim 1 , wherein determining the identity of the person is based on detected facial features of the person.

7. The method of claim 1 , wherein determining the identity of the person includes applying a classifier to at least a portion of the image of the physical scene.

8. The method of claim 7 , further comprising:
determining whether the classifier generates the identity for the person within the image of the physical scene; and
in accordance with a determination that the classifier cannot generate the identity for the person within the image of the physical scene:
presenting, via the display, a notification prompting a user of the electronic device to provide the identity for the person within the image of the physical scene; and
updating the classifier based on a response to the notification.

9. A device comprising:
a scene camera;
a display;
one or more processors;
a non-transitory memory; and
one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to:
while presenting a computer-generated reality (CGR) experience via the display:
capture, via the scene camera, an image of a physical scene; and
detect a person in the image of the physical scene;

present, via the display, an avatar of the person within the CGR experience according to a determination that an identity of the person is included among a list of identities, wherein the avatar for the person that is different from an appearance of the person detected within the image of the physical scene; and
present, via the display, an unaltered version of the person within the CGR experience according to a determination that the identity of the person is not included among the list of identities, wherein the unaltered version of the person corresponds to the appearance of the person detected within the image of the physical scene.

10. The device of claim 9 , wherein the one or more programs further cause the device to:
present, via the display, an inclusion indicia adjacent to the avatar according to the determination that the identity of the person is included among the list of identities.

11. The device of claim 9 , wherein the one or more programs further cause the device to:
present, via the display, data associated with the person or the avatar of the person according to the determination that the identity of the person is included among the list of identities.

12. The device of claim 9 , wherein the one or more programs further cause the device to:
while presenting the CGR experience, capture, via a microphone, audio content from the person;
alter the audio content from the person and provide the altered audio content according to the determination that the identity of the person is included among the list of identities; and
provide the captured audio content from the person according to the determination that the identity of the person is not included among the list of identities.

13. The device of claim 9 , wherein detecting the person includes applying an object detecting neural network to the image of the physical scene to generate a pixel set corresponding to a representation of the person and wherein determining the identity of the person is based on the pixel set.

14. The device of claim 9 , wherein determining the identity of the person is based on detected facial features of the person.

15. The device of claim 9 , wherein determining the identity of the person includes applying a classifier to at least a portion of the image of the physical scene.

16. A non-transitory memory storing one or more programs, which, when executed by one or more processors of a device with a scene camera and a display, cause the device to:
while presenting a computer-generated reality (CGR) experience via the display:
capture, via the scene camera, an image of a physical scene; and
detect a person in the image of the physical scene;

present, via the display, an avatar of the person within the CGR experience according to a determination that an identity of the person is included among a list of identities, wherein the avatar for the person that is different from an appearance of the person detected within the image of the physical scene; and
present, via the display, an unaltered version of the person within the CGR experience according to a determination that the identity of the person is not included among the list of identities, wherein the unaltered version of the person corresponds to the appearance of the person detected within the image of the physical scene.

17. The non-transitory memory of claim 16 , wherein the one or more programs further cause the device to:
present, via the display, an inclusion indicia adjacent to the avatar according to the determination that the identity of the person is included among the list of identities.

18. The non-transitory memory of claim 16 , wherein the one or more programs further cause the device to:
present, via the display, data associated with the person or the avatar of the person according to the determination that the identity of the person is included among the list of identities.

19. The non-transitory memory of claim 16 , wherein the one or more programs further cause th

Assignees

• Apple Inc

Inventors

• Ulbricht Daniel
• K C Amit Kumar
• Blechschmidt Angela
• Lee Chen-Yu
• Verma Eshan
• Baig Mohammad Haris
• Batra Tanmay

Classifications

• G02B27/017

  Head mounted · CPC title

• A63F13/79

  involving player-related data, e.g. identities, accounts, preferences or play histories · CPC title

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• A63F13/75

  Enforcing rules, e.g. detecting foul play or generating lists of cheating players · CPC title

• G06F3/0304

  Detection arrangements using opto-electronic means (constructional details of pointing devices not related to the detection arrangement using opto-electronic means G06F3/033; optical digitisers G06F3/042) · CPC title