Identity-based inclusion/exclusion in a computer-generated reality experience

Patent metadata

Field	Value
Publication number	US-11783552-B2
Application number	US-202117557805-A
Country	US
Kind code	B2
Filing date	Dec 21, 2021
Priority date	Sep 27, 2018
Publication date	Oct 10, 2023
Grant date	Oct 10, 2023

Abstract

In one implementation, a method of including a person in a CGR experience or excluding the person from the CGR experience is performed by a device including one or more processors, non-transitory memory, and a scene camera. The method includes, while presenting a CGR experience, capturing an image of scene; detecting, in the image of the scene, a person; and determining an identity of the person. The method includes determining, based on the identity of the person, whether to include the person in the CGR experience or exclude the person from the CGR experience. The method includes presenting the CGR experience based on the determination.

First claim

What is claimed is:

1. A method comprising:
at an electronic device including one or more processors, non-transitory memory, a scene camera, one or more microphones, one or more speakers, and a display:
while presenting a computer-generated reality (CGR) experience via the display, capturing, via the scene camera, an image of a physical environment;
detecting a person within the image of the physical environment; and
in response to detecting the person within the image of the physical environment:
capturing, via the one or more microphones, audio content from the person;
altering the captured audio content from the person and providing the altered audio content, via the one or more speakers, according to a determination that an identity of the person is included among the list of identities; and
providing the captured audio content from the person, via the one or more speakers, according to a determination that the identity of the person is not included among the list of identities.

2. The method of claim 1 , further comprising:
in response to detecting the person within the image of the physical environment:
presenting, via the display, an avatar of the person within the CGR experience according to the determination that the identity of the person is included among the list of identities, wherein the avatar for the person that is different from an appearance of the person detected within the image of the physical environment; and
presenting, via the display, an unaltered version of the person within the CGR experience according to the determination that the identity of the person is not included among the list of identities, wherein the unaltered version of the person corresponds to the appearance of the person detected within the image of the physical environment.

3. The method of claim 1 , further comprising:
in response to detecting the person within the image of the physical environment:
presenting, via the display, a representation of the person within the CGR experience according to the determination that the identity of the person is included among the list of identities; and
forgoing, via the display, presentation of the representation of the person within the CGR experience according to the determination that the identity of the person is not included among the list of identities.

4. The method of claim 3 , further comprising:
presenting, via the display, an inclusion indicia adjacent to the representation of the person according to the determination that the identity of the person is included among the list of identities.

5. The method of claim 1 , further comprising:
presenting, via the display, data associated with the person according to the determination that the identity of the person is included among the list of identities.

6. The method of claim 1 , wherein detecting the person includes applying an object detecting neural network to the image of the physical environment to generate a pixel set corresponding to a representation of the person and wherein determining the identity of the person is based on the pixel set.

7. The method of claim 1 , further comprising:
determining the identity of the person within the image of the physical environment.

8. The method of claim 7 , wherein determining the identity of the person is based on detected facial features of the person.

9. The method of claim 7 , wherein determining the identity of the person includes applying a classifier to at least a portion of the image of the physical environment.

10. The method of claim 9 , further comprising:
determining whether the classifier generates the identity for the person within the image of the physical environment; and
in accordance with a determination that the classifier cannot generate the identity for the person within the image of the physical environment:
presenting, via the display, a notification prompting a user of the electronic device to provide the identity for the person within the image of the physical environment; and
updating the classifier based on a response to the notification.

11. A device comprising:
a scene camera;
a display;
one or more microphones;
one or more speakers;
one or more processors;
a non-transitory memory; and
one or more programs stored in the non-transitory memory, which, when executed by the one or more processors, cause the device to:
while presenting a computer-generated reality (CGR) experience via the display, capture, via the scene camera, an image of a physical environment;
detecting a person within the image of the physical environment; and
in response to detecting the person within the image of the physical environment:
capture, via the one or more microphones, audio content from the person;
alter the captured audio content from the person and provide the altered audio content, via the one or more speakers, according to a determination that an identity of the person is included among the list of identities; and
provide the captured audio content from the person, via the one or more speakers, according to a determination that the identity of the person is not included among the list of identities.

12. The device of claim 11 , wherein the one or more programs further cause the device to:
in response to detecting the person within the image of the physical environment:
present, via the display, an avatar of the person within the CGR experience according to the determination that the identity of the person is included among the list of identities, wherein the avatar for the person that is different from an appearance of the person detected within the image of the physical environment; and
present, via the display, an unaltered version of the person within the CGR experience according to the determination that the identity of the person is not included among the list of identities, wherein the unaltered version of the person corresponds to the appearance of the person detected within the image of the physical environment.

13. The device of claim 11 , wherein the one or more programs further cause the device to:
in response to detecting the person within the image of the physical environment:
present, via the display, a representation of the person within the CGR experience according to the determination that the identity of the person is included among the list of identities; and
forgo, via the display, presentation of the representation of the person within the CGR experience according to the determination that the identity of the person is not included among the list of identities.

14. The device of claim 13 , wherein the one or more programs further cause the device to:
present, via the display, an inclusion indicia adjacent to the representation of the person according to the determination that the identity of the person is included among the list of identities.

15. The device of claim 11 , wherein the one or more programs further cause the device to:
present, via the display, data associated with the person according to the determination that the identity of the person is included among the list of identities.

16. The device of claim 11 , wherein the one or more programs further cause the device to:
determine the identity of the person within the image of the physical environment, wherein determining the identity of the person is based on at least one of detecting facial features of the person or applying a classifier to at least a portion of the image of the physical environment.

17. A non-transitory memory storing one or more programs, which, when exec

Assignees

• Apple Inc

Inventors

• Ulbricht Daniel
• K C Amit Kumar
• Blechschmidt Angela
• Lee Chen-Yu
• Verma Eshan
• Baig Mohammad Haris
• Batra Tanmay

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• A63F13/212

  using sensors worn by the player, e.g. for measuring heart beat or leg activity · CPC title

• A63F13/825

  Fostering virtual characters · CPC title

• G02B27/017

  Head mounted · CPC title

• G06F3/011

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title