Scene classification

Patent metadata

Field	Value
Publication number	US-11756294-B2
Application number	US-202117397747-A
Country	US
Kind code	B2
Filing date	Aug 9, 2021
Priority date	Apr 13, 2018
Publication date	Sep 12, 2023
Grant date	Sep 12, 2023

Abstract

An exemplary process for identifying a type of a physical environment amongst a plurality of types of physical environments is provided. The process includes obtaining, using the one or more cameras, image data corresponding to a physical environment. The process further includes identifying at least one portion of an entity in the physical environment based on the image data; determining, based on the identified at least one portion of the entity, whether the entity is an entity of a first type; determining a type of the physical environment if the entity is an entity of the first type; and presenting one or more virtual objects and a representation of the entity.

First claim

What is claimed is:

1. An electronic device, comprising:
one or more processors; and
memory storing one or more programs configured to be executed by the one or more processors, the one or more programs including instructions for:
presenting one or more virtual objects and a representation of a first entity from a set of one or more entities in a physical environment, wherein a virtual object of the one or more virtual object performs one or more tasks corresponding to the physical environment;
receiving a set of one or more inputs representing a selection of the virtual object of the one or more presented virtual objects; and
in response to receiving the set of one or more inputs representing the selection of the virtual object, performing the one or more tasks corresponding to the physical environment, wherein performing the one or more tasks includes changing a state of a second entity from the set of one or more entities in the physical environment.

2. The electronic device of claim 1 , further comprising:
prior to presenting the one or more virtual objects and the representation of the first entity corresponding to the physical environment:
identifying at least one portion of the first entity from a set of one or more entities in the physical environment;
determining whether the at least one portion of the first entity corresponds to at least one portion of a previously-identified entity;
in accordance with a determination that the at least one portion of the first entity does not correspond to at least one portion of a previously-identified entity, storing data indicative of the at least one portion of the entity;
determining, based on the at least one portion of the first entity, whether the first entity is of a first type; and
in accordance with a determination that the first entity is of the first type, determining a type of the physical environment based on the first entity.

3. The electronic device of claim 2 , wherein identifying the at least one portion of the first entity in the physical environment comprises:
identifying a first portion of the first entity without identifying an entire first entity; and
foregoing identifying a second portion of the first entity.

4. The electronic device of claim 2 , wherein determining, based on the identified at least one portion of the first entity, whether the first entity is an entity of the first type comprises:
determining, based on the identified at least one portion of the first entity in the physical environment, one or more properties of the entity; and
determining, based on the one or more properties of the entity, whether the first entity is an entity of the first type.

5. The electronic device of claim 4 , wherein determining, based on the identified at least one portion of the first entity, the one or more properties of the first entity comprises:
classifying the first entity based on the identified at least one portion of the first entity; and
obtaining, based on the classification of the first entity, one or more properties of the first entity from a plurality of properties.

6. The electronic device of claim 4 , wherein the one or more properties of the first entity comprise at least one of:
mobility of the first entity;
an indication of whether the first entity is a building structure; and
consistency of an appearance of the first entity.

7. The electronic device of claim 4 , wherein the determining, based on the one or more properties of the first entity, whether the first entity is an entity of the first type comprises:
determining whether a combination of the one or more properties of the first entity exceeds a confidence threshold.

8. The electronic device of claim 2 , wherein determining the type of the physical environment based on the first entity comprises:
determining whether the first entity corresponds to at least one of a plurality of types of the physical environments; and
in accordance with a determination that the first entity corresponds to at least one of the plurality of types of the physical environments, selecting one of the plurality of types of the physical environments.

9. The electronic device of claim 8 , further comprising:
determining one or more additional types of the physical environments;
presenting the determined one or more additional types of the physical environments; and
receiving, from a user, a selection of one of the determined types of the physical environments.

10. A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors of an electronic device, the one or more programs including instructions for:
presenting one or more virtual objects and a representation of a first entity from a set of one or more entities in a physical environment, wherein a virtual object of the one or more virtual object performs one or more tasks corresponding to the physical environment;
receiving a set of one or more inputs representing a selection of the virtual object of the one or more presented virtual objects; and
in response to receiving the set of one or more inputs representing the selection of the virtual object, performing the one or more tasks corresponding to the physical environment, wherein performing the one or more tasks includes changing a state of a second entity from the set of one or more entities in the physical environment.

11. The non-transitory computer-readable storage medium of claim 10 , the one or more programs further including instructions for:
prior to presenting the one or more virtual objects and the representation of the first entity corresponding to the physical environment:
identifying at least one portion of the first entity from a set of one or more entities in the physical environment;
determining whether the at least one portion of the first entity corresponds to at least one portion of a previously-identified entity;
in accordance with a determination that the at least one portion of the first entity does not correspond to at least one portion of a previously-identified entity, storing data indicative of the at least one portion of the entity;
determining, based on the at least one portion of the first entity, whether the first entity is of a first type; and
in accordance with a determination that the first entity is of the first type, determining a type of the physical environment based on the first entity.

12. The computer-readable storage medium of claim 11 , wherein identifying the at least one portion of the first entity in the physical environment comprises:
identifying a first portion of the first entity without identifying an entire first entity; and
foregoing identifying a second portion of the first entity.

13. The computer-readable storage medium of claim 11 , wherein determining, based on the identified at least one portion of the first entity, whether the first entity is an entity of the first type comprises:
determining, based on the identified at least one portion of the first entity in the physical environment, one or more properties of the entity; and
determining, based on the one or more properties of the entity, whether the first entity is an entity of the first type.

14. The computer-readable storage medium of claim 13 , wherein determining, based on the identified at least one portion of the first entity, the one or more properties of the first entity comprises:
classifying the first entity based on the identified at least one portion of the first entity; and
obtaining, based on the classification of the first entity, one or more properties of the first entity

Assignees

• Apple Inc

Inventors

• Meier Peter
• Rockwell Michael J

Classifications

• G06F3/011 Primary

  Arrangements for interaction with the human body, e.g. for user immersion in virtual reality (blind teaching G09B21/00) · CPC title

• G06V20/10 Primary

  Terrestrial scenes (scenes under surveillance with static cameras G06V20/52; scenes perceived from the exterior of a vehicle G06V20/56; scenes perceived from the interior of a vehicle G06V20/59) · CPC title

• G06V20/35 Primary

  Categorising the entire scene, e.g. birthday party or wedding scene · CPC title

• G06F18/24

  Classification techniques · CPC title

• G06V20/70

  Labelling scene content, e.g. deriving syntactic or semantic representations · CPC title