Privacy-sensitive consumer cameras coupled to augmented reality systems

Patent metadata

Field	Value
Publication number	US-2016335802-A1
Application number	US-201615155000-A
Country	US
Kind code	A1
Filing date	May 14, 2016
Priority date	May 14, 2015
Publication date	Nov 17, 2016
Grant date	—

Abstract

An augmented reality display device comprises a housing for one or more components for the augmented reality display device, wherein the one or more components comprises a plurality of sensors to capture information pertaining to the user's surroundings, and wherein at least one sensor of the plurality of sensors is an image-based sensor, a processing module communicatively coupled to the housing to process a set of data retrieved from the plurality of sensors, wherein the processing module comprises a gating mechanism that selectively allows data to be uploaded to the cloud, and a detachable camera removably attached to a housing of the augmented reality display device, such that when the detachable camera is attached to the housing of the augmented reality display device, the gating mechanism is opened such that data retrieved from the detachable camera is uploaded to the cloud.

First claim

1 . An augmented reality display system, comprising:
a housing for one or more components for the augmented reality display system, wherein the one or more components comprises at least a spatial light modulator to project light associated with one or more virtual images to a user and a plurality of sensors to capture information pertaining to the user's surroundings, and wherein at least one sensor of the plurality of sensors is an image-based sensor; a processing module communicatively coupled to the housing to process a set of data retrieved from the plurality of sensors, wherein the processing module comprises a gating mechanism that selectively allows the set of data retrieved from the plurality of sensors to be transmitted to a mobile platform; and a detachable camera removably attached to the housing of the augmented reality display system, wherein the gating mechanism selectively allows the set of data retrieved from the plurality of sensors to pass through to the mobile platform based at least in part on whether the detachable camera is detected to be attached to the housing of the augmented reality display system.

2 . The augmented reality display device of claim 1 , wherein when the detachable camera is detected to be attached to the housing of the augmented reality display system, the gating mechanism allows the set of data to pass through to the mobile platform.

3 . The augmented reality display system of claim 1 , wherein when the detachable camera is not detected to be attached to the housing of the augmented reality display system, the gating mechanism allows only a portion of the set of data to pass through to the mobile platform.

4 . The augmented reality display system of claim 1 , wherein at least some of the set of data retrieved from the plurality of sensors corresponds to image or video data.

5 . The augmented reality display system of claim 1 , further comprising a VPU that extracts geometric information from a set of images corresponding to the set of data retrieved from the plurality of sensors.

6 . The augmented reality display system of claim 5 , wherein, when the detachable camera is not detected to be attached to the housing of the augmented reality display system, the gating mechanism allows the geometric information to pass through to the mobile platform, but prevents the set of images from passing through to the mobile platform.

7 . The augmented reality display system of claim 5 , wherein, when the detachable camera is detected to be attached to the housing of the augmented reality display system, the gating mechanism allows the geometric information and the set of images captured when the detachable camera is detected to be attached to the augmented reality display system to pass through to the mobile platform.

8 . The augmented reality display system of claim 5 , wherein the geometric information corresponds to pose information pertaining to the set of images.

9 . The augmented reality display system of claim 5 , wherein the geometric information corresponds to 2D or 3D points pertaining to the set of images.

10 . The augmented reality display system of claim 5 , wherein the geometric information corresponds to depth information pertaining to the set of images.

11 . The augmented reality display system of claim 5 , wherein the geometric information is transmitted to a map database, the map database comprising coordinate information of real objects of the world, such that virtual content is displayed to the user in relation to the real objects of the world.

12 . The augmented reality display system of claim 5 , wherein the VPU comprises an image storage module to store images captured through the plurality of sensors.

13 . The augmented reality display system of claim 5 , wherein the VPU comprises a geometry storage module to store the geometric information pertaining to the images captured through the plurality of sensors.

14 . The augmented reality display system of claim 5 , wherein the gating mechanism allows geometric information to be passed through to the mobile platform even when the detachable camera is not detected to be attached to the housing of the augmented reality display system.

15 . The augmented reality display system of claim 1 , further comprising a camera storage module to store images captured through the detachable camera.

16 . The augmented reality display system of claim 15 , wherein, when the detachable camera is not detected to be attached to the housing of the augmented reality display system, the camera storage module does not transmit the images to mobile platform.

17 . The augmented reality display system of claim 15 , wherein, when the detachable camera is detected to be attached to the housing of the augmented reality display system, the camera storage module transmits the images captured while the detachable camera is detected to be attached to the mobile platform.

18 . The augmented reality display system of claim 1 , wherein the detachable camera comprises an indicator that signals that the detachable camera has been attached to the housing of the augmented reality display system.

19 . The augmented reality display system of claim 18 , wherein the indicator comprises a light that is turned on when the detachable camera has been attached to the housing of the augmented reality display system.

20 . The augmented reality display system of claim 18 , wherein the indicator is a shutter that physically opens or blocks the lens of the detachable camera.

21 . The augmented reality display system of claim 1 , wherein the detachable camera is a third-party camera.

22 . The augmented reality display system of claim 1 , wherein the detachable camera is electrically attached to the housing of the augmented reality device is an electrical means.

23 . The augmented reality display system of claim 1 , wherein the detachable camera is mechanically attached to the housing of the augmented reality device is a mechanical means.

24 . The augmented reality display system of claim 1 , further comprising an electrical contact that completes a circuit when the detachable camera is attached to the detachable camera to detect that the detachable camera has been attached to the housing of the augmented reality display system.

25 . The augmented reality display system of claim 1 , wherein the plurality of sensors comprises an IMU device.

26 . The augmented reality display system of claim 1 , wherein the plurality of sensors comprises an eye-tracking device.

27 . The augmented reality display system of claim 1 , wherein the plurality of sensors comprises a depth camera.

28 . The augmented reality display system of claim 1 , wherein the plurality of sensors comprises a field-of-view camera.

29 . The augmented reality display system of claim 1 , wherein the plurality of sensors comprises an infrared camera.

30 . A method for displaying augmented reality, comprising:
projecting, through a spatial light modulator, one or more virtual images to a user; capturing, through a plurality of sensors, a set of data pertaining to the user's surroundings, wherein the set of data includes an image; processing, through a processing module, the set of data retrieved from the plurality of sensors, the processing module comprising a gating mechanism that selecti

Assignees

• Magic Leap Inc

Inventors

• Bradski Gary R

Classifications

• H04N23/51

  Housings · CPC title

• A63G31/16

  creating illusions of travel · CPC title

• G02B27/0093

  with means for monitoring data relating to the user, e.g. head-tracking, eye-tracking · CPC title

• G02B2027/0138

  comprising image capture systems, e.g. camera · CPC title

• G06F3/013

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title