Touchscreen image mapping for augmented reality devices

Patent metadata

Field	Value
Publication number	US-10318744-B2
Application number	US-201514809572-A
Country	US
Kind code	B2
Filing date	Jul 27, 2015
Priority date	Jul 27, 2015
Publication date	Jun 11, 2019
Grant date	Jun 11, 2019

Abstract

As disclosed herein a method, executed by a computer, includes receiving, from an augmented reality device, a pairing request, negotiating with the augmented reality device to generate a new encryption key and a mapping plan for presenting optically readable codes, that correspond to one or more images, on a touch sensitive display device. The method further includes presenting, on the touch sensitive display device, the optically readable codes, and receiving a user selection from the touch sensitive display device. The optical codes are overlayed by the wearers of an augmented reality system with an un-encrypted or otherwise modified form of the optical code. A computer system, and a computer program product corresponding to the above method are also disclosed herein.

First claim

What is claimed is:

1. An augmented reality display device comprising:
an augmented reality display hardware set for simultaneously displaying displays that include: (i) a display portion corresponding to the real world, and (ii) a computerized overlay portion;
a processor(s) set;
a storage medium; and
computer code stored on the storage medium, with the computer code including data and instructions to cause the processor(s) set to perform at least the following operations:
receiving a set of decryption key(s) for decrypting QR codes,
receiving a first image that includes in the image a representation of a touchscreen display device display including a QR keyboard that includes a plurality of QR codes respectively corresponding to alphanumeric keys,
decrypting the QR codes of the QR keyboard respectively corresponding to alphanumeric keys to obtain respectively corresponding sets of alphanumeric character(s),
generating computerized overlay information corresponding to the sets of alphanumeric character(s) respectively corresponding to the decrypted QR codes of the QR keyboard in the first image,
creating a first augmented reality display data set including the sets of alphanumeric character(s) overlaying the respectively corresponding QR codes of the QR keyboard portion of the first image, and
displaying a first augmented reality display according to the first augmented reality display data set.

2. The augmented reality display device of claim 1 wherein the augmented reality display hardware set includes:
a transparent portion for allowing the display portion corresponding to the real world to be visually communicated to a user; and
a computerized overlay displaying hardware set for displaying the computerized overlay portion.

3. The augmented reality display device of claim 1 wherein the augmented reality display hardware set includes:
an image combining module for combining the display portion corresponding to the real world with the computerized overlay portion to created augmented reality images; and
a display for displaying the augmented reality images.

4. A system comprising:
computer device comprising:
a touchscreen display user interface device,
a processor(s) set,
a storage medium, and
computer code stored on the storage medium, with the computer code including data and instructions to cause the processor(s) set to perform at least the following operations:
receiving a mapping plan data set including information indicative of: (i) a plurality of alphanumeric keys for a display of a keyboard, and (ii) relative locations among and between the alphanumeric keys of the keyboard,
receiving a set of encryption key(s) for encrypting the alphanumeric keys into a plurality of respectively corresponding QR codes,
encrypting the alphanumeric keys of the mapping plan into a plurality of respectively corresponding QR codes using the set of encryption key(s),
creating a QR keyboard display data set that corresponds to a display of a QR keyboard including the plurality of QR codes arranged into a keyboard display according to the relative locations of the mapping plan data set,
displaying the QR keyboard on the touchscreen display user interface device, and
receiving user input corresponding to keystrokes made by a user on the QR keyboard displayed on the touchscreen display user interface device; and

an augmented reality display device comprising:
an augmented reality display hardware set for simultaneously displaying displays that include: (i) a display portion corresponding to the real world, and (ii) a computerized overlay portion,
a processor(s) set,
a storage medium, and
computer code stored on the storage medium, with the computer code including data and instructions to cause the processor(s) set to perform at least the following operations:
receiving a set of decryption key(s) for decrypting QR codes,
receiving a first image that includes in the image a representation of a touchscreen display device display including the QR keyboard that includes the plurality of QR codes respectively corresponding to alphanumeric keys,
decrypting the QR codes of the QR keyboard respectively corresponding to alphanumeric keys to obtain respectively corresponding sets of alphanumeric character(s),
generating computerized overlay information corresponding to the sets of alphanumeric character(s) respectively corresponding to the decrypted QR codes of the QR keyboard in the first image,
creating a first augmented reality display data set including the sets of alphanumeric character(s) overlaying the respectively corresponding QR codes of the QR keyboard portion of the first image, and
displaying a first augmented reality display according to the first augmented reality display data set.

5. The system of claim 4 wherein the computer device is in the form of a smart phone.

6. The system of claim 4 wherein:
each alphanumeric key respectively corresponds to a set of alphanumeric character(s); and
the encryption of each given alphanumeric key includes encrypting the set of alphanumeric character(s) respectively corresponding to the given alphanumeric key.

7. The system of claim 4 wherein the user is viewing an augmented reality display with a computerized overlay portion of the augmented reality display showing alphanumeric characters in place of the QR codes of the QR keyboard.

8. The system of claim 4 wherein the augmented reality display hardware set includes:
a transparent portion for allowing the display portion corresponding to the real world to be visually communicated to the user; and
a computerized overlay displaying hardware set for displaying the computerized overlay portion.

9. The system of claim 4 wherein the augmented reality display hardware set includes:
an image combining module for combining the display portion corresponding to the real world with the computerized overlay portion to created augmented reality images; and
a display for displaying the augmented reality images.

Assignees

• IBM

Inventors

• Dow Eli M
• Fitzsimmons Thomas D
• Harvey Joseph D
• Rohde Douglas E

Classifications

• G02B2027/014

  comprising information/image processing systems · CPC title

• G02B2027/0187

  slaved to motion of at least a part of the body of the user, e.g. head, eye · CPC title

• G06F21/602 Primary

  Providing cryptographic facilities or services · CPC title

• G06F21/31

  User authentication · CPC title

• G09G2354/00

  Aspects of interface with display user · CPC title