Display and processing methods and related apparatus

Patent metadata

Field	Value
Publication number	US-10921979-B2
Application number	US-201515781618-A
Country	US
Kind code	B2
Filing date	Dec 7, 2015
Priority date	Dec 7, 2015
Publication date	Feb 16, 2021
Grant date	Feb 16, 2021

Abstract

A method includes obtaining, by a head mounted display, a trigger instruction and determining coordinates of a line of sight focus of a user on a screen of a portable electronic device, where the trigger instruction is used to instruct the head mounted display to perform a display operation, sending the coordinates to the portable electronic device, when the coordinates trigger the portable electronic device to determine that an object corresponding to the coordinates can be processed by the portable electronic device, obtaining, by the portable electronic device, related content of the object, obtaining the related content of the object of the portable electronic device, and displaying the related content of the object on a virtual screen.

First claim

What is claimed is:

1. A display method, applied to a head mounted display keeping a wireless coupling to a portable electronic device, comprising:
detecting an instruction and determining coordinates of a fixation point of a user on a screen of the portable electronic device;
sending the coordinates and an indication identifying the instruction from the head mounted display to the portable electronic device;
receiving related content of an object corresponding to the coordinates from the portable electronic device;
enlarging the related content according to a preset multiple to form enlarged related content;
displaying the enlarged related content only on a virtual screen corresponding to the portable electronic device and not displaying the enlarged related content on the screen of the portable electronic device;
displaying a mark of the fixation point;
detecting whether the mark of the fixation point remains at a single point for a time exceeding a preset threshold;
blinking the mark of the fixation point when the time exceeds the preset threshold; and
selecting a target object associated with the single point when the time exceeds the preset threshold.

2. The display method of claim 1 , wherein the related content of the object corresponding to the coordinates comprises a screenshot of a preset range on the screen of the portable electronic device, and wherein the display method further comprises:
determining another group of coordinates of the fixation point of the user on the screenshot displayed on the virtual screen;
changing the other group of coordinates into coordinates applicable to the screen of the portable electronic device;
sending the changed other group of coordinates to the portable electronic device;
receiving another related content of an object corresponding to the changed other group of coordinates from the portable electronic device; and
displaying the other related content on the virtual screen corresponding to the portable electronic device.

3. The display method of claim 1 , wherein displaying the enlarged related content on the virtual screen corresponding to the portable electronic device comprises displaying all the related content on the virtual screen corresponding to the portable electronic device and on which preset zooming is performed.

4. The display method of claim 1 , further comprising:
detecting another instruction instructing the head mounted display and the portable electronic device to perform an operation of stop displaying the related content and stop displaying the related content on the virtual screen; or
closing the virtual screen.

5. A head mounted display, keeping a wireless coupling to a portable electronic device, comprising:
a processor configured to detect an instruction and determine coordinates of a fixation point of a user on a screen of the portable electronic device;
a transmitter coupled to the processor and configured to send the coordinates and an indication identifying the instruction from the head mounted display to the portable electronic device; and
a receiver coupled to the processor and the transmitter and configured to receive related content of an object corresponding to the coordinates from the portable electronic device, and
wherein the processor is further configured to:
enlarge the related content according to a preset multiple to form enlarged related content;
display the enlarged related content only on a virtual screen corresponding to the portable electronic device and not display the enlarged related content on the screen of the portable electronic device;
display a mark of the fixation point of the user;
detect whether the mark of the fixation point remains at a single point for a time exceeding a preset threshold;
blink the mark of the fixation point when the time exceeds the preset threshold; and
select a target object associated with the single point when the time exceeds the preset threshold.

6. The head mounted display of claim 5 , wherein the related content of the object corresponding to the coordinates comprises a screenshot of a preset range on the screen of the portable electronic device, and wherein the processor is further configured to:
determine another group of coordinates of the fixation point of the user on the screenshot displayed on the virtual screen; and
change the other group of coordinates into coordinates applicable to the screen of the portable electronic device,
wherein the transmitter is further configured to send the changed other group of coordinates to the portable electronic device,
wherein the receiver is further configured to receive another related content of an object corresponding to the changed other group of coordinates from the portable electronic device, and
wherein the processor is further configured to display the other related content on the virtual screen corresponding to the portable electronic device.

7. The head mounted display of claim 5 , wherein the processor is further configured to display all the related content on the virtual screen corresponding to the portable electronic device and on which preset zooming is performed.

8. The head mounted display of claim 5 , wherein the processor is further configured to:
detect another instruction instructing the head mounted display and the portable electronic device to perform an operation of stop displaying the related content and stop displaying the related content on the virtual screen; or
close the virtual screen.

9. The display method of claim 1 , further comprising obtaining three-dimensional coordinates of the portable electronic device in a three-dimensional coordinate system.

10. The display method of claim 1 , further comprising tracking eye movement of the user.

11. The display method of claim 1 , wherein the portable electronic device comprises a smart watch or a smart phone.

12. The display method of claim 1 , wherein the instruction is configured to prompt the portable electronic device to run an application on the portable electronic device.

13. The display method of claim 1 , wherein the instruction is configured to prompt the portable electronic device to open a folder on the portable electronic device.

14. The display method of claim 1 , wherein the instruction is configured to prompt the portable electronic device to perform a query for an object located in the fixation point.

15. The display method of claim 1 , wherein the instruction is configured to prompt the portable electronic device to perform a translation.

16. The display method of claim 1 , wherein the head mounted display and the portable electronic device are directly coupled together using a BLUETOOTH connection.

17. The display method of claim 1 , wherein the mark of the fixation point comprises a red spot, wherein the preset threshold comprises two seconds, and wherein the method further comprises blinking the red spot when the time exceeds the two seconds.

18. The head mounted display of claim 5 , wherein the mark of the fixation point comprises a red spot, wherein the preset threshold comprises two seconds, and wherein the processor is further configured to blink the red spot when the time exceeds the two seconds.

19. A computer program product comprising computer-executable instructions for storage on a non-transitory computer-readable medium that, when executed by a processor, cause an apparatus to:
detect an instruction and determine coordinates of a fixation point of a user on a screen of a portable electronic device;
send the coor

Assignees

• Huawei Tech Co Ltd

Inventors

• Gao Wenmei
• Fan Shunan
• Wang Yahui
• Fu dian
• Kong Weiguang

Classifications

• G09G3/003

  to produce spatial visual effects · CPC title

• G06F3/04883 Primary

  for inputting data by handwriting, e.g. gesture or text · CPC title

• G06F3/013 Primary

  Eye tracking input arrangements (G06F3/015 takes precedence) · CPC title

• G06F3/1423

  controlling a plurality of local displays, e.g. CRT and flat panel display · CPC title

• G06F3/167

  Audio in a user interface, e.g. using voice commands for navigating, audio feedback · CPC title