Program, information processing method, and information processing terminal

Patent metadata

Field	Value
Publication number	US-11734898-B2
Application number	US-202117381019-A
Country	US
Kind code	B2
Filing date	Jul 20, 2021
Priority date	Jan 28, 2019
Publication date	Aug 22, 2023
Grant date	Aug 22, 2023

Abstract

There is provided a program for causing an information processing terminal to execute a step of acquiring position information indicating a position of the information processing terminal, a step of transmitting the position information to an information processing apparatus, a step of receiving display data related to one or more products which is associated with the position information from the information processing apparatus, a step of acquiring a first region which satisfies a predetermined condition related to safety of a user, the first region being a region in an image taken by an image pickup unit, and a step of outputting the display data related to the one or more products to the acquired first region in a real space corresponding to the image or the image.

First claim

What is claimed is:

1. A computer-readable non-transitory storage medium storing a program for causing an information processing device utilized by a user to execute:
a step of acquiring position information indicating a position of the information processing device;
a step of transmitting the position information to an information processing apparatus;
a step of receiving display data related to one or more products which is associated with the position information from the information processing apparatus;
a step of receiving 3D model data of a seller of a first product of the one or more products, the 3D model data generated by capturing an image of the seller;
a step of acquiring a first region where the user can safely walk, the first region being a region in an image taken by an image pickup unit;
a step of acquiring a second region where the user may be exposed to danger if the user were to linger, the second region identified from the image taken by the image pickup unit; and
a step of outputting the display data related to the one or more products to a position overlapping with the first region and not overlapping with the second region in: (i) a real space corresponding to the image, or (ii) the image;
wherein the step of outputting comprises outputting a 3D model of the seller using the received 3D model data, the 3D model of the seller displayed in association with display data related to the first product of the one or more products.

2. The computer-readable non-transitory storage medium according to claim 1 , wherein
the step of outputting comprises outputting the display data related to the one or more products such that the display data is arranged on a boundary of the first region.

3. The computer-readable non-transitory storage medium according to claim 1 , wherein
the program causes the information processing device to further execute:
a step of accepting selection of the one or more products from the user; and
a step of transmitting information for identification of an accepted product to the information processing apparatus.

4. The computer-readable non-transitory storage medium according to claim 1 , wherein
the program causes the information processing device to further execute a step of acquiring pathway information indicating a pathway, by which the user moves, from the information processing apparatus or a different information processing apparatus, and
the step of outputting comprises outputting the product data related to the one or more products along the pathway indicated by the pathway information.

5. The computer-readable non-transitory storage medium according to claim 1 , wherein
the step of receiving comprises further receiving attribute data of the one or more products from the information processing apparatus, and
the step of outputting comprises outputting attribute data of a product corresponding to first display data included in the display data of the one or more products when the information processing device moves in a direction toward the first display data.

6. The computer-readable non-transitory storage medium according to claim 1 , wherein
the display data includes at least one of image data of a product, three-dimensional model data of the product, and discount information of the product.

7. An information processing device utilized by a user, comprising:
one or more processors;
a storage device communicatively coupled to the one or more processors; and
one or more programs stored in the storage device and configured for execution by the one or more processors, the one or more programs comprising instructions for:
acquiring position information indicating a position of the information processing device;
transmitting the position information to an information processing apparatus;
receiving display data related to one or more products which is associated with the position information from the information processing apparatus;
receiving 3D model data of a seller of a first product of the one or more products, the 3D model data generated by capturing an image of the seller;
acquiring a first region where the user can safely walk, the first region being a region in an image taken by an image pickup unit;
acquiring a second region where the user may be exposed to danger if the user were to linger, the second region identified from the image taken by the image pickup unit;
outputting the display data related to the one or more products to a position overlapping with the first region and not overlapping with the second region in: (i) a real space corresponding to the image, or (ii) the image; and
outputting a 3D model of the seller using the received 3D model data, the 3D model of the seller displayed in association with display data related to the first product of the one or more products.

8. The information processing device of claim 7 , wherein the first region where the user can safely walk comprises a sidewalk, and wherein the display data related to the one or more product is to be displayed along an edge of the sidewalk.

9. The information processing device of claim 7 , wherein the one or more programs further comprise instructions for:
after outputting the display data, receiving an indication from the user that a product of the one or more products is of interest to the user; and
marking the product as a favorite of the user.

10. The information processing device of claim 7 , wherein the information processing device comprises artificial reality (AR) glasses.

11. The information processing device of claim 7 , wherein the one or more programs further comprise instructions for, prior to acquiring the first region, transmitting a destination of the user to the information processing apparatus, wherein the first region is based on the position of the information processing device and the destination of the user.

12. The information processing device of claim 7 , wherein the outputting comprises outputting the display data related to the one or more products such that the display data is arranged on a boundary of the first region.

13. The information processing device of claim 7 , wherein the one or more programs further comprise instructions for:
accepting selection of the one or more products from the user; and
transmitting information for identification of an accepted product to the information processing apparatus.

14. The information processing device of claim 7 , wherein the one or more programs further comprise instructions for:
acquiring pathway information indicating a pathway, by which the user moves, from the information processing apparatus or a different information processing apparatus, and
the step of outputting comprises outputting the product data related to the one or more products along the pathway indicated by the pathway information.

15. The information processing device of claim 7 , wherein:
the receiving comprises further receiving attribute data of the one or more products from the information processing apparatus, and
the outputting comprises outputting attribute data of a product corresponding to first display data included in the display data of the one or more products when the information processing device moves in a direction toward the first display data.

16. The information processing device of claim 7 , wherein the display data includes at least one of image data of a product, three-dimensional model data of the product, and discount information of the product.

17. The information processing device of claim 7 , wherein the second region correspond

Assignees

• Mercari Inc

Inventors

• Yanagisawa Satoshi

Classifications

• G06T19/006 Primary

  Mixed reality (object pose determination, tracking or camera calibration for mixed reality G06T7/00) · CPC title

• G06V20/20

  in augmented reality scenes · CPC title

• G06V20/647 Primary

  by matching two-dimensional images to three-dimensional objects · CPC title

• G06Q30/0639

  Locating goods or services, e.g. based on physical position of the goods or services within a shopping facility · CPC title

• G06Q30/0643

  graphically representing goods, e.g. 3D product representation · CPC title