Portable information device having real-time display with relevant information

Patent metadata

Field	Value
Publication number	US-11979654-B2
Application number	US-202318127043-A
Country	US
Kind code	B2
Filing date	Mar 28, 2023
Priority date	Aug 8, 2008
Publication date	May 7, 2024
Grant date	May 7, 2024

Abstract

A portable information device includes: an image sensor; a position information acquisition portion that outputs position information; an orientation sensor; and a display that displays an image of an object output from the image sensor and displays relevant information on the object, the relevant information on the object being obtained based on at least measurement data from the position information acquisition portion and the orientation sensor.

First claim

What is claimed is:

1. An information search server that sends and receives information to and from a portable information device via an information communication network by wireless communication, wherein the information search server is programmed to:
receive, from the portable information device, (i) image information of a building structure as a search target, the image information generated by the portable information device imaging the building structure, and (ii) position information indicating a position of the building structure at the time of the imaging of the building structure by the portable information device;
determine a similarity between (1) a characteristic amount obtained by image recognition of the image information of the building structure received from the portable information device as the search target, and (2) a characteristic amount obtained by image recognition of a building structure image stored in a database, the building structure image selected from the database based on the position information received from the portable information device;
identify the building structure of the search target as corresponding to the building structure image that was selected from the database when the similarity that was determined is equal to or more than a set value; and
send, to the portable information device, search result information indicating a result of searching for information related to the building structure of the search target.

2. The information search server according to claim 1 , wherein:
the information search server receives from the portable information device, in addition to the position information, imaging direction information indicating an imaging direction at the time of the imaging of the building structure, and
the building structure of the search target is identified using the position information and the imaging direction information.

3. The information search server according to claim 1 , wherein:
if the building structure of the search target that has been identified is a store, the search result information includes commercial information related to the store; and
the commercial information includes at least one of an in-store image of the store and information related to a product that is being sold at the store.

4. The information search server according to claim 3 , wherein:
the search result information includes post information related to the store.

5. The information search server according to claim 2 , wherein:
if the building structure of the search target that has been identified is a store, the search result information includes commercial information related to the store; and
the commercial information includes at least one of an in-store image of the store and information related to a product that is being sold at the store.

6. The information search server according to claim 5 , wherein:
the search result information includes post information related to the store.

7. The information search server according to claim 1 , wherein the information search server is further programmed to:
when the similarity that was determined is less than the set value:
determine a similarity between (a) the characteristic amount obtained by image recognition of the image information of the building structure received from the portable information device as the search target, and (b) a characteristic amount obtained by image recognition of a second building structure image stored in the database, the second building structure image selected from the database based on the position information received from the portable information device; and

identify the building structure of the search target as corresponding to the second building structure image that was selected from the database when the similarity that was determined is equal to or more than the set value.

Assignees

• Nikon Corp

Inventors

• Inoue Hideya
• Iwaoka Toru
• Noborisaka Michiko
• Hatori Masayuki
• Hamada Tomohide

Classifications

• H04N23/631 Primary

  Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters · CPC title

• H04N1/00244 Primary

  with a server, e.g. an internet server (fax-servers or the like for store and forward H04N1/324) · CPC title

• H04N1/00307

  with a mobile telephone apparatus (constructional features of portable telephone sets H04M1/0202; mobile radio systems H04W84/00) · CPC title

• H04N1/00411

  the display also being used for user input, e.g. touch screen · CPC title

• H04N1/32117

  in a separate transmission or protocol signal prior to or subsequent to the image data transmission, e.g. in digital identification signal [DIS], in non standard setup [NSS] or in non standard field [NSF] (for mode signalling H04N1/333) · CPC title