Portable information device having real-time display with relevant information

Patent metadata

Field	Value
Publication number	US-11445117-B2
Application number	US-202117142722-A
Country	US
Kind code	B2
Filing date	Jan 6, 2021
Priority date	Aug 8, 2008
Publication date	Sep 13, 2022
Grant date	Sep 13, 2022

Abstract

A portable information device including a body, a position information portion that is provided in the body and that outputs position information, an image sensor that is provided in the body, a display system that is provided in the body and that displays, on a local field in a screen field, relevant information based on the position information such that the displayed relevant information is overlapped on a real-time image, an input portion that is provided on the body and, by use of which user operates an icon, the icon being displayed on the local field such that the displayed icon is overlapped on the real-time image and an imaging button that is provided, separately from the input portion, on the body and that is operated by user to capture a subject image via the image sensor.

First claim

What is claimed is:

1. An information acquisition system comprising:
a body;
a locator provided in the body;
an orientation sensor provided in the body;
an image sensor provided in the body;
a database that stores building structure information;
a server configured to find a building structure based on position information from the locator and orientation information from the orientation sensor and to output structure information of the building structure found by the server, the building structure information including peripheral image data of the building structure; and
a display system having a display and a controller, each being provided in the body and configured to:
compare the peripheral image data with the building structure information stored in the database to detect whether a threshold is met;
when the threshold the threshold is met, select, as the building structure, a first building structure that is located at closest distance from the body;
when the threshold is not met, select, as the building structure, a second building structure that is located at a next closest distance from the body;
display, on a local field in a real-time field, the structure information such that the displayed structure information is overlapped on at least a live view; and
display, on the local field in the real-time field, a peripheral image around the building structure, the peripheral image being based on the peripheral image data, and being displayed such that the peripheral image is overlapped on at least the live view.

2. The information acquisition system according to claim 1 ,
wherein the local field, where the structure information is displayed, includes an area located adjacent one side or another side of the display in a width direction.

3. The information acquisition system according to claim 1 ,
wherein the display system displays an operation icon while the structure information is displayed, the operation icon being displayed on the local field such that the displayed operation icon is overlapped on at least the live view, the operation icon being operable to cause performance of an operation.

4. The information acquisition system according to claim 1 ,
wherein the display system displays the structure information that corresponds to a thumbnail image selected by a user among a plurality of thumbnail images displayed on the display.

5. A portable information device configured to be in wireless communication with an information search server via an information communication network, the portable information device comprising:
a body;
an image sensor that is provided in the body and is configured to image a subject and output image information regarding an image of the subject;
a locator that is provided in the body and is configured to output position information;
a transmitter that is provided in the body and is configured to transmit the image and the position information to the information search server and receive search result information from the information search server;
a memory that is provided in the body;
a touch panel display that is provided in the body; and
a controller that is provided in the body and is configured to:
in response to a touch operation to execute a search about a building structure as a search target while a search target building structure is imaged through the image sensor and a target structure image is displayed on the touch panel display, display the search result information on the touch panel display along with the target structure image, the search result information including relevant image information about the search target building structure and relevant text information about the relevant image information, the search result information being based on the position information corresponding to the target structure image and a result of an image recognition for the target structure image; and
in response to another touch operation while the search result information is displayed on the touch panel display, display additional information about the search result information on the touch panel display.

6. The portable information device according to claim 5 , wherein the search result information is output, based on a determination of the search target building structure, by comparing a characteristic amount of the target structure image with a characteristic amount of relevant structure image extracted from a database of the information search server by using the position information.

7. The portable information device according to claim 5 , further comprising:
an orientation sensor, wherein the position information is used along with orientation information from the orientation sensor,
wherein the controller is configured to control the transmitter to transmit the orientation information along with the position information to the information search server, and
the search result information is based on the position information corresponding to the target structure image, the orientation information corresponding to the target structure image and a result of an image recognition for the target structure image.

8. The portable information device according to claim 5 , wherein the additional information includes text information about the search target building structure and image information about the search target building structure.

9. The portable information device according to claim 5 ,
wherein the search target building structure is a store, and
the additional information includes commercial information, the commercial information including an image captured inside of the store and product information.

10. The portable information device according to claim 5 ,
wherein the search target building structure is a store, and
the additional information includes posted information about the store.

11. The portable information device according to claim 5 , wherein
in response to a touch operation in which a user selects one structure image among a plurality of structure images while the plurality of structure images, which are past images photographed in the past and read from the memory, are displayed as thumbnail images on the touch panel display, display the search result information on the touch panel display along with the target structure image, the search result information including relevant image information about the search target building structure and relevant text information about the relevant image information, the search result information being based on the position information corresponding to the target structure image and a result of an image recognition for the target structure image; and
in response to another touch operation while the search result information is displayed on the touch panel display, display additional information about the search result information on the touch panel display.

12. An information search server configured to be in wireless communication with a portable information device, the server is configured to:
receive, from the portable information device, a target structure image as a search target and positional information, which corresponds to a position where a target structure is imaged; and
transmit search result information to the portable information device, the search result information being based on the positional information and image recognition information, the image recognition information being based on comparison of a characteristic amount of the target structure image with a characteristic amount of a building structure image stored in a database to detect whether a threshold is met, the search result information including rele

Assignees

• Nikon Corp

Inventors

• Inoue Hideya
• Iwaoka Toru
• Noborisaka Michiko
• Hatori Masayuki
• Hamada Tomohide

Classifications

• H04N1/00244 Primary

  with a server, e.g. an internet server (fax-servers or the like for store and forward H04N1/324) · CPC title

• H04N23/631 Primary

  Graphical user interfaces [GUI] specially adapted for controlling image capture or setting capture parameters · CPC title

• H04N23/633

  for displaying additional information relating to control or operation of the camera · CPC title

• H04N23/64

  Computer-aided capture of images, e.g. transfer from script file into camera, check of taken image quality, advice or proposal for image composition or decision on when to take image · CPC title

• H04N23/63

  by using electronic viewfinders · CPC title