Portable information device having real-time display with relevant information

Patent metadata

Field	Value
Publication number	US-11647276-B2
Application number	US-202217882052-A
Country	US
Kind code	B2
Filing date	Aug 5, 2022
Priority date	Aug 8, 2008
Publication date	May 9, 2023
Grant date	May 9, 2023

Abstract

A portable information device including a body, a position information portion that is provided in the body and that outputs position information, an image sensor that is provided in the body, a display system that is provided in the body and that displays, on a local field in a screen field, relevant information based on the position information such that the displayed relevant information is overlapped on a real-time image, an input portion that is provided on the body and, by use of which user operates an icon, the icon being displayed on the local field such that the displayed icon is overlapped on the real-time image and an imaging button that is provided, separately from the input portion, on the body and that is operated by user to capture a subject image via the image sensor.

First claim

What is claimed is:

1. A portable information device configured to be in wireless communication with an information search server via an information communication network, the portable information device comprising:
a body;
an image sensor that is provided in the body and is configured to image a subject and output image information regarding an image of the subject;
a locator that is provided in the body and is configured to output position information;
a transmitter that is provided in the body and is configured to transmit the image and the position information to the information search server and receive search result information from the information search server;
a memory that is provided in the body;
a touch panel display that is provided in the body; and
a controller that is provided in the body and is configured to:
in response to a touch operation, display a plurality of images stored in the memory on the touch panel display,
in response to a touch operation to select a building structure image as a search target among the plurality of displayed images, execute a search about the selected building structure image,
display a result of the search as search result information on the touch panel display along with the target structure image, the search result information including relevant image information about the search target building structure and relevant text information about the relevant image information, the search result information being based on the position information corresponding to the target structure image and a result of an image recognition for the target structure image; and
in response to another touch operation while the search result information is displayed on the touch panel display, display additional information about the search result information on the touch panel display.

2. The portable information device according to claim 1 , wherein the search result information is output, based on a determination of the search target building structure, by comparing a characteristic amount of the target structure image with a characteristic amount of relevant structure image extracted from a database of the information search server by using the position information.

3. The portable information device according to claim 1 , further comprising:
an orientation sensor, wherein the position information is used along with orientation information from the orientation sensor, wherein
the controller is configured to control the transmitter to transmit the orientation information along with the position information to the information search server, and
the search result information is based on the position information corresponding to the target structure image, the orientation information corresponding to the target structure image and a result of an image recognition for the target structure image.

4. The portable information device according to claim 1 , wherein the additional information includes text information about the search target building structure and image information about the search target building structure.

5. The portable information device according to claim 1 , wherein
the search target building structure is a store, and
the additional information includes commercial information, the commercial information including an image captured inside of the store and product information.

6. The portable information device according to claim 1 , wherein the search target building structure is a store, and the additional information includes posted information about the store.

7. An information acquisition system comprising:
an imager that captures an image of a search target and outputs image data;
a position sensor that acquires position information of an imaging position;
a memory that stores the image data and the position information;
a display that displays images and text, and detects a touch operation as an instruction operation by a user;
an information retrieval server that performs information retrieval based on the image data from the imager and the position information from the position sensor; and a computer having a first mode and a second mode, one of which is selected in response to the touch operation by the user, wherein
in the first mode,
(a) a target image is displayed on the display based on image data of a target captured via the imager and recognized by the user,
(b) in response to a touch operation by the user, target data of a search target is transmitted to the information retrieval server, the target data including data of the target image and position information, the position information being acquired via the position sensor and corresponding to the target image,
(c) first search result information is displayed so that both the target image and searched image data are displayed on the display together, the first search result information including the searched image data and text data as a primary search result received from the information retrieval server based on the target image and the position information, and
(d) second search result information transmitted from the information retrieval server is displayed on the display based on a further search based on signals transmitted to the information retrieval server, the further search being executed in response to a touch operation by the user in a state in which the searched image data is displayed on the display together with the target image, and
in the second mode,
(e) a plurality of images stored in the memory are displayed on the display,
(f) in response to a touch operation by the user to select a target image among the plurality of displayed images, information retrieval is performed based on the selected target image and position information corresponding to the selected target image,
(g) third search result information is displayed on the display together with the selected target image, the third search result information including searched image data and text data as a primary search result from the information retrieval server, and
(h) fourth search result information transmitted from the information retrieval server is displayed on the display based on a further search, the further search being executed in response to a touch operation by the user in a state in which the third search result information is displayed on the display together with the selected target image.

8. The information acquisition system according to claim 7 , wherein the information retrieval server is configured (1) to compare amount of features by image recognition between the captured image data of the search target and stored image data in a database and (2) to determine whether the captured image data corresponds to the search target based on the comparison result.

9. An imaging device comprising:
an imager;
a position sensor;
a memory;
a transmitter;
a display; and
a controller configured to control the imager, the memory, the transmitter, and the display and having a first mode and a second mode, one of which is selected in response to a touch operation by a user, wherein
in the first mode,
(a) a subject image is displayed on the display based on image data of a subject captured via the imager and recognized by the user,
(b) in response to a touch operation by the user, subject data of a search target is transmitted to an information retrieval server via the transmitter, the subject data including data of the subject image and position information, the position information being acquired via the position sensor and corresponding to the subject image,
(c) first search result information is displayed so that both the target image and searched image data are displayed on the display togeth

Assignees

• Nikon Corp

Inventors

• Inoue Hideya
• Iwaoka Toru
• Noborisaka Michiko
• Hatori Masayuki
• Hamada Tomohide

Classifications

• H04N2201/3253

  Position information, e.g. geographical position at time of capture, GPS data · CPC title

• H04N1/00411

  the display also being used for user input, e.g. touch screen · CPC title

• H04N1/32117

  in a separate transmission or protocol signal prior to or subsequent to the image data transmission, e.g. in digital identification signal [DIS], in non standard setup [NSS] or in non standard field [NSF] (for mode signalling H04N1/333) · CPC title

• H04N2201/3273

  Display · CPC title

• H04N2201/3205

  of identification information, e.g. name or ID code (H04N2201/3209 takes precedence) · CPC title