Method and system for performing adaptive image acquisition

Patent metadata

Field	Value
Publication number	US-9854147-B2
Application number	US-201113336812-A
Country	US
Kind code	B2
Filing date	Dec 23, 2011
Priority date	Aug 10, 2004
Publication date	Dec 26, 2017
Grant date	Dec 26, 2017

Abstract

An adaptive image acquisition system and method that generates virtual view of a surveillance scene to a user (operator), in which, the user operates the system. Through viewing the virtual view, the user controls sensors that create the virtual view. The sensors comprise at least one first sensor having a higher resolution than at least one second sensor. Images from the second sensor are processed to create an image mosaic that is overlaid with images from the higher resolution first sensor. In one embodiment of the invention, the first sensor is moved using Saccade motion. In another embodiment of the invention, a user's intent is used to control the Saccade motion.

First claim

The invention claimed is:

1. An adaptive image acquisition system comprising:
a registration module for registering a first image and a second image to create a plurality of registered images, wherein the first image has a higher resolution than the second image;
a virtual view generation module for creating a virtual view of a scene corresponding to a scene represented in the first image and the second image, wherein the virtual view is created using the registered images; and
an interface, coupled to the virtual view generation module, for enabling a user to control one or more imaging sensors while viewing the virtual view, wherein at least one of the one or more imaging sensors are moved according to a motion pattern in response to a reaction of the user to the virtual view in order to capture objects of interest to the user, wherein the motion pattern is algorithmically determined by the adaptive image acquisition system.

2. The system of claim 1 wherein the plurality of registered images is produced by the one or more imaging sensors.

3. The system of claim 1 wherein the second image is formed by sub-sampling the first image.

4. The system of claim 2 wherein the one or more imaging sensors comprise a first sensor, wherein the first sensor is a narrow field of view camera and a second sensor, wherein the second sensor is a wide field of view camera.

5. The system of claim 4 wherein the second sensor is moved according to the motion pattern, and the motion pattern is Saccade motion.

6. The system of claim 5 wherein the virtual view comprises an image mosaic of images captured by the second sensor that is overlaid with images from the first sensor.

7. The system of claim 1 further comprising an intent estimation module for deriving a user's intent from a user's control of the interface and using the user's intent to control the motion.

8. The system of claim 7 wherein the intent estimation module comprises an exploration task estimation module and a user intent estimation module.

9. The system of claim 8 wherein the exploration task estimation module uses a hidden Markov model.

10. The system of claim 7 wherein the intent estimation module creates an interest map of a user's interest in specific regions and/or objects that have been imaged.

11. The system of claim 1 wherein a user indirectly controls the at least one sensor through the interface.

12. The system of claim 1 further comprising an alert generation module coupled to the virtual view generation module for identifying the objects of interests, possible targets and changes in behavior in the first image and the second image and notifying the user.

13. The system of claim 12 , wherein the alert generation module analyzes behavior of possible targets and compares the behavior to an ontology of target behavior.

14. The system of claim 12 wherein the alert generation module is coupled with an object and event detection module.

15. A method of performing adaptive image acquisition comprising:
creating a virtual view of a scene corresponding to captured images;
enabling a user to control one or more imaging sensors while viewing the virtual view;
processing a user's historical control of the one or more imaging sensors to estimate an intent of the user with respect to controlling the one or more sensors; and
controlling the one or more imaging sensors in response to the estimated intent of the user, wherein at least one of the one or more imaging sensors are moved according to a motion pattern in response to a reaction of the user to the virtual view in order to capture objects of interest to the user, wherein the motion pattern is algorithmically determined.

16. The method of claim 15 wherein the captured images comprise a first image and a second image, where the first image is produced by a first sensor and the second image is produced by a second sensor, wherein the first sensor has a higher resolution than the second sensor.

17. The method of claim 16 wherein the second image is formed by sub-sampling the first image.

18. The method of claim 16 wherein the first sensor is a narrow field of view camera and the second sensor is a wide field of view camera.

19. The method of claim 16 wherein the at least one first sensor is moved using Saccade motion.

20. The method of claim 19 wherein the virtual view comprises an image mosaic of images captured by the second sensor that is overlaid with images from the first sensor.

Assignees

• Cheng Hui
• Stanford Res Inst Int

Inventors

• Cheng Hui

Classifications

• G06T1/0007 Primary

  Image acquisition · CPC title

• G08B13/19643

  wherein the cameras play different roles, e.g. different resolution, different camera type, different control type · CPC title

• G06V20/13

  Satellite images · CPC title

• G06V10/16

  using multiple overlapping images; Image stitching · CPC title

• G06T2200/32

  involving image mosaicing · CPC title