Image data processing method, device and security inspection system based on VR or AR

Patent metadata

Field	Value
Publication number	US-11699223-B2
Application number	US-201715858868-A
Country	US
Kind code	B2
Filing date	Dec 29, 2017
Priority date	Dec 29, 2016
Publication date	Jul 11, 2023
Grant date	Jul 11, 2023

Abstract

A method, a device and a security system for image data processing based on VR or AR are disclosed. In one aspect, an example image data processing method includes reconstructing, based on three-dimensional (3D) scanned data of a containing apparatus in which objects are contained, a 3D image of the containing apparatus and the objects contained in the containing apparatus. The reconstructed 3D image is stereoscopically displayed. Manipulation information is determined for at least one of the objects in the containing apparatus based on positioning information and action information of a user. At least a 3D image of the at least one object is reconstructed based on the determined manipulation information. The at least one reconstructed object is presented on the displayed 3D image.

First claim

What is claimed is:

1. An image data processing method for security inspection, comprising:
reconstructing, based on three-dimensional (3D) scanned data of a containing apparatus in which objects are contained, a 3D image of the containing apparatus and the objects contained in the containing apparatus;
stereoscopically displaying the reconstructed 3D image in a virtual space;

determining structural and material properties of at least one object, based on the 3D scanned data;
displaying the determined structural and material properties of the at least one object around the at least one object;
determining manipulation information for the at least one object in the containing apparatus based on positioning information and action information of a user being mapped to the virtual space, wherein the positioning information of the user is provided by positioning a location of the user;
determining feedback information for the user based at least in part on the manipulation information and/or at least one of the structural and material properties of the at least one object, wherein determining the feedback information based at least in part on the manipulation information comprises determining, according to the manipulation information, and a volume and a density value of the at least one object, a direction and an amount of a reaction force applied by the at least one object on the user;
providing the user with a perceptive feedback based on the feedback information, wherein providing the user with the perceptive feedback based on the feedback information comprises applying, according to the determined direction and amount of the reaction force, the reaction force to the user via a force/tactile feedback device worn by the user and a communication interface;
reconstructing at least a 3D image of the at least one object based on the determined manipulation information; and
presenting the at least one reconstructed object on the displayed 3D image.

2. The image data processing method of claim 1 , wherein the 3D scanned data comprise at least one of X-ray Computerized Tomography (X-CT) scanned data and multi-view X-ray scanned data.

3. The image data processing method of claim 1 , further comprising:
controlling a scanning device which generates the 3D scanned data to further scan an area where the at least one object is located, based on the determined manipulation information.

4. The image data processing method of claim 3 , wherein the scanning device is controlled to further scan the area where the at least one object is located, and wherein the further scan has an increased intensity or denser scanning interval.

5. An image data processing device for security inspection, comprising:
a communication interface, configured to receive three-dimensional (3D) scanned data of a containing apparatus in which objects are contained; and
a processor, configured to reconstruct, based on the received 3D scanned data, a 3D image of the containing apparatus and the objects contained in the containing apparatus for display in a virtual space,
wherein the communication interface is further configured to receive positioning information and action information of a user to be mapped to the virtual space, wherein the positioning information of the user is provided by positioning a location of the user,
wherein the processor is further configured to determine structural and material properties of at least one object, based on the 3D scanned data, determine manipulation information for the at least one object in the containing apparatus based on the received positioning information and action information to be mapped to the virtual space, determine feedback information for the user based at least in part on the manipulation information and/or at least one of the structural and material properties of the at least one object, provide the determined feedback information to a force/tactile feedback device worn by the user via the communication interface, and reconstruct at least a 3D image of the at least one object based on the determined manipulation information,
wherein the communication interface is further configured to transmit the determined structural and material properties to a display to be displayed around the at least one object,
wherein determining the feedback information for the user based at least in part on the manipulation information comprises determining, according to the manipulation information, and a volume and a density value of the at least one object, a direction and an amount of a reaction force applied by the at least one object on the user, and
wherein providing the determined feedback information to the force/tactile feedback device worn by the user via the communication interface comprises applying, according to the determined direction and amount of the reaction force, the reaction force to the user via the force/tactile feedback device worn by the user and the communication interface.

6. The image data processing device of claim 5 , wherein the 3D scanned data comprise at least one of X-ray Computerized Tomography (X-CT) scanned data and multi-view X-ray scanned data.

7. The image data processing device of claim 5 , wherein the processor is further configured to generate, based on the determined manipulation information, a control instruction to further scan an area where the at least one object is located by a scanning device which generates the 3D scanned data, and transmit the generated control instruction to the scanning device via the communication interface.

8. A security inspection system, comprising:
a three-dimensional (3D) scanning device, configured to 3D scan a containing apparatus in which objects are contained to generate 3D scanned data;
a location tracking device, configured to position a location of a user for providing positioning information of the user;
a motion tracking device, configured to determine a motion of the user for providing action information of the user;
an image data processing device comprising:
a communication interface, configured to receive three-dimensional (3D) scanned data of the containing apparatus in which objects are contained; and
a processor, configured to reconstruct, based on the received 3D scanned data, a 3D image of the containing apparatus and the objects contained in the containing apparatus for display in a virtual space,
wherein the communication interface is further configured to receive positioning information and action information of a user to be mapped to the virtual space,
wherein the processor is further configured to determine structural and material properties of at least one object, based on the 3D scanned data, determine manipulation information for the at least one object in the containing apparatus based on the received positioning information and action information to be mapped to the virtual space, determine feedback information for the user based at least in part on the manipulation information and/or at least one of structural and material properties of the at least one object, and reconstruct at least a 3D image of the at least one object based on the determined manipulation information,
wherein the communication interface is further configured to transmit the determined structural and material properties to a stereoscopic display to be displayed around the at least one object, and
wherein determining the feedback information based at least in part on the manipulation information comprises determining, according to the manipulation information, and a volume and a density value of the at least one object, a direction and an amount of a reaction force applied by the at least one object on the user;

a wearable force/tactile feedback device, configured to pr

Assignees

• Nuctech Co Ltd
• Univ Tsinghua

Inventors

• Zhao Ziran
• Gu Jianping
• Liu Bicheng
• Wang Qi
• Yi Xi

Classifications

• G06T7/0004 Primary

  Industrial image inspection · CPC title

• G01T1/00

  Measuring X-radiation, gamma radiation, corpuscular radiation, or cosmic radiation (G01T3/00, G01T5/00 take precedence) · CPC title

• G06T2207/10012

  Stereo images · CPC title

• G01P13/00

  Indicating or recording presence, absence, or direction, of movement (electric switches H01H; counting moving objects G06M7/00) · CPC title

• G06Q50/30

  Physics · mapped topic