Sensing system, sensing data acquisition method, and control device

Patent metadata

Field	Value
Publication number	US-12147249-B2
Application number	US-202017595339-A
Country	US
Kind code	B2
Filing date	Nov 27, 2020
Priority date	Nov 27, 2020
Publication date	Nov 19, 2024
Grant date	Nov 19, 2024

Abstract

The sensing system S extracts a target area to be subjected to short-distance sensing by the UGV 2 on the basis of the long-distance sensing data obtained by the UAV 1 in the air performing long-distance sensing on a lower place, perform movement control for moving the UGV 2 toward the target area. And then, the sensing system S acquires short-distance sensing data obtained by performing short-distance sensing on the whole or a part of the target area by the UGV 2 that has moved according to the movement control.

First claim

The invention claimed is:

1. A sensing system comprising:
at least one memory configured to store program code; and
at least one processor configured to access the program code and operate as instructed by the program code, the program code including:
first acquisition code configured to cause the at least one processor to acquire first sensing data obtained by a flying object in the air sensing a lower place;
extraction code configured to cause the at least one processor to extract a target area sensed by a traveling object capable of traveling on the ground on the basis of the first sensing data;
control code configured to cause the at least one processor to perform movement control for moving the traveling object toward the target area; and
second acquisition code configured to cause the at least one processor to acquire second sensing data obtained by sensing a whole or a part of the target area by the traveling object that has moved according to the movement control,
wherein the extraction code is configured to cause the at least one processor to extract, as the target area, an area where a difference from a predetermined appropriate temperature is equal to or larger than a threshold value in a thermal image included in the first sensing data.

2. The sensing system according to claim 1 , the program code further including display control code configured to cause the at least one processor to cause a display terminal to display information regarding the target area on the basis of the second sensing data.

3. The sensing system according to claim 1 , wherein the extraction code causes the at least one processor to extract, as the target area, an area having a high probability that an abnormality has occurred in the lower place, on the basis of the first sensing data.

4. The sensing system according to claim 3 , wherein the abnormality is an abnormality of any one of a plant, soil, and a road.

5. The sensing system according to claim 4 , the program code further including transmission code configured to cause the at least one processor to transmit the second sensing data to a terminal owned by a manager of any one of the plant, the soil, and the road.

6. The sensing system according to claim 1 , wherein the first sensing data includes a vegetation activity image, and
the extraction code causes the at least one processor to extract the target area on the basis of an activity in the vegetation activity image.

7. The sensing system according to claim 1 , wherein the second sensing data includes data related to at least one of moisture content, temperature, salinity concentration, electrical conductivity, and acidity in the ground of the target area.

8. The sensing system according to claim 1 , wherein the second sensing data is acquired using a sensor inserted in the ground of the target area.

9. The sensing system according to claim 1 , the program code further including determination code configured to cause the at least one processor to determine whether or not an abnormality has occurred in the target area on the basis of the second sensing data.

10. The sensing system according to claim 1 , wherein the control code causes the at least one processor to cause the traveling object to continuously perform sensing of the target area a plurality of times while moving the traveling object so that data on the entire target area is included in the second sensing data.

11. The sensing system according to claim 1 , the program code further including third acquisition code configured to cause the at least one processor to acquire third sensing data obtained by sensing the ground by the traveling object before the traveling object is moved toward the target area,
wherein the control code causes the at least one processor to perform movement control for moving the traveling object toward the target area on the basis of the first sensing data and the third sensing data.

12. The sensing system according to claim 11 , wherein the control code causes the at least one processor to perform movement control for moving the traveling object toward the target area detected from the third sensing data by matching the first sensing data with the third sensing data.

13. The sensing system according to claim 1 , wherein when the target area is extracted, the control code causes the at least one processor to move the traveling object into a range of the sensing performed by the flying object in a hovering state, and thereafter, performs movement control for moving the traveling object toward the target area on the basis of the first sensing data.

14. The sensing system according to claim 13 , wherein
the flying object performs the sensing while moving along a predetermined flight route, and
when the target area is extracted, the control code causes the at least one processor to cause the flying object to interrupt the movement along the route and then shift to the hovering state.

15. The sensing system according to claim 1 , wherein a distance between the target area and the flying object is longer than a distance between the target area and the traveling object.

16. The sensing system according to claim 1 , wherein a range of the sensing performed by the flying object is larger than a range of the sensing performed by the traveling object.

17. The sensing system of claim 1 , wherein the extraction code is configured to cause the at least one processor to extract, as the target area, an area where a difference from a predetermined appropriate plant color is equal to or larger than a threshold value in an RGB image included in the first sensing data.

18. A sensing data acquisition method including:
acquiring first sensing data obtained by a flying object in the air sensing a lower place;
extracting a target area sensed by a traveling object capable of traveling on the ground on the basis of the first sensing data;
performing movement control for moving the traveling object toward the target area; and
acquiring second sensing data obtained by sensing a whole or a part of the target area by the traveling object that has moved according to the movement control,
wherein the extracting comprises extracting, as the target area, an area where a difference from a predetermined appropriate temperature is equal to or larger than a threshold value in a thermal image included in the first sensing data.

19. A control device comprising:
at least one memory configured to store program code; and
at least one processor configured to access the program code and operate as instructed by the program code, the program code including:
first acquisition code configured to cause the at least one processor acquire first sensing data obtained by a flying object in the air sensing a lower place;
extraction code configured to cause the at least one processor to extract a target area sensed by a traveling object capable of traveling on the ground on the basis of the first sensing data;
control code configured to cause the at least one processor to perform movement control for moving the traveling object toward the target area; and
second acquisition code configured to cause the at least one processor to acquire second sensing data obtained by sensing a whole or a part of the target area by the traveling object that has moved according to the movement control,
wherein the extraction code is configured to cause the at least one processor to extract, as the target area, an area where a difference from a predetermined appropriate temperature is equal to or larger than a thre

Assignees

• Rakuten Group Inc

Inventors

• Nakazawa Mitsuru
• Takizawa Jun

Classifications

• G05D1/683

  Intercepting moving targets (docking at a base station G05D1/661) · CPC title

• G05D1/46

  Control of position or course in three dimensions [3D] · CPC title

• G05D1/249

  from positioning sensors located off-board the vehicle, e.g. from cameras · CPC title

• G05D1/225

  operated by off-board computers · CPC title

• G06T2207/30188

  Vegetation; Agriculture · CPC title