Robotic mower boundary detection system

Patent metadata

Field	Value
Publication number	US-11480973-B2
Application number	US-201916511511-A
Country	US
Kind code	B2
Filing date	Jul 15, 2019
Priority date	Jul 15, 2019
Publication date	Oct 25, 2022
Grant date	Oct 25, 2022

Abstract

A computer-implemented method includes obtaining image data representing a set of images of a worksite captured by an image capture component of a mobile computing device, identifying a set of virtual markers associated with the set of images, each virtual marker having a corresponding position in one of the images, and determining, for each virtual marker, a set of coordinates in a coordinate system based on the corresponding position of the virtual marker. Based on the set of coordinates, boundary data is generated that represents a boundary on the worksite. The boundary data is communicated to a robotic mower for control of the robotic mower within an operating area defined based on the boundary.

First claim

What is claimed is:

1. A method comprising:
obtaining image data representing a set of images of a worksite captured by an image capture component of a mobile computing device that is separate from a robotic mower;
receiving sensor data generated by one or more sensors on the mobile computing device, the sensor data corresponding to the image data and representing a geographic position of the mobile computing device on the worksite and an orientation of the mobile computing device at the geographic position;
identifying a set of virtual markers associated with the set of images, each virtual marker having a corresponding position in a corresponding image in the set of images;
determining, for each virtual marker, a set of coordinates in a coordinate system based on the sensor data and the corresponding position of the virtual marker in the corresponding image;
based on the set of coordinates, generating boundary data that represents a boundary on the worksite; and
wirelessly communicating, to the robotic mower, the boundary data for control of the robotic mower within an operating area defined based on the boundary.

2. The method of claim 1 , wherein determining a set of coordinates for each virtual marker comprises:
translating the corresponding position of the virtual marker into the set of coordinates.

3. The method of claim 2 , wherein the set of coordinates comprises local coordinates in a local coordinate system, and further comprising:
transforming the local coordinates into global coordinates in a global coordinate system.

4. The method of claim 3 , wherein transforming comprises transforming each local coordinate into a corresponding global coordinate based on global coordinates associated with one or more reference points identified on the worksite.

5. The method of claim 3 , wherein the one or more reference points comprises at least two reference points each having corresponding global coordinates defined by Real-Time Kinematic (RTK) position enhancement of Global Positioning System (GPS) data.

6. The method of claim 1 , wherein the set of coordinates are determined by performing visual odometry to track movement of the mobile computing device relative to the worksite.

7. The method of claim 2 , wherein the visual odometry comprises concurrent odometry and mapping (COM).

8. The method of claim 1 , and further comprising:
determining the orientation based on a sensor signal from at least one of:
an accelerometer on the mobile computing device, or
a gyroscope on the mobile computing device; and

determining the geographic position based on a sensor signal from a Global Navigation Satellite System (GNSS) receiver on the mobile computing device.

9. The method of claim 1 , and further comprising:
generating, on the mobile computing device, a user interface display that displays a field of view of the image capture component and includes a virtual marker user input mechanism actuatable to generate a virtual marker on a current image displayed in the user interface display; and
generating the set of virtual markers based on user actuation of the virtual marker user input mechanism.

10. The method of claim 9 , and further comprising:
generating a virtual marker modification user input mechanism actuatable to modify one or more of the virtual markers.

11. The method of claim 1 , and further comprising:
obtaining a particular image in the set of images;
receiving a virtual marker input associated with the particular image; and
based on the virtual marker input, selecting a position in the particular image and creating a virtual marker corresponding to the selected position in the particular image.

12. The method of claim 11 , wherein the receiving the virtual marker input comprises receiving the virtual marker input through the mobile computing device.

13. A computing system comprising:
image acquisition logic configured to obtain image data representing a set of images of a worksite captured by an image capture component of a mobile computing device that is separate from a robotic mower;
virtual marker identification logic configured to identify a set of virtual markers associated with the set of images, each virtual marker having a corresponding position in a corresponding image in the set of images;
coordinate generation logic configured to determine, for each virtual marker, a set of coordinates in a coordinate system based on sensor data and the corresponding position of the virtual marker in the corresponding image, wherein the sensor data is generated by one or more sensors on the mobile computing device, corresponds to the image data, and represents a geographic position of the mobile computing device on the worksite and an orientation of the mobile computing device at the geographic position;
boundary generator logic configured to generate, based on the set of coordinates, boundary data that represents a boundary on the worksite; and
a communication system configured to wirelessly communicate, to the robotic mower, the boundary data for control of the robotic mower within an operating area defined based on the boundary.

14. The computing system of claim 13 , wherein the set of coordinates are determined by performing, visual odometry to track movement of the mobile computing device relative to the worksite.

15. The computing system of claim 13 , wherein
the one or more sensors comprises at least one of an accelerometer, a gyroscope, or a Global Navigation Satellite System (GNSS) receiver.

16. The computing system of claim 13 , wherein the set of coordinates comprises local coordinates in a local coordinate system, and further comprising:
coordinate translation logic configured to:
transform the local coordinates into global coordinates in a global coordinate system based on global coordinates associated with one or more reference points identified on the worksite.

17. The computing system of claim 16 , and further comprising:
user interface logic configured to:
generate a user interface display that displays a field of view of the image capture component and includes a virtual marker user input mechanism actuatable to generate a virtual marker on a current image displayed in the user interface display; and
generate the set of virtual markers based on user actuation of the virtual marker user input mechanism.

18. The computing system of claim 13 , and further comprising:
virtual marker generator logic configured to:
receive a virtual marker input associated with a particular image in the set of images; and
based on the virtual marker input, select a position in the particular image and create a virtual marker corresponding to the selected position in the particular image.

19. The computing system of claim 18 , wherein the virtual marker input is received through the mobile computing device.

20. A mobile computing device comprising:
a display device;
an image capture component configured to capture an image of a worksite;
one or more sensors configured to generate sensor data representing an orientation of the image capture component relative to the worksite;
a communication system configured to wirelessly communicate with a robotic mower;
at least one processor; and
memory storing instructions executable by the at least one processor, wherein the instructions, when executed, provide:
an augmented reality virtual boundary generation system configured to:
generate a user interface display on the display device that displays a field of view of the

Assignees

• Deere & Co

Inventors

• Ackerman Chad A
• Ogden Nathan
• Sprabery Brittany
• Heltsley Blake
• Feltes Patrick
• Trask Aaron
• Moran Mark D

Classifications

• A01D2101/00

  Lawn-mowers · CPC title

• A01D34/008

  for automated or remotely controlled operation · CPC title

• G05D1/0278

  using satellite positioning signals, e.g. GPS · CPC title

• G05D1/0268

  using internal positioning means · CPC title

• G05D2201/0208

  Physics · mapped topic