Fish biomass, shape, and size determination

Patent metadata

Field	Value
Publication number	US-2022012479-A1
Application number	US-202117474893-A
Country	US
Kind code	A1
Filing date	Sep 14, 2021
Priority date	Jan 25, 2018
Publication date	Jan 13, 2022
Grant date	—

Abstract

Methods, systems, and apparatuses, including computer programs encoded on a computer-readable storage medium for estimating the shape, size, and mass of fish are described. A pair of stereo cameras may be utilized to obtain right and left images of fish in a defined area. The right and left images may be processed, enhanced, and combined. Object detection may be used to detect and track a fish in images. A pose estimator may be used to determine key points and features of the detected fish. Based on the key points, a three-dimensional (3-D) model of the fish is generated that provides an estimate of the size and shape of the fish. A regression model or neural network model can be applied to the 3-D model to determine a likely weight of the fish.

First claim

What is claimed is:

1 . (canceled)

2 . A computer-implemented method comprising:
determining, by a camera system, that an image that was generated by the camera system is of a fish that is at least partially occluded; determining, by the camera system, a respective position of each of one or more key points of the fish that are visible in the image; determining, by the camera system, a respective position of each of one or more key points of the fish that are occluded within the image based at least on the determined respective position of each of the one or more key points of the fish that are visible in the image; and generating, by the camera system, a weigh estimate of the fish that is at least partially occluded based on the determined respective position of each of the one or more key points of the fish that are occluded within the image; and providing, by the camera system, the weight estimate of the fish for output.

3 . The method of claim 2 , wherein the camera system is an underwater stereo camera system.

4 . The method of claim 2 , wherein the one or more key points of the fish that are occluded within the image include an eye, a nostril, or an operculum.

5 . The method of claim 2 , wherein the respective position of each of the one or more key points of the fish that are occluded within the image are determined using a probability model.

6 . The method of claim 2 , wherein the respective position of each of the one or more key points of the fish that are occluded within the image are determined using data that indicates likely positions of key points of the fish.

7 . The method of claim 2 , wherein generating the weight estimate comprises generating a 3D model of the fish based on the positions of the key points.

8 . The method of claim 2 , comprising ranking 3D models of the fish that are generated based on the positions of the key points.

9 . A system comprising:
one or more processing devices; one or more machine-readable storage devices for storing instructions that are executable by the one or more processing devices to perform operations comprising: determining, by a camera system, that an image that was generated by the camera system is of a fish that is at least partially occluded; determining, by the camera system, a respective position of each of one or more key points of the fish that are visible in the image; determining, by the camera system, a respective position of each of one or more key points of the fish that are occluded within the image based at least on the determined respective position of each of the one or more key points of the fish that are visible in the image; and generating, by the camera system, a weigh estimate of the fish that is at least partially occluded based on the determined respective position of each of the one or more key points of the fish that are occluded within the image; and providing, by the camera system, the weight estimate of the fish for output.

10 . The system of claim 9 , wherein the camera system is an underwater stereo camera system.

11 . The system of claim 9 , wherein the one or more key points of the fish that are occluded within the image include an eye, a nostril, or an operculum.

12 . The system of claim 9 , wherein the respective position of each of the one or more key points of the fish that are occluded within the image are determined using a probability model.

13 . The system of claim 9 , wherein the respective position of each of the one or more key points of the fish that are occluded within the image are determined using data that indicates likely positions of key points of the fish.

14 . The system of claim 9 , wherein generating the weight estimate comprises generating a 3D model of the fish based on the positions of the key points.

15 . The system of claim 9 , wherein the operations comprise ranking 3D models of the fish that are generated based on the positions of the key points.

16 . A non-transitory machine-readable storage medium for storing instructions that are executable by the one or more processing devices to perform operations comprising:
determining, by a camera system, that an image that was generated by the camera system is of a fish that is at least partially occluded; determining, by the camera system, a respective position of each of one or more key points of the fish that are visible in the image; determining, by the camera system, a respective position of each of one or more key points of the fish that are occluded within the image based at least on the determined respective position of each of the one or more key points of the fish that are visible in the image; and generating, by the camera system, a weigh estimate of the fish that is at least partially occluded based on the determined respective position of each of the one or more key points of the fish that are occluded within the image; and providing, by the camera system, the weight estimate of the fish for output.

17 . The medium of claim 16 , wherein the camera system is an underwater stereo camera system.

18 . The medium of claim 16 , wherein the one or more key points of the fish that are occluded within the image include an eye, a nostril, or an operculum.

19 . The medium of claim 16 , wherein the respective position of each of the one or more key points of the fish that are occluded within the image are determined using a probability model.

20 . The medium of claim 16 , wherein the respective position of each of the one or more key points of the fish that are occluded within the image are determined using data that indicates likely positions of key points of the fish.

21 . The medium of claim 16 , wherein generating the weight estimate comprises generating a 3D model of the fish based on the positions of the key points.

Assignees

• X Dev Llc

Inventors

• James Barnaby John
• Rapoport Evan Douglas
• Messana Matthew
• Kimball Peter

Classifications

• G06V20/05

  Underwater scenes · CPC title

• G06N3/09

  Supervised learning · CPC title

• G06N3/0464

  Convolutional networks [CNN, ConvNet] · CPC title

• G06T2207/30168

  Image quality inspection · CPC title

• G06T7/593 Primary

  from stereo images · CPC title