Detection, Tracking and Length Estimation of Free-Swimming Fish with a Range-Gated Camera

Abstract

With a global decline in fish stocks and a growing demand for seafood, there is a need for non-intrusive technological systems that can improve automatic data collection for conservation efforts and economic efficiency in the fishing industry. Previous studies have utilized computer vision and machine learning techniques to analyse data captured by underwater cameras and extract information regarding the size and species of the fish present in the images. Underwater footage is often of poor quality, and challenging to analyse due to the high variability in the fish's shape and intensity. SINTEF has developed an innovative range-gated camera system capable of 3D estimation underwater. This thesis presents an algorithm which utilizes the depth map produced by this system to detect, track and estimate the lengths of the fish observed by the camera. Multiple segmentation techniques, based on global and local thresholding, edge detection and background subtraction, were applied to 93 hand-segmented frames and evaluated using four validation metrics. A highly customizable tracking algorithm based on a minimum-cost flow network was developed, capable of tracking the fish through occlusions and missed detections. Using a dataset with three known fish lengths, ways of alleviating length estimation errors due to inaccuracies in the depth map or the segmentation were explored. The tracker ensures that the length estimates can be refined using multiple observations of the same fish, increasing the likelihood of the fish being observed in an optimal position for length estimation. The final algorithm was able to accurately detect and track all fish appearing in a test set consisting of 501 frames with three free-swimming codfish, using Euclidean distance to link detections and the estimated velocity direction of the fish to resolve occlusions. The average percentage error in the length estimation was approximately 10%. These results are promising, but more data with known fish lengths is needed in order to thoroughly analyse the length estimation algorithm. The algorithm was applied to a dataset containing schooling fish with unknown lengths; in this case the fish were only partially tracked due to a high number of occlusions and the velocity directions of the fish being largely the same.