Automated fish age reading and analysis of growth patterns using deep learning
In recent years, the use of artificial intelligence has increased massively in various disciplines. Can this technology also be applied in fisheries science? This question is to be clarified in a PhD project.
Fish stock assessment relies on accurate estimates of fish age to derive important trends and measurements such as growth and mortality rates. The most widely accepted method has its roots from dendrochronology where the age of a tree is obtained through the counting of rings on its trunk cross-section. In the same manner, fish age can be derived by isolating certain fish structures, particularly the otoliths (ear stones), where growth ring patterns similar to tree-rings are formed at regular intervals in response to changing seasons.
To illustrate the manual age reading process, Figure 1 shows an image of an otolith where the so-called annuli (annual growth rings) were annotated by an expert age reader and the age was estimated to be 5 years. To make the method more accurate and less prone to subjective errors, the age readers undergo years of training and workshops to learn various guidelines on how to correctly distinguish the annual growth rings which can differ from one species (or even stock) to another. The goal is to prevent over- and under-estimation of fish ages since this can cause detrimental effects on fish stock management.
Due to the subjective and logistic limitations of this conventional method, it is therefore timely to utilize the advances in the field of AI and to make use of various deep learning algorithms to provide objective estimates of fish age from otolith images.
Scientists, Fisheries management
This project will employ the so-called Convolutional Neural Networks (CNNs), which are proven to be effective in handling image datasets. Two different methods are to be explored, corresponding to the two proposed ways of viewing the problem:
Firstly, fish age estimation can be treated as a semantic segmentation problem where the otolith image is partitioned into separate distinct zones namely the core, the winter rings and the summer rings (see Fig. 1). To efficiently perform this, the so-called U-Net algorithm is employed which is a CNN method popularly applied on biomedical image analysis (Ronneberger et al., 2015).
The title figure shows a sample prediction made by a preliminary U-Net model applied on one of the segregated test images. It can be seen that it is able to segment the otolith into the different zones and hence, can derive age estimates through the segmentation counts.
Secondly, another way of looking at the problem is to consider it as an object detection problem where the annual growth rings are treated as objects of interest to be detected at certain reading axes similar to the standard practice employed by manual age readers. For this method, a different algorithm is used which is called Mask R-CNN. It is a type of region-based CNN algorithm where multiple networks are utilized to simultaneously predict the object's class, location and segmentation (He et al., 2017).
The figure below shows the resulting automated annotations on a test image when a preliminary Mask R-CNN model is applied. At this stage, it is already able to pinpoint even those annual zones situated very close to each other. Also, certain scores are given where it tells us how confident it is that a particular object detected is indeed a proper annual growth zone.
Various experiments are still being done with the use of the two proposed deep learning algorithms for automated fish age reading. Moreover, this project also aims at creating a new method that combines both algorithms in order to obtain better results and even derive other biological variables in addition to fish age.
Nevertheless, with these preliminary results, the capabilities of the algorithms are beginning to be highlighted. Even at this early stage, it is already becoming apparent how the use of deep learning can be a promising leap forward into the future of fish age reading.
He K., Gkioxari G., Dollár P. and Girshick R. 2017. Mask R-CNN. IEEE International Conference on Computer Vision (ICCV), Venice, 2017, pp. 2980-2988, doi: 10.1109/ICCV.2017.322.
Ronneberger O., Fischer P., Brox T. 2015. U-Net: Convolutional Networks for Biomedical Image Segmentation. In: Navab N., Hornegger J., Wells W., Frangi A. (eds) Medical Image Computing and Computer-Assisted Intervention – MICCAI 2015. MICCAI 2015. Lecture Notes in Computer Science, vol 9351. Springer, Cham. doi.org/10.1007/978-3-319-24574-4_28
3.2021 - 2.2024
Project status: ongoing