DanioSense: Automated High-Throughput Quantification of Zebrafish Larvae Group Movement

The capability to obtain detailed motility information of model organisms is fundamental to reveal their functional and social behavior characteristics. Zebrafish is a powerful vertebrate model organism. Despite recent success in the automatic quantification of adult zebrafish movement, it remains a laborious task for group zebrafish larval tracking due to their similar appearance, frequent occlusions, and highly discontinuous kinematics. This article presents DanioSense (DS), an automatic tracker for group larval zebrafish, to overcome these tracking challenges. The integration of a light convolutional neural network and a centerline extraction algorithm enables the tracker to localize individuals even in occlusion cases where objects' identities are prone to switch. With reliable detections, an adaptive Kalman filter is designed to optimally estimate locomotive parameters, which is also used for object reidentification accomplished by a two-stage data association protocol. Experimental results demonstrated a tracking accuracy of over 97%, median errors of $102~{\mathrm{\mu m}}$ , and 8.8° for the position and orientation measurement, and a processing speed of over 30 frames/s with a normal computer configuration. DS provides detailed quantitative data for a large-scale larvae group in nearly real time, highly boosting the efficiency of characterizing individual phenotypes and analyzing social interactions. Note to Practitioners - This article aimed to tackle the problem of automated tracking groups of zebrafish larvae, an ideal vertebrate model organism for large-scale chemical and genetic screens. The task of group tracking is to record each individual's movement and calculate their position, velocity, direction, and other parameters for further analysis, where the correct identity of each individual must be maintained. Existing algorithms either switch larvae' identities easily or are unable to achieve online tracking due to the limitations of their methods to address individuals' intersections. DanioSense (DS) adopts a convolutional neural network to identify larval heads whenever they intersect and uses an adaptive Kalman filter to calculate the movement parameters optimally. Besides, a range of visualization options is designed to bring insight into underlying patterns through massive amounts of data. Theoretically, this algorithm's approach to solving intersections and calculating movement statistics can also apply to other fish-like animals. Its visualization options are applicable to other tracking systems. The key advantage of Daniosense over existing trackers is the capability to track each larva within a group and output detailed quantitative data in nearly real time. The tracking performance of DS is based on the quality of image segmentation and the success rate of classifying samples. Many state-of-the-art image segmentation and classification neural networks can be adopted to extend this system's applications to more complex environments but at a higher computation and time cost, which is a tradeoff between efficiency and capability. Some applications require a higher video sampling rate, so the system's processing speed needs to be further improved with better hardware and software framework optimization. The next steps include improving the processing efficiency, providing more tracking modules and visualization options, and extending its application fields.