Deep Learning for Laying Hen Activity Recognition Using Wearable Sensors

Abstract

Laying hen activities in modern intensive housing systems can dramatically influence the policies needed for the optimal management of such systems. Intermittent monitoring of different behaviors during daytime cannot provide a good overview, since daily behaviors are not equally distributed over the day. This paper investigates the application of deep learning technology in the automatic recognition of laying hen behaviors equipped with body-worn inertial measurement unit (IMU) modules in poultry systems. Motivated by the human activity recognition literature, a sophisticated preprocessing method is tailored on the time-series data of IMU, transforming it into the form of so-called activity images to be recognized by the deep learning models. The diverse range of behaviors a laying hen can exhibit are categorized into three classes: low-, medium-, and high-intensity activities, and various recognition models are trained to recognize these behaviors in real-time. Several ablation studies are conducted to assess the efficacy and robustness of the developed models against variations and limitations common for an in situ practical implementation. Overall, the best trained model on the full-feature acquired data achieves a mean accuracy of almost 100%, where the whole process of inference by the model takes less than 30 milliseconds. The results suggest that the application of deep learning technology for activity recognition of individual hens has the potential to accurately aid successful management of modern poultry systems.

1. Introduction

Today’s livestock industry faces important challenges in terms of monitoring, control, economics, etc. Recognizing animals’ behaviors plays an important role in addressing most of these challenges, since animals’ different activities in modern intensive housing systems can dramatically influence the required management systems [1,2]. For an effective management of such systems, the monitoring task should be performed continuously during the daytime. Such an intensive monitoring task is no longer feasible to perform manually by humans, as it is tedious, time-consuming, and also no longer cost-effective. Therefore, a sort of automation is crucial for monitoring systems of today’s livestock production, as well as moving towards precision livestock farming [3,4,5,6,7,8].

There are several reasons supporting the need for development of automatic activity recognition models for hen housing systems, most notably: (i) A significant source of fine dust emissions comes from the agricultural industry. Animal matters, such as feces, hairs, and feathers, are among important contributors in this regard [9,10,11,12,13]. An effective recognition model could detect high-intensity activities of the hens, and proper control policies on illumination, feeding, humidity, and other possible inputs can be implemented accordingly; (ii) Detecting and preventing abnormal behaviors of hens could decrease the rate of injuries among them, since intrusive precautions such as beak trimming are being or have been banned in most developed countries [14,15]. It could also lead to the early detection of animal illnesses, when the behaviors of an individual hen or a flock are monitored day-to-day.

Generally, two different approaches can be taken to perceive different behaviors of hens. Remote sensing through computer vision is one option that provides rich data from the scene. For example, a deep convolutional neural network is trained in [16] to detect and segment hens with the goal to assess the plumage condition. A vision-based posture analysis is developed in [17] to detect sick broilers and give early warnings. Although getting closer to being operational in low intensive environments, it is truly difficult to use it for tracking individual hens in today’s loose housing systems [18]. The visual similarities between hens, the large domain of movements provided in these new housing systems, and occlusion incidences that could happen quite frequently hinder the effective application of this solution. Moreover, recognition of the moderate- and high-intensity activities, accomplished in a short period of time, requires high-frequency data sampling, which is not affordably available through vision sensing. Nevertheless, there are good opportunities for the study of flock behaviors using long-shot imagery, which is not the focus of the present work.

The other option, which is more suited for capturing the behavior of individual subjects, is to make use of wearable sensors that can be safely attached to a target limb of hens. Inertial measurement unit (IMU) modules are commonly used for motion capturing in various domains including human and animal activity recognition [19,20,21]. Whereas, other local positioning systems, such as the RFID technology that could help in positioning tagged subjects within a covered area by antenna receivers, have not received similar attention mostly due to their expensive and time-consuming setups (see [22] for a recent review). The use of IMU sensors for motion capturing and activity recognition has seen a breakthrough after the emergence of MEMS-based modules, which are low-cost, lightweight, easy-to-log, and low-power devices. The attachment of such sensors on animals such as hens does not introduce significant restrictions, and the subject can hopefully behave seamlessly [23]. IMU can provide data about the translational and angular movements of the subject in 3D, which can then be analyzed to infer high-level information about the type of activities being performed [19,21].

Real-time sensor-based hen activity recognition models dealing with the time-series IMU signals typically make use of machine learning technology. Machine learning provides a framework for large datasets to extract hidden features and patterns of signals that are otherwise impossible to be identified by manual or physics-based analyses [24]. A heavy preprocessing practice is required before machine learning-based classifiers can digest the time-series sensory data. Generally, this is accomplished by handcrafting a limited number of features, typically of a statistical flavor, then extracting these features from the segmented raw data, and finally allowing machine learning techniques to learn the correlations between these features and the output.

Based on the intensity of behaviors, Kozak et al. [25] identified three super classes a typical hen can perform: low-, medium-, and high-intensity activities. Accordingly, in [26] we presented a machine learning framework for recognizing these classes using 9-axis IMU modules worn by different hen subjects. A custom dataset was established for this study, and each data sample was manually labeled with the aid of the video streams recorded for this purpose. From the segments of raw data, skewness, kurtosis, mean, standard deviation, variance, minimum, maximum, entropy, energy, and covariance were manually calculated as representative features, and a number of standard machine learning techniques including support vector machine (SVM), and Bagged Trees were trained while the influences of different parameters and settings were thoroughly analyzed. The best predictor model, namely the Bagged Trees, achieved 89% accuracy for the classification.

In a binary classification task, Hepworth et al. [27] showed that applying a SVM model to routinely collected bird farm management data enables the prediction of sickness status with a superior accuracy of up to 99.5%. The activity recognition of hens in a non-cage maintenance system was studied in [28], where wireless accelerometers were installed on the subjects for real-time activity monitoring. The entropy and mean were extracted from the segments of recorded data, and several machine learning models including the Decision Tree and Radial Basis Function were employed to classify the static (sit/sleep), dynamic (walk/run), and resource (feed/drink) activities of the subjects. Accelerometers were also examined in classifying cow behaviors into three classes: standing, walking, and lying down [29]. If further research supports behavioral monitoring to identify animal health status, accelerometers could be used for larger populations of animals at risk for diseases [29].

Apart from the reviewed literature that focused on the training of classical machine learning algorithms using handcrafted features of data, a recent body of literature has studied the autonomous feature learning capabilities of Deep Learning (DL) technology. This paradigm has received particular attention in the field of human activity recognition (see [30] for a recent review). The efficacy of the methods in this scheme has further been enhanced by the rapid advancements in the hardware (i.e., lightweight sensors for large data acquisition, and powerful processors for deep networks training), and software (advanced framework for optimization and parallel computations). Motivated by the outstanding performance of DL models in image classification and object detection, a number of studies have investigated the possibility of encoding IMU signals into a form of images that can readily be recognized by the well-known DL-based models [19,21,31,32,33,34]. This encoding technique is sometimes referred to as the signal imaging.

The primary objective of the present study is to adopt the signal imaging and deep feature learning techniques to the problem of individual hen activity recognition. We would like to explore to what extent the introduced methodology can improve the results obtained by taking a traditional machine learning approach in our earlier study [26]. As a second objective, this study benchmarks the influence of different setups, parameters, and properties pertaining to the data collection and preprocessing, model architecture, and model training process. The results of this benchmark could aid further studies in examining different possibilities through remote/wearable sensing to move from individual hen analysis to a large-scale practical implementation being useful in optimally managing the future’s livestock systems.

The reset of this paper is organized as follows. In Section 2, we start by presenting the pipeline of the work, followed by the description of the dataset establishment. Next, the sophisticated data preprocessing presented in this paper is carefully detailed, and deep learning recognition architectures and the training process are described. Section 3 is devoted to the presentation of results and discussion. Here, the proposed model is investigated in terms of efficacy, robustness, and generalizability. Finally, the paper is concluded in Section 4.

2. Materials and Methods

2.1. System Overview

As outlined in the introduction, the diverse types of hen behaviors can be categorized into three super classes: low-intensity, moderate-intensity, and high-intensity activities. Figure 1 depicts an overview of the behavior recognition system developed to automatically identify these three groups of activities. The process starts by acquiring movement data from the wearable sensor the subject hens are equipped with (Section 2.2). The movement data includes the linear acceleration, angular velocity, and the attitude determined from the magnetic field data. The acquired sensory signals are then sampled through a specific time windowing step (Section 2.3.1), and then transformed into a set of so-called signal images following an encoding algorithm described in Section 2.3.2. The signal images are further transformed into activity images applying Discrete Fourier Transform (DFT). The resulting images serve as the input to a deep neural network (DNN) performing the actual classification task (Section 2.4 and Section 2.5). The rest of this section is devoted to a detailed presentation of all materials and methods involved in the introduced system.

2.2. Data Acquisition, Labeling, and Bundling

Data plays a central role in the success of any deep learning model. Here, we describe how the devised experiments are performed to acquire data, the way by which the time-series data are labeled, and in what combinations they can be bundled for the next steps.

2.2.1. Data Acquisition

An experiment was designed in a part of a commercial laying hen poultry farm to collect data on the behavior of laying hens [26]. The experimental environment was separated from the other parts of the poultry house with a fence. In terms of facilities, this environment was similar to other parts of the poultry farm. The hen house lights were on at 4:30 a.m. and off at 7:45 p.m. Fifteen laying hens that were 34 weeks old were placed in the experimental environment. Two comfortable backpacks to contain a light IMU module of 16 grams were installed on two individual hens for data collection. The backpack attachment is inspired by a common back support belt. The first two days of the experiments were only meant for hens to get used to the new situations [35], and the actual data acquisition was started from the third day.

The MTw2 Awinda Wireless 3DOF Motion Tracker (IMU) from Xsens was used for inertial measurement. The module houses an accelerometer, a gyroscope, and a magnetometer whose specifications can be consulted in

Table 1. Technical properties of MTw2 Awinda Wireless 3DOF Motion Tracker (www.xsens.com, accessed on 12 December 2022).

Parameter	Angular Velocity	Acceleration	Magnetic Field
Dimensions	Three axes	3 axes	3 axes
Full scale	2000 deg/s	160 m/s${}^2$	1.9 Gauss
Non-linearity	0.1% of FS	0.5% of FS	0.1% of Fs
Bias stability	10 deg/h	0.1	-
Noise	0.01 deg/s/$\sqrt{HZ}$	0.01 $\mathsf{\mu }$g/$\sqrt{HZ}$	02 mGauss/$\sqrt{HZ}$
Alignment error	0.1 deg	0.1 deg	0.1 deg
Bandwidth	180 Hz	180 Hz	10–60 Hz (var.)

. It contains a built-in battery providing sufficient power for the course of experiments, and the logged data are saved in an internal memory. Please refer to [26] for more details on data acquisition.

2.2.2. Data Labeling

To distinguish the hens from each other, two distinct colors are used for the backpacks, namely green and blue. Accordingly, we shall refer to the subjects as blue hen and green hen, as shown in Figure 2. The green hen data includes 2 h and 20 min of wearable sensor log at a rate of 1000 Hz, whereas the blue hen data includes only 29 min at the same sampling rate. For the sake of labeling, video recording was also performed and synced by the movement signals. An in-house expert has carefully processed recorded sequences frame by frame and labeled the data accordingly. As outlined earlier, only three super classes are considered for labeling, which include different types of low-, moderate-, and high-intensity activities listed in

Table 2. Classification of physical activity of laying hens based on their intensity [26].

Low-Intensity	Moderate-Intensity	High-Intensity
Sleep-like resting	Preening	Walking
Neck shortening resting	Foraging & pecking	Running
Sleeping	Drinking & eating	Jumping
Quiet sitting/standing	Small wing adjustments	Wing flapping
Small postural	Scratching & stretching	Controlled aerial ascent/descent
head/shoulder/neck movements	Head shaking	Full-body shaking
Perching	Feather fluffing	Shaking phase of dust bathing
Egg laying	Searching behavior
Side-laying phase of dust bathing

. The distribution of data among the three classes is shown in Figure 3. Note that the dataset is imbalanced, since the high-intensity (Class 3) samples are much fewer than the other two classes. Additionally, the green hen dataset is considerably larger than the blue one.

2.2.3. Data Bundling

The deployed 9-axis Xsens IMU generates rich movement data including three-dimensional linear acceleration, angular velocity, and magnetic field data, symbolized here by “A”, “V”, and “M”, respectively. One can argue that an efficient training of the activity recognition model may not need all these axes of measurements. For example, if the recognition model trained by only linear acceleration data is proven to be effective, this simply means that practically less expensive wearable sensors and computational power are demanded. Therefore, it is justified to study the different combinations of sensor outputs, which are hereon referred to as the data bundles. Four different data bundles are considered in our experiments, as listed in

Table 3. Data bundles separately considered for the training of activity recognition models in this study.

Data Bundle	Axes	Signal Image Size
AVM	Acceleration—Velocity—Magnetic field	$36\times 50$
AV	Acceleration—Velocity	$14\times 50$
AM	Acceleration—Magnetic field	$14\times 50$
A	Acceleration	$10\times 50$

. Notice the presence of linear acceleration (A) in all bundles, since it naturally provides the most informative movement data for a hen. Additionally, the size of signal images corresponding to each data bundle, reported in the last column of the table, is determined by the time windowing and signal imaging techniques to be presented in the next section.

2.3. Data Preprocessing

2.3.1. Time Windowing

A form of sampling is usually needed for time-series signals to be digested by DNNs. This is mainly because the durations of different activities, embodied through special patterns in the signals, are substantially different, while DNNs prefer to receive constant-size inputs. As such, a time windowing technique is applied in which a sliding window of a specific size spans the signals, yielding sampled (segmented) pieces of data. As illustrated in Figure 4, an overlap can be considered between every two adjacent segments to ensure a smooth transition. Unless otherwise specified, a sliding window of 50 data samples and 50% overlap are considered in this study. Figure 5 shows how this windowing technique affects the size of data for the green hen. It is easy to show that letting x be the number of samples in a class, the number of data segments after windowing will be $\frac{x}{25}$. The width of DNN inputs is determined by the windowing size.

2.3.2. Signal Imaging

One more step is required for the sampled (segmented) data to be transformed into a suitable form for the DNN recognition model. As outlined in the introduction, the traditional way is the handcrafted feature extraction, which is studied in our previous work [26]. Although proven to be effective, it suffers from a few shortcomings. Most notably, it features a form of abstraction in the sense that not all distinctive features embodied in the raw data are necessarily captured by the handcrafted features. Moreover, there is no guarantee that the extracted features are the best representatives of the data. As briefly introduced earlier, an alternative which has been proven to be effective in closely related applications is the signal imaging technique. The encoded time-series data into signal images form two-dimensional arrays showing features and patterns that are hardly found in the original one-dimensional signals [36]. Various encoding techniques, e.g., the Gramian Angular Field [37,38] and the Markov Transition Field [38], have been investigated in the literature. The signal imaging technique used in the present work is adopted from [19], the method that has demonstrated outstanding performance specifically in the realm of human activity recognition. We refer to this method as the Signal Imaging Algorithm (SIA).

Following the SIA proposed in [19], for the AVM data bundle (Table 3) the size of resulting signal image is $36\times 50$ (see Figure 6), in which the number of columns corresponds to the sliding window width, that is 50 data samples. The 36 rows of the image are formed by stacking the nine signal channels in an order defined by the algorithm so that every channel is repeated four times and there is a chance for every channel to be adjacent to every other channel. This order is systematically generated by the algorithm. For the particular case of nine channel signals, the order would be “123456789135792468147158259369483726”, where each digit corresponds to a specific channel in the windowed signal. Similarly, for the AV and AM data bundles (Table 3), since the input signal has six channels, the resulting signal images would have 14 rows, as shown in Figure 6. In this case, the generated order by the algorithm would be “12345613514624”.

For the A data bundle, which includes only the three-channel linear accelerations, the SIA is no longer applicable. Here, we introduce a duplicate method resulting in $10\times 50$ signal images, in which the 10 rows are formed by stacking the three channels in the order “1231231”. Note that the obtained height for the images of this data bundle is the minimum height acceptable by the implemented DNNs, as will be discussed later. Finally, all signal images are converted to activity images of the same size applying the two-dimensional DFT. It was revealed during the preliminary experiments that applying the DFT would result in more discriminative features.

2.4. Behavior Recognition Model

As outlined earlier, the main objective of the present study is to investigate the power of Deep Learning (DL) classifiers in recognizing different behaviors of laying hens represented by activity images. In the last decade, DL has emerged as one of the best performing predictor classes within machine learning in various implementation areas [39]. Ever-increasing processors capabilities, decreasing computational costs, and advancing learning algorithms promote DL [40]. In particular, Deep Convolutional Neural Networks (DCNNs) have demonstrated outstanding performance in image classification, object detection, and tracking. Numerous successful implementations have revolved around breakthrough DCNN architectures such as LeNet [41], VGG [42], AlexNet [43], ResNet [44], and MobileNet [45]. We will examine a few of these architectures as well as a simple shallow CNN model in the experiments.

The LeNet5 architecture with modified input and layer sizes is illustrated in Figure 7 for the AVM data bundle (see Table 3). The network accepts the activity images as input and gives the probability of occurrence of the three behavior classes as output. The activity images are normalized in the interval $[0,255]$ before feeding to the network. The first two pairs of layers apply convolution and average pooling sequentially, extracting deep features of the images such as key patterns. A flatten layer then vectorizes the feature map, making it ready for densification by three consecutive fully-connected layers, which output the probabilities of each class occurrence.

Table 4. Adopted LeNet5 properties.

Layer Index	Type	Filter Size	Activation	No. of Filters
1	Convolution	$3\times 3$	Relu	6
2	Average Pooling	$2\times 2$	-	-
3	Convolution	$3\times 3$	Relu	16
4	Average Pooling	$2\times 2$	-	-
5	Flatten	-	-	-
6	Dense	-	Relu	-
7	Dense	-	Relu	-
8	Dense	-	SoftMax	-

reports the number and size of filters as well as the respective activation functions in each layer when applicable. Additionally, the obtained shapes of feature maps in each layer for different data bundles are given in

Table 5. Shapes of feature maps in each layer of LeNet5 model for different data bundles.

	Bundle	AVM	AV	AM	A
Layer
Input		(36 × 50 × 1)	(14 × 50 × 1)	(14 × 50 × 1)	(10 × 50 × 1)
Conv 1		(34 × 48 × 6)	(12 × 48 × 6)	(12 × 48 × 6)	(8 × 48 × 6)
Pooling 1		(17 × 24 × 6)	(6 × 24 × 6)	(6 × 24 × 6)	(4 × 24 × 6)
Conv 2		(15 × 22 × 16)	(4 × 22 × 16)	(4 × 22 × 16)	(2 × 22 × 16)
Pooling 2		(7 × 11× 16)	(2 × 11 × 16)	(2 × 11× 16)	(1 × 11 × 16)
Flatten		1232	352	352	176

.

We also consider two alternative network architectures; the first one is the well-known ResNet50 model [44], and the other is a simple shallow CNN network, or the shallow model for short. This network consists of two convolution layers, a fully-connected layer followed by a dense layer, which provides the output. The convolution layers include, respectively, 64 and 128 filters each having a ($4\times 4$) shape. As for the activation functions, Relu and SoftMax are used for the convolution and dense layers, respectively.

2.5. Network Training and Evaluation Metrics

All DNN models are programmed and trained in Python 3.8 using TensorFlow 2.9.1 [46] on a normal PC with an i7 9th Gen. Intel processor and 16 GB of RAM. The networks are trained using Adam optimization technique [47] while a categorical cross-entropy loss function [48] is considered. The evaluation policy is to shuffle the activity images randomly. Unless otherwise specified, 70% of the shuffled images are used for the model training while the rest are used for testing. Note that our preliminary experiments revealed that considering a further split between validation and test sets did not influence the results. As such, and since the size of samples in the high-intensity class is relatively small, we decided to follow this train-test policy and, therefore, the terms test and validation are assumed equivalent in this paper.

To illustrate the classification performance, we evaluate commonly used performance metrics illustrated in Figure 8. A true positive (TP) is an outcome where the model correctly predicts the true class. Similarly, a true negative (TN) is an outcome where the model correctly predicts the false class. A false positive (FP) is for the case where the model incorrectly predicts the true class, and a false negative (FN) means the model incorrectly predicts the false class. Accuracy describes the general performance across all classes. It is the ratio of the number of correct predictions to the total number of predictions. Recall calculates the model’s capability to detect the true samples, i.e., it computes the ratio of true samples correctly classified as true to the total number of true samples. Finally, precision measures the ratio between the number of correctly classified true samples to the total number of true classified samples. These performance metrics can take values in the range $[0,1]$ by definition.

3. Results and Discussion

The proposed activity recognition model is thoroughly investigated in terms of efficacy, robustness, and generalizability. First, only the green hen dataset is used in the analyses to provide a focused study on the influence of the following settings: (1) exploring different DCNN architectures; (2) reducing the bundle dimension and dataset size; and (3) introducing noise to the data to evaluate robustness. Then, both the blue and green hens’ datasets are combined to see how well the model can generalize.

3.1. Full-Feature Data Bundle Results

In order to illustrate the best performance achievable by the proposed framework, we train the three introduced models on the green hen activity images of the AVM data bundle, which includes its full-feature dataset. The training process for all models went smoothly, as shown in Figure 9, where the accuracy and loss during training are plotted. All accuracies converged to 1 while the corresponding training losses were ever decreasing. As for testing,

Table 6. Evaluation of performance indices for the considered recognition models when using the AVM data bundle of the green hen test data.

Performance Index	LeNet5	ResNet50	Shallow Model
Accuracy	1	0.9991	0.9998
Validation Loss	0.0001	0.0024	0.0003
Recall	$\left[\begin{array}{ccc}1& 1& 1\end{array}\right]$	$\left[\begin{array}{ccc}0.9985& 0.9995& 1\end{array}\right]$	$\left[\begin{array}{ccc}0.9997& 1& 1\end{array}\right]$
Precision	$\left[\begin{array}{ccc}1& 1& 1\end{array}\right]$	$\left[\begin{array}{ccc}1& 0.9996& 0.9250\end{array}\right]$	$\left[\begin{array}{ccc}1& 0.9998& 1\end{array}\right]$
Average elapsed time for generating an activity image	0.6 ms	0.6 ms	0.6 ms
Average elapsed time for a network inference	28.8 ms	64.82 ms	47.93 ms

reports the evaluation of performance indices presented earlier on the test set of the green hen. Additionally, the confusion matrices for all the three models are shown in Figure 10.

The results demonstrate absolutely accurate and precise classifications by all the three models. The perfect confusion matrix of LeNet5, shown in the left panel of Figure 10, implies that the model has succeeded in correctly predicting the intensity level of all 10,000 instances in the test set. Even for the confusion matrices of ResNet50 and shallow model, there are extremely few instances falling off-diagonal. Notice the accurate predictions of high-intensity instances by all three models, despite the small data size in this class. The best accuracy achieved in our previous study [26] was 89.4%. The significant improvement achieved in the present work demonstrates the effectiveness of the deep feature learning and the data preprocessing method through the implemented signal imaging technique. On average, generating an activity image by the signal imaging algorithm and its inference by the trained LeNet5 network on the processing unit introduced in Section 2.5 take 0.6 and 29 milliseconds, respectively, as reported in Table 6. This computation time is short enough for a real-time implementation, given that each activity image accounts for a 50 milliseconds data segment. Note that in terms of computational efficiency, again the LeNet5 model performs slightly better than the ResNet50 and shallow model under the same training policy and conditions.

Overall, the results suggest that the developed model is capable of recognizing different intensity activities of the subject quite accurately, and the computation time complies with a real-time implementation. The next few analyses will further investigate the performance of the model with respect to some important parameters and settings that could be crucial for a large-scale practical implementation. Without loss of generally and for brevity, hereon only the LeNet5 model will be used in the experiments.

3.2. Influence of Bundle Dimension and Trainset Size Reduction

Data collection for animal activity recognition can be cumbersome because of the strict regulations of dealing with animals as well as the lack of continuous control on the sensor proper attachment during the acquisition process. Data labeling can also become challenging in some situations since the subjects do not generally move in any programmable directions for video data recording. As such, it is worth exploring how the performance indices of our model vary with respect to the amount of training data. To this end, we investigate different test-to-train ratios as well as different data bundles introduced in Table 3. A test-to-train ratio of 30/70 implies that 70% of the data are used for training and 30% for testing, which is the default choice in this paper except for the discussion that follows here.

Figure 11 illustrates the distribution of green hen data when different test-to-train ratios are considered. The numbers provided underneath the plot are the sample counts for each subclass used for training. Note that the amount of data used for the high-intensity class is kept constant, because it is already small. In Figure 12a,b, we plot the test accuracy and loss versus different test-to-train ratios for different data bundles. Overall, the model demonstrates satisfactory performance even when extremely small datasets are used for training. Subtle drops in accuracy are seen when the ratio is smaller than 20/80, where the testset size is small, and the model tends to overfit as such. One can see the best performance when the ratio is around 30/70, as expected. As for the different data bundles used for training, no major differences are seen in the results for the small and medium test-to-train ratios. However, we will have more to say about this in the following analysis where we consider noisy data.

3.3. Robustness Analysis

Apart from the size of labeled data discussed in the previous section, accuracy and precision in data measurement are other important factors influencing the robustness of the model. It is clear that a clean or less noisy measured dataset would result in more accurate data interpretations by the recognition model. However, one should take into account the trade-off between accuracy and cost, especially if the wearable sensors are to be implemented in a relatively large scale. To evaluate the robustness of the model against inaccuracies arisen from the measurement device, Gaussian noise is intentionally added to the green hen trainset. In a series of separate experiments, zero-centered Gaussian noises of different standard deviations corrupt the data before training the LeNet5 model.

Figure 13 illustrates the accuracy and validation loss of the model trained by the AVM data bundle of the green hen against the increasing noise levels. As can be seen, the model exhibits a very high level of robustness against even large corruptions on the training data. We expect that this significant performance mainly owes to the effective data preprocessing through the signal imaging adopted for the hen motion data. Representing the motion data in the form of activity images enables the model to take advantages of the now-matured DCNN structures, the tools whose efficacy is demonstrated for real-world image understanding and classification tasks. The results might also imply that the proposed recognition model would perform almost equally well when low-cost wearable sensors are used instead.

Introducing noise to the data will allow us to also study the effect of factors that are otherwise hard to see, such as the windowing size presented in Section 2.3, and different data bundles introduced in Section 2.2. Figure 14 shows the accuracy of the LeNet5 recognition model trained by different data bundles whose associated activity images are generated using three different windowing widths, being 25, 50, and 100 data samples with a 50% overlap. The figure contains three panels corresponding to situations in which zero-, mid-, and high-level noises are introduced to the green hen training set. Multiple observations can be made from this analysis as follows.

First, the considered variations on the data bundle and window width do not present remarkable effects on the performance when uncorrupted data are used for training. Second, when noisy data are involved, however, the performance starts to degrade. In this case, the effect of window size is revealed earlier even with the mid-level noises (middle panel of Figure 14), while the difference between different data bundles remains subtle until high-level noises are considered, as in the right panel of the figure. Third, irrespective of the window width, the noise corruption shows stronger effects on poorer data bundles. This can be seen when comparing the accuracies of the AVM and A data bundles in the right panel of Figure 14. Additionally, notice in the same figure that the AV data bundle performs better than the AM data bundle, suggesting that the angular velocity data may be more useful (i.e., discriminative) than the magnetic field data in the performed experiment and analysis. Finally, when it comes to comparing the performance of models with different window widths, it looks as though there is an optimum value, as can be seen in the right panel of Figure 14. Small window widths result in poor performance because the noise affects the 2D patterns of the data more severely in this case. On the other hand, large window widths are also not desirable, most likely because in this case the number of data segments, i.e., activity images, becomes small and the network cannot train satisfactorily on the small dataset.

3.4. Performance Illustration on Unseen Dataset

Lastly, we would like to study how well the developed algorithms and models are applied to a different subject. To do so, we first train a fresh model on the blue hen dataset following the similar steps taken for the green hen. The results are very similar to what were achieved for the green hen.

Table 7. Evaluation of the performance indices when the AVM data bundle of blue hen is used for both training and testing of the LeNet5 model.

Performance Index	Amount
Accuracy	0.9961
Validation Loss	0.0027
Recall	$\left[\begin{array}{ccc}0.9844& 1& 0.9643\end{array}\right]$
Precision	$\left[\begin{array}{ccc}0.9977& 0.9956& 1\end{array}\right]$

reports the evaluation of different performance indices considered in this study. Additionally, Figure 15 shows the corresponding confusion matrix, indicating the success of the model in predicting the correct class, even for the high-intensity activities for which only a few instances were present. Overall, this experiment corroborates the repeatability of the presented approach on a new subject.

Next, the datasets of both green and blue hens were shuffled, and the LeNet5 model was trained on this combined dataset. Although some of the performance indices suffered slightly in this case (see

Table 8. Evaluation of the performance indices when the combined dataset of both hens is used for training and testing of the LeNet5 model.

Performance Index	Amount
Accuracy	0.9991
Validation Loss	0.0032
Recall	$\left[\begin{array}{ccc}0.9981& 1& 0.9687\end{array}\right]$
Precision	$\left[\begin{array}{ccc}1& 0.9987& 1\end{array}\right]$

), most of them remained remarkably high. This is also evident in the corresponding confusion matrix in Figure 16, where the majority of the instances are on the main diagonal. Overall, the results support the generalizability of the recognition model presented in this work.

One may wonder about the generalizability criterion on the leave-one-out experiment scenario. This refers to the scenario in which the data of multiple subjects are used for training, while one subject’s data are kept reserved for testing. Given that the dataset established in this study contains only two subjects, our results for the leave-one-out experiment were not satisfactory, as expected. To achieve decent performance on this type of experiment requires the involvement of multiple subjects with enough diversity in terms of important factors defining the problem domain.

One final note about typical multi-subject experiments is discussed here on the sensor misalignment across different subjects. To exclude the possibility of having this issue dominantly affecting the leave-one-out experiment, we performed the following offline analysis. The average 3D accelerations for both green and blue hens are calculated from the respective datasets. The rotational transformation between these two vectors could give an indication of the possible misalignment between the coordinate frames independently affixed to the subjects. This rotation matrix R for our dataset is obtained as

$$R=\left[\begin{array}{ccc}0.98& 0.17& 0\\ 0.17& 0.98& 0\\ 0& 0& 1\end{array}\right],$$

which is close to the identity matrix. This suggests that the sensors’ frames on the two subjects were almost perfectly aligned.

4. Conclusions

In this study, a deep learning framework was developed for accurately recognizing the activity levels of individual laying hens in a poultry system, using body-worn IMU modules. The framework employs a signal imaging technique that encodes the time-series data of sensors to some form of synthetic images for classification by off-the-shelf deep learning models. The approach allows the deep learning model to learn hidden patterns and significant correlations while end-to-end training, resulting in superior performance metrics compared to classical machine learning techniques and handcrafted feature selection. The developed model offers high performance even with a reduced trainset size and affected data richness, indicating the potential for deploying simpler IMU modules or different sensing technologies fulfilling minimal data requirements. The model also exhibits outstanding robustness to the Gaussian noise of different levels, and even a simple shallow model can perform as well as deeper structures such as ResNet. Also discussed was the generalizability of the model by illustrating its performance on unseen datasets.

The findings highlight the relevance of high-autonomy activity recognition models for monitoring modern poultry houses. They have the potential to effectively detect anomaly behaviors indicative of malfunctions in the management system or the emergence of animal diseases when combined with state-of-the-art unsupervised learning models such as auto-encoders. Future research shall focus on identifying correlations between individual and flock behaviors in poultry systems and exploring noninvasive sensing technologies such as machine vision. For the latter, the objective could be to identify features that fit well into the signal imaging paradigm.