# Monitoring Stable Soundscapes: AI Model Selection and Development Plans - Savonia AMK

> AI audio analysis detects abnormal stable sounds, improving early intervention and supporting horse welfare monitoring.

## Project Overview
In the second pilot phase of our project, we focus on detecting abnormal sound events within horse stables through advanced sound analysis using machine learning (ML). This pilot aims to alert caretakers promptly, especially during times when horses are unattended, such as nighttime, thus significantly improving animal welfare.

Horses frequently face health or safety issues, such as becoming trapped or sustaining injuries, that often go unnoticed, especially overnight. Early intervention is critical in mitigating risks and preventing serious complications.

## Methodology: Audio Event Classification
We are concentrating on audio event classification. This narrower focus maximizes the efficiency of our development team, as audio labeling generally demands fewer resources and less time compared to video labeling, and we can leverage specialized ML models that excel in audio processing tasks. Nevertheless, we continue using video recordings to support and validate our audio annotations.

Due to the infrequent and unpredictable nature of abnormal stable events, conventional supervised learning methods become challenging, given the limited availability of training examples. Therefore, our approach emphasizes methods capable of effectively learning from predominantly normal ambient sound data or very sparse data.

## ML Approaches
Recent research in sound anomaly detection suggests several promising ML approaches:
* **Autoencoders:** Well-established in anomaly detection, they learn to reconstruct normal soundscapes. Abnormal events produce larger reconstruction errors, signaling anomalies. While successful in industrial predictive maintenance, their effectiveness in varying stable environments requires further study.
* **Supervised Sound Classifiers:** Particularly convolutional neural networks (CNNs).

## Two-Step Development Approach
We use a two-step approach that balances power and practicality:

1. **Feature extraction with PANNs (a pre-trained CNN):**
   * PANNs (specifically the CNN14 model) have been trained on millions of everyday sounds in Google’s AudioSet.
   * For every 2-second audio clip, it produces a compact 2,048-number "fingerprint" (embedding) that captures pitch, rhythm, and texture.
   * Because this CNN is frozen, we get state-of-the-art audio features without heavy training.

2. **Task-specific classifier with dense layers:**
   * We stack a small feed-forward network (three dense layers plus dropout) on top of the embeddings.
   * This network learns to recognize patterns (e.g., kick vs. non-kick) using significantly less data and compute than training a full CNN.
   * Training is quick, as only about one million parameters are updated.

## Challenges and Hybrid Strategy
We currently face two primary challenges:
1. **Ambiguity in Defining Abnormal Events:** Lack of clear definitions for "abnormal" sounds complicates labeling.
2. **Scarcity of Training Data:** Limited or no initial recordings of abnormal events.

To overcome these, we propose a hybrid approach:
* **Broad Initial Labeling:** Identify common stable sound sources to establish a baseline for normal conditions.
* **Anomaly Detection Pipeline:** An autoencoder-based model trained on normal soundscapes identifies potential abnormal segments for labeling.
* **Supervised Classifier Integration:** A compact MLP classifier uses PANNs embeddings. Candidate segments from the autoencoder serve as seed examples for human verification, which are then used to iteratively fine-tune the classifier.
* **Integrated Hybrid System:** Initially, the anomaly detector and supervised classifier operate simultaneously. Over time, we will implement a voting or cascading strategy to leverage the strengths of both.

## Authors
* **Johannes Geisler**, RDI Specialist, DigiCenter
* **Osman Torunoglu**, RDI Specialist, DigiCenter
* **Finlay Hare**, RDI project worker, DigiCenter
* **Aki Happonen**, Digital Development Manager, DigiCenter
* **Heli Suomala**, Project Manager and Expert, Finnish Horse Information Centre

## References
* Zaman, Khalid, et al. “A survey of audio classification using deep learning.” IEEE access 11 (2023): 106620-106649.
* Koizumi, Yuma, et al. “Unsupervised detection of anomalous sound based on deep learning and the neyman–pearson lemma.” IEEE/ACM Transactions on Audio, Speech, and Language Processing 27.1 (2018): 212-224.
* Kong, Qiuqiang, et al. “Panns: Large-scale pretrained audio neural networks for audio pattern recognition.” IEEE/ACM Transactions on Audio, Speech, and Language Processing 28 (2020): 2880-2894.

***
*This work is part of the “Tekoälyä talleille” project at Savonia University of Applied Sciences.*