Executive Summary:

For a leading US-based pharmaceutical company, manually reviewing hours of behavioral video footage was slowing drug research and introducing inconsistency into clinical trial data. This case study details how InXiteOut used computer vision and AI-powered video analytics to automate behavioral analysis of non-rodent subjects, reducing video processing time by 70%, improving tagging accuracy by 20%, and delivering standardized, bias-free observational data to accelerate pharma R&D cycles.

Client Context

A leading US-based pharmaceutical major evaluates the impact of drug compounds on non-rodent subjects (specifically, dogs) by continuously monitoring their behavior in a controlled environment. Subject activity is captured via video feeds, providing critical observational data for the drug research team.

The Challenge

The existing video analysis process relied entirely on human reviewers to manually tag minute behavioral patterns across multiple subjects. This created several critical bottlenecks:

• Laborious & Time-Consuming: Reviewers had to watch hours of footage to track multiple subjects, severely delaying time-to-insight.
• Sub-Optimal Accuracy: Manually identifying and logging rapid or subtle behaviors was highly prone to error.
• Inconsistent Data: Behavioral tagging varied from reviewer to reviewer, introducing subjective human bias into objective clinical research.

The client needed a Computer Vision and Video Analytics solution to automatically extract, standardize, and accelerate the generation of actionable research data.

The InXiteOut Approach

We engineered an end-to-end, AI-powered video analysis suite to replace manual observation with high-precision automated tracking. The complete implementation involved the following stages:

Custom Data Annotation and Alignment

We collaborated closely with the client's R&D team to define the exact parameters of over 20 distinct behavior classes (such as drooling, yawning, wagging, and drowsing). Leveraging CVAT (an AI-assisted open-source tool), we annotated vast amounts of training data and converted it into a Darknet format. This preparation ensured the foundation of our custom model was perfectly aligned with strict clinical expectations.

High-Precision Object Detection and Tracking

We trained a custom object detection model built on the YOLOv11 architecture to automatically detect the subjects individually and localize their exact coordinates. To accelerate this intensive process, training was conducted in Distributed Data Parallel (DDP) mode across multiple GPUs via Azure Kubeflow on an AKS cluster. Additionally, we implemented a Strong-Sort based Multiple Object Tracking (MOT) solution to consistently track the subjects across frames, ensuring continuous movement identification.

Automated Behavioral Tagging Pipeline

We deployed an end-to-end video analytics pipeline that processes the tracked footage to analyze sequences of frames containing detected objects. The pipeline uses LSTM on DINOv2 embeddings of sequential frames to identify behaviors / actions based on contextual patterns and automatically generates highly structured JSON outputs. These outputs capture the detected subjects, their specific behavior classes, precise location coordinates, and exact frame timelines, delivering the data in a ready-to-consume format for immediate downstream analysis by the research team.

Technology Stack Used

• Models: Ultralytics YOLOv11 (custom object detection), Strong-Sort MOT (multiple object tracking), Meta DINOv2 (frame embeddings), custom LSTM on PyTorch (behavior class recognition on sequential frames).
• Training Orchestration: Distributed Data Parallel (DDP) mode for multi-GPU acceleration, orchestrated via Azure Kubeflow on an AKS cluster.
  
  

Benefits Delivered

The automated AI analysis suite transformed the client's observational research process, delivering immediate operational and scientific value:

• ~70% Reduction in Processing Time: The automated pipeline drastically cut the manual effort required to analyse video data, freeing researchers to focus on core scientific evaluation.
• ~20% Boost in Tagging Accuracy: The custom computer vision model significantly outperformed manual human review, ensuring highly precise identification of complex and subtle behavioural patterns.
• Accelerated R&D Cycles: Faster video processing enabled much quicker feedback loops on drug effects, directly reducing the overall time to actionable research insights.
• Elimination of Human Bias: Standardized the behavioural tagging process, completely removing subjective human variability and ensuring reliable, clinical-grade data consistency.