CAIRN Institute CAIRN Institute

Public Dataset Release: 10B Tokens of Humpback Whale Song

Author: CAIRN Institute
Published: August 4, 2026
Read time: 8–10 minutes
#DeepLearning #Audio #MarineBiology #NeuralAudio #OpenData

Quick Summary

We transformed 322.6 hours of raw underwater recordings from NOAA's SanctSound hydrophone network into a 796-million-token training dataset suitable for neural language models. Using a two-stage filtering process and a 9-codebook audio codec, we retained 88.5% of the raw data while ensuring high signal quality. The result: a structured, publicly available dataset that teaches transformer models to predict whale vocalization patterns.

Scope note: the 796M-token figure refers to the detector-filtered training subset used for model training (detector_score ≥ 0.7). The full DAC-tokenized SanctSound humpback corpus is much larger: ~10.4 billion tokens across ~485,804 chunks.

The Challenge: From Hours to Models

Training neural networks on audio is hard. Training them on whale audio is harder still — the data is noisy, sparse, and expensive to acquire. Traditional approaches either waste data through aggressive filtering or contaminate training sets with poor-quality samples.

We needed a pipeline that could:

  • ✓ Process hundreds of hours of continuous recordings
  • ✓ Identify and filter segments containing whale calls
  • ✓ Preserve full audio fidelity for neural codecs
  • ✓ Make the data publicly available for reproducible research

The Dataset: SanctSound Hawaiian Deployment

Our source data comes from NOAA's SanctSound project — a network of underwater hydrophones deployed across U.S. national marine sanctuaries. We focused on recordings from four Hawaiian stations monitoring humpback whales:

Station Purpose Period
HI01 Humpback whale habitat monitoring 2018–2022
HI03 Hawaiian Islands Humpback National Marine Sanctuary 2018–2022
HI04 Additional humpback monitoring site 2018–2022
HI05 Extended deployment with high whale activity 2018–2022

Raw data specs:

  • Format: Long-duration FLAC files (typically 1–4 hours each)
  • Sample rate: 44,100 Hz
  • Total duration: 322.6 hours across 286 source files
  • Data source: NOAA Passive Bioacoustic Monitoring (GCS bucket) — publicly available, anonymously accessible
Key Insight: This dataset is public! If you want to reproduce this work or build your own whale models, the source recordings are freely available on Google Cloud Storage.

The 7-Stage Processing Pipeline

Each recording passes through a carefully sequenced pipeline designed to maximize signal quality while minimizing data loss:

Diagram of the seven-stage SanctSound processing flow: calibration-tone removal, bandpass filtering, segmentation, peak and loudness normalization, heuristic quality filtering, and DAC 9-codebook tokenization. It also shows the separate detector-score threshold that selects the high-confidence training subset from the full release corpus.
The seven processing stages create the full DAC9 release corpus. A separate detector-score threshold (≥0.7) selects the high-confidence training subset used by the public model configurations.

Stage 1: Skip Test Tone

Hydrophone deployments begin with a calibration tone to verify that the recorder is functioning correctly. These tones—typically a steady sine wave at 1 kHz—would contaminate the training data if left in place.

Action: Automatically detect and remove the test tone (usually the first 30–60 seconds of each file).

Stage 2: Bandpass Filter

Not all frequencies matter. Whale vocalizations occupy a specific frequency range; boat noise, wave action, and other underwater sounds occupy different ranges.

Filter specs:

  • Frequency range: 80 Hz – 4 kHz (where humpback whale calls concentrate)
  • Effect: Attenuates boat noise (typically >8 kHz), wind-driven waves (<50 Hz)
  • Result: ~6 dB SNR improvement on average
Real SanctSound humpback sample before and after preprocessing, comparing the raw aligned source excerpt against the processed training chunk used by the pipeline.
Real SanctSound humpback sample before and after preprocessing. Using the same dB scale and a 0–4 kHz whale-band view, the processed chunk shows reduced broadband background and more legible harmonic structure after 80 Hz–20 kHz bandpass filtering, silence handling, peak normalization, and loudness normalization.
Listen to the same example:

Raw aligned source excerpt

Processed training chunk

Species Note: This bandpass is optimized for humpback whales. Blue and fin whales have fundamentals <400 Hz and would be over-filtered. Orca calls extend higher (up to 8 kHz), requiring a modified filter.

Stage 3: Segment

Continuous multi-hour recordings are sliced into fixed 30-second chunks. This chunk size is a deliberate trade-off:

  • Too short (e.g., 10s): Can't capture full song phrases (humpback songs typically last 10–20s)
  • Too long (e.g., 5 min): Makes quality scoring expensive
  • Sweet spot (30s): Captures most whale vocalizations while remaining manageable

Stages 4–7: Processing & Tokenization

Stages 4 and 5 normalize audio (peak and loudness). Stage 6 applies fast heuristic filtering using three signal-processing metrics (whale_cv, energy_ratio, min_whale_rms). Stage 7 encodes filtered chunks into DAC 9-codebook tokens.

Tokenization: The DAC 9-Codebook Encoding

The audio codec is the bridge between raw waveforms and transformer models. We use the DAC 44 kHz model, which produces tokens across 9 codebooks—each codebook captures progressively finer details of the audio signal.

Codec Specifications

Parameter Value
Model DAC 44kHz
Sample rate 44,100 Hz
Codebooks 9 (multi-scale residuals)
Total vocabulary size 9,219
30-second chunk → tokens ~23,247 tokens

Interleaved Codebook Format

The 9 codebooks are flattened into a single 1D sequence using interleaved encoding:

CB0_t0, CB1_t0, CB2_t0, ..., CB8_t0, CB0_t1, CB1_t1, ..., CB8_t1, ...
Diagram showing how the stored 9-by-time DAC code matrix is assigned codebook-specific token ranges and flattened in time-major order into the 1D sequence used for training.
Each stored DAC frame contributes nine consecutive model tokens—one from every codebook. Codebook-specific offsets keep those token ranges distinct before the matrix is flattened in time-major order.

Quality Filtering: Two-Stage Approach

We apply quality filtering at two points:

  • Stage 1 (Heuristic): Fast signal-processing heuristics filter out obviously empty chunks
  • Stage 2 (Neural): Each chunk receives a confidence score from a whale vocalization detector
Metric Value
Total chunks processed 38,709
Passing detector ≥ 0.7 34,262 (88.5%)
Mean detector score 0.856
Training audio hours 285.5 hours (88.5% retention)
What This Means: 88.5% data retention is excellent for audio filtering. We're not throwing away most of the data; we're just eliminating the most obviously bad chunks.

Chunk Adjacency: Building Longer Training Sequences

30-second chunks are too short for transformers to learn long-range structure. We concatenate multiple chunks to create longer training windows, using special tokens to mark temporal continuity:

  • SEP token (9218): Adjacent chunks—they were recorded consecutively
  • SEP_GAP token (9217): Non-adjacent chunks—different sessions or temporal gap

Adjacency Statistics

Run Length Count Duration
Isolated (1 chunk) 6,296 30s
2 adjacent chunks 2,317 60s
3 adjacent chunks 1,277 90s
6–10 adjacent 1,186 3–5 min
Longest run 29 chunks 14.5 min continuous

Mean run length: 2.7 chunks (~81 seconds of continuous audio)

Distribution of adjacent chunk run lengths in the detector-filtered training subset, reconstructed from chunk_scores.csv with detector_score at least 0.7.
Distribution of adjacent chunk run lengths in the detector-filtered training subset (detector_score ≥ 0.7), reconstructed from chunk_scores.csv. This is the same filtered subset summarized in the table above.
What This Means: Our training windows often contain genuinely continuous whale audio, not just random concatenations. For a 128k-token context window, a typical training window might contain 5–6 chunks with some continuous segments joined by SEP tokens.

Frequently Asked Questions

What's DAC?
DAC (Descript Audio Codec) is a neural audio codec—a neural network trained to compress audio into discrete codes using residual vector quantization (RVQ). Unlike traditional audio codecs (MP3, AAC), DAC produces a sequence of discrete tokens that can be fed to transformer language models, enabling next-token prediction on audio.
Why 9 codebooks?
Residual vector quantization works by first encoding the coarse signal, then iteratively refining it. Each codebook captures finer details. 9 codebooks is a good balance—more codebooks = higher fidelity but slower inference; fewer codebooks = faster but lower quality. For whale audio, 9 is sufficient to preserve vocalizations accurately.
Can I use this dataset?
Yes! The raw SanctSound recordings are publicly available on Google Cloud Storage (gs://noaa-passive-bioacoustic/) with anonymous access. You can download them, process them, and use them for your own research.
How do I reproduce this?
1. Download raw FLAC files from gs://noaa-passive-bioacoustic/
2. Run the processing pipeline (7 stages)
3. Use the tokenized .npy files for training
4. For curated release code, dataset links, checkpoint links, and publication updates, see the CAIRN Institute homepage

Further Reading