REAL-TSE Challenge

The challenge does not define an official training set. Participants may use any open-source datasets for training, and pre-trained models are allowed, provided that all data sources and checkpoints are clearly documented.

Usage: Every dataset and pre-trained model must be reported in the system description with citations or links. The development and test sets of AliMeeting, AISHELL-4, AMI, DipCo, and CHiME6 must not be used at any stage, including pre-training, training, or data augmentation. However, the official training splits of these corpora are permitted. Note that some corpora (e.g., AMI) have multiple partition schemes — please refer to the FAQ: What data can I use for training? for details on which sessions are excluded.

The development set is derived from REAL-T, a real-world conversational dataset for target speaker extraction (to be released soon). The data originates from five speaker diarization corpora—AISHELL-4, AliMeeting, AMI, DipCo, and CHiME6—covering Mandarin and English in scenarios such as meetings and dinner parties. Each sample includes a multi-speaker mixture, enrollment utterance(s) for the target speaker, and the clean target reference.

The data is constructed through an automated pipeline that extracts naturally overlapping segments as mixtures and selects non-overlapping speech segments of at least 5 seconds as enrollment utterances. Unlike synthetic mixtures, it captures realistic overlap patterns, reverberation, ambient noise, and conversational turn-taking. The development set is intended for validation, model comparison, and hyper-parameter tuning only, and must not be used for training or fine-tuning.

Usage: The development set may be used for hyper-parameter selection, model comparison, and validation, but must not be used for model training or fine-tuning.

The evaluation set has been released and consists of two subsets that together assess both in-domain robustness and real-world generalization, with a total of 5,000 mix-enroll pairs:

Usage: The evaluation set must not be used for training, fine-tuning, hyper-parameter tuning, or any form of model optimization. It is reserved exclusively for final evaluation.

Composite Score

The four metrics described below are computed by the official evaluation pipeline. For each metric, valid submissions are ranked independently using dense ranking, and the final challenge ranking is determined by averaging the dense rankings across the four metrics.

Each team may submit up to three submissions per track. Each team may submit up to one submission per track per day, but on the final deadline day (Jun 20, 2026 Jun 25, 2026, AoE), this quota increases to three submissions per track. Only the best-performing submission from each team appears on the final leaderboard and is considered for the official ranking. Detailed formulations for each sub-metric will be specified in the challenge description paper and released with the official scoring toolkit.

Online Track — Low-Latency TSE

Key Requirements

1. Latency constraint: The end-to-end algorithmic latency must not exceed 100 ms. Systems may use limited future context (lookahead) as long as this latency budget is satisfied.
2. Streaming output: Models should support frame-wise or chunk-wise input with continuous output generation, without relying on full-utterance buffering.

Latency

Offline Track — Full-Context TSE

Key Requirements

1. Full context access: Systems may utilize the entire utterance and global context information.
2. No restriction on architecture: There is no limitation on model type, inference duration, or computational cost.
3. Encouraged exploration: Participants are encouraged to explore high-performance architectures, long-context modeling, and generation-based methods.

Detailed data usage constraints—including allowed training sources, prohibited corpora, and the usage scope of the development and evaluation sets—are specified in the Data section above. All participants must comply with those requirements.

Leaderboard Submission

We recommend using the public leaderboard for official submissions. Track pages are available for Track 1: Online Target Speaker Extraction and Track 2: Offline Target Speaker Extraction.

1. Package contents: For official evaluation, submit one .zip package containing outputs for both EVAL-1 and EVAL-2. EVAL-1-only or EVAL-2-only submissions are not accepted.
2. Evaluation target: On the submission page, select EVAL1+EVAL2 as the evaluation target and follow the Zip structure instructions shown on the page.
3. Brief system description: Each leaderboard submission must include a brief system description in the submission form. For Track 1, also include a brief latency description covering algorithmic latency, look-ahead, buffering, or any other latency-inducing components.
4. Official system description: For a submission to be considered valid for the final challenge, a detailed official system description must be submitted to the organizers by 1 July 2026 (AoE).
5. Evaluation status: Evaluation runs asynchronously after upload and may take around 2-3 hours, depending on queue and system load. You can monitor progress in the submission history.
6. Submission quota: Each team may submit up to three submissions per track. Each team may submit up to one submission per track per day, but on the final deadline day (Jun 20, 2026 Jun 25, 2026, AoE), each team may submit up to three submissions per track. Only successful submissions count toward this quota, and only the best-performing valid submission from each team will appear on the final leaderboard and be considered for the official ranking.
7. Final ranking: The leaderboard uses the official REAL-TSE metrics: TER, Timing F1, SIM, and DNSMOS OVRL. The final ranking is computed by averaging dense ranks across the metrics.
8. Encouraged open source: Teams are encouraged to make their models or scripts publicly available after the challenge.