Pyannote rs
pyannote audio diarization in rust
- Compute 1 hour of audio in less than a minute on CPU. - Faster performance with DirectML on Windows and CoreML on macOS. - Accurate timestamps with Pyannote segmentation. - Identify speakers with wespeaker embeddings. The project is written primarily in Rust, distributed under the MIT License license, first published in 2024. Key topics include: asr, diarization, onnxruntime, rust, speech-recognition.
pyannote-rs
Pyannote audio diarization in Rust
Features
- Compute 1 hour of audio in less than a minute on CPU.
- Faster performance with DirectML on Windows and CoreML on macOS.
- Accurate timestamps with Pyannote segmentation.
- Identify speakers with wespeaker embeddings.
Install
consolecargo add pyannote-rs
Usage
See Building
Examples
See examples
<details> <summary>How it works</summary>pyannote-rs uses 2 models for speaker diarization:
- Segmentation: segmentation-3.0 identifies when speech occurs.
- Speaker Identification: wespeaker-voxceleb-resnet34-LM identifies who is speaking.
Inference is powered by onnxruntime.
- The segmentation model processes up to 10s of audio, using a sliding window approach (iterating in chunks).
- The embedding model processes filter banks (audio features) extracted with knf-rs.
Speaker comparison (e.g., determining if Alice spoke again) is done using cosine similarity.
</details>Credits
Big thanks to pyannote-onnx and kaldi-native-fbank
Contributors
Showing top 3 contributors by commit count.
Related Repositories
m-bain/whisperX
WhisperX: Automatic Speech Recognition with Word-level Timestamps (& Diarization)
modelscope/FunASR
Industrial-grade speech recognition toolkit: 170x realtime, 50+ languages, speaker diarization, emotion detection, streaming, and OpenAI-compatible API.
NVIDIA-NeMo/NeMo
A scalable generative AI framework built for researchers and developers working on Large Language Models, Multimodal, and Speech AI (Automatic Speech Recognition and Text-to-Speech)
alphacep/vosk-api
Offline speech recognition API for Android, iOS, Raspberry Pi and servers with Python, Java, C# and Node
k2-fsa/sherpa-onnx
Speech-to-text, text-to-speech, speaker diarization, speech enhancement, source separation, and VAD using next-gen Kaldi with onnxruntime without Internet connection. Support embedded systems, Android, iOS, HarmonyOS, Raspberry Pi, RISC-V, RK NPU, Axera NPU, Ascend NPU, x86_64 servers, websocket server/client, support 12 programming languages
PaddlePaddle/PaddleSpeech
Easy-to-use Speech Toolkit including Self-Supervised Learning model, SOTA/Streaming ASR with punctuation, Streaming TTS with text frontend, Speaker Verification System, End-to-End Speech Translation and Keyword Spotting. Won NAACL2022 Best Demo Award.