English | 简体中文
Homepage • Document • Model • Dataset • Database
Embeat is a music recommendation system built on Spotify acoustic feature data. It encodes audio features into vectors via a contrastive learning model and combines them with a multi-channel recall strategy to deliver high-quality music recommendations.
Key Features:
- Acoustic Similarity: The EmbeatMLP model, trained on Spotify Audio Features (key, tempo, energy, valence, etc.), encodes acoustic features into 64-dim vectors
- Genre Awareness: Leverages 6,000+ micro-genre tags to precisely assign genres to 2M+ artists, preventing "acoustically similar but stylistically different" recommendations
- Multi-Channel Recall: 5 parallel recall channels (Acoustic Similarity / Same-Genre Popular / Same Artist / Similar Artists / Playlist Collaborative Filtering), merged and scored for final output
- Playlist Collaborative Filtering: Track2Vec (Word2Vec-inspired) learns track co-occurrence patterns from 1.88M Spotify playlists
- Millisecond-Level Response: Powered by the Qdrant vector database, retrieval across 45M tracks completes in 30–100ms
If you find this project helpful, please give it a ⭐️. It means a lot to a personal project, thanks!
- 2026-06-26: Open-source initial codebase + EmbeatMLP model weights
- 2026-06-26: Open-source 45M tracks dataset + Technical document
- 2026-07-02: Open-source Qdrant database (Password: 0616)
- 1K Stars: Open-source Track2Vec model weights + 1.8M playlists dataset
Below are example recommendation results from Embeat (please unmute before playing)
Uptown Funk - Bruno Mars [dance pop, pop]
| Seed Track | Embeat #1 | Embeat #2 | Embeat #3 |
|---|---|---|---|
| Uptown Funk - Bruno Mars | CAN'T STOP THE FEELING! - Justin Timberlake | Happy - Pharrell Williams | I Like to Move It - will.i.am |
demo_1_seed_track.mp4 |
demo_1_embeat_1.mp4 |
demo_1_embeat_2.mp4 |
demo_1_embeat_3.mp4 |
杀死那个石家庄人 - 万能青年旅店 [chinese indie rock]
| Seed Track | Embeat #1 | Embeat #2 | Embeat #3 |
|---|---|---|---|
| 杀死那个石家庄人 - 万能青年旅店 | 大石碎胸口 - 万能青年旅店 | 凄美地 - 郭顶 | 不要停止我的音乐 - 痛仰乐队 |
demo_2_seed_track.mp4 |
demo_2_embeat_1.mp4 |
demo_2_embeat_2.mp4 |
demo_2_embeat_3.mp4 |
Sis puella magica! - 梶浦由記 [anime score, japanese vgm]
| Seed Track | Embeat #1 | Embeat #2 | Embeat #3 |
|---|---|---|---|
| Sis puella magica! - 梶浦由記 | Decretum - 梶浦由記 | Zoltraak - Evan Call | Arrietty's Song - Cécile Corbel |
demo_3_seed_track.mp4 |
demo_3_embeat_1.mp4 |
demo_3_embeat_2.mp4 |
demo_3_embeat_3.mp4 |
Gizeh - Oskar Schuster [compositional ambient]
| Seed Track | Embeat #1 | Embeat #2 | Embeat #3 |
|---|---|---|---|
| Gizeh - Oskar Schuster | Vleurgat - Oskar Schuster | Sleeping Lotus - Joep Beving | Travelling - James Spiteri |
demo_4_seed_track.mp4 |
demo_4_embeat_1.mp4 |
demo_4_embeat_2.mp4 |
demo_4_embeat_3.mp4 |
Using the LLM-as-a-Judge method (GPT-5.5 / Gemini Flash 3.5 / Claude Sonnet 4.6), Embeat was blindly evaluated against Netease Cloud Music in AB tests:
| Judge Model | Embeat Wins | Netease Wins | Tie |
|---|---|---|---|
| Claude Sonnet 4.6 | 8 | 2 | 0 |
| Gemini Flash 3.5 | 9 | 1 | 0 |
| GPT 5.5 | 6 | 4 | 0 |
Conclusions:
- Embeat's core strength lies in its balance between style precision and artist diversity, with a particularly notable advantage in niche-style scenarios that span across languages and cultures
- Netease Cloud Music retains some reference value only in its deep mining of Mandarin-language local content
- For detailed comparison, please refer to the Technical document
EmbeatMLP - Acoustic Feature Encoding Model
- Input: 64-dim discrete features (key, mode, tempo, time_signature) + 64-dim continuous features (energy, valence, danceability, etc., 7 dimensions)
- Architecture: Dual-tower MLP (Discrete Tower + Acoustic Tower -> Backbone)
- Output: 64-dim L2-normalized vectors
- Training: Masked InfoNCE Loss, batch_size=4096, converges in ~70 steps
- Extremely small parameter count, supports real-time CPU-only inference
Track2Vec - Playlist Collaborative Filtering Model
- Based on Word2Vec Skip-Gram, treating playlists as "sentences" and tracks as "words"
- Training data: 1.88M Spotify playlists
- Vocabulary: 1.09M tracks, 64-dim vectors
- Supports real-time CPU-only inference, single query latency < 200ms
Input seed track: track_id / track_name + artist_name
│
├─ Channel 1 [similar]: Acoustic Similarity Recall (genre filtering + EmbeatMLP cosine similarity)
├─ Channel 2 [popular]: Same-Genre Popular Recall (genre filtering + popularity ranking)
├─ Channel 3 [same_artist]: Same Artist Recall (same artist + EmbeatMLP cosine similarity)
├─ Channel 4 [related_artist]: Similar Artists Recall (similar artists + EmbeatMLP cosine similarity)
├─ Channel 5 [related_track]: Playlist Collaborative Filtering (Track2Vec cosine similarity)
│
├─ ISRC Deduplication / Re-ranking / Same-Artist Ratio Control
│
└─ Output: Top-K Recommendation List
Embeat/
├── assets/ # Static assets folder
├── checkpoints/ # Model weights folder
│ ├── EmbeatMLP/ # EmbeatMLP model weights
│ └── Track2Vec/ # Track2Vec model weights (requires separate download)
├── data/ # Data processing folder (not fully organized)
├── infer/ # Inference code folder
│ ├── Embeat.py # Embeat recommendation system core
│ ├── EmbeatUtils.py # Embeat extension utilities
│ ├── infer.py # EmbeatMLP inference entry point
│ ├── eval_infer.py # EmbeatMLP evaluation utilities
│ └── hf_to_qdrant.py # Convert HF Dataset to Qdrant database
├── train/ # Training code folder
│ ├── model.py # EmbeatMLP model definition
│ ├── dataset.py # HF Dataset processing
│ ├── sampler.py # Positive/negative sample sampler
│ ├── loss.py # Masked InfoNCE Loss
│ ├── trainer.py # EmbeatMLP trainer
│ ├── train.py # EmbeatMLP training entry point
│ └── train_track2vec.py # Track2Vec training entry point
├── .env.example # Environment variables example for .env
├── requirements.txt
└── LICENSE
- Python >= 3.10
- PyTorch >= 2.6, < 2.7 (required for training)
- CUDA >= 12.0 (required for training)
- Qdrant (required for inference)
conda create -n embeat python=3.10
conda activate embeat
# Install PyTorch (CUDA 12.x), see https://pytorch.org/get-started/previous-versions/
pip install "torch>=2.6,<2.7" --index-url https://download.pytorch.org/whl/cu126
pip install -r requirements.txt# 1. Download the HuggingFace tracks dataset to `data/datasets/`, then rename it to `spotify_45m_tracks_metadata`
# 2. If you want to make more detailed adjustments to the training parameters, please review the code
cd train
python train.py# 1. Prepare the playlist training data (txt format, one playlist per line, space-separated track_ids)
# 2. Rename it to `spotify_playlists.txt`, and place it in the `train` folder
# 3. If you want to make more detailed adjustments to the training parameters, please review the code
cd train
python train_track2vec.py# 1. Get the acoustic feature data from HuggingFace tracks dataset
# 2. Or you can find some existed examples from infer/eval_infer.py
from infer.infer import infer
# 晴天 - Jay Chou (G major with fast tempo)
song_a = {"key": 7, "mode": 1, "tempo": 137, "time_signature": 4,
"danceability": 0.54, "energy": 0.56, "speechiness": 0.02,
"instrumentalness": 0.0, "valence": 0.41, "acousticness": 0.23,
"liveness": 0.1}
# 夜曲 - Jay Chou (F minor with slow tempo)
song_b = {"key": 5, "mode": 0, "tempo": 87, "time_signature": 4,
"danceability": 0.67, "energy": 0.65, "speechiness": 0.05,
"instrumentalness": 0.03, "valence": 0.57, "acousticness": 0.27,
"liveness": 0.19}
# Compute acoustic similarity via EmbeatMLP
similarity = infer(sample_a=song_a, sample_b=song_b,
checkpoint_path="checkpoints/EmbeatMLP/model.pt")
# Similarity: 0.7312
print(f"Similarity: {similarity:.4f}")# 1. Start the Qdrant service and import the database
# 2. Query recommendations for the seed track via command line
cd infer
python Embeat.py -t 5pIcwtJYNJx93l420oR2Vm # Query by Spotify Track ID
python Embeat.py -s "晴天 - Jay Chou" # Query by track name and artist
python Embeat.py -a "Jay Chou" # Query by artist name
# Output result for "晴天 - Jay Chou":
Query track_id: 5pIcwtJYNJx93l420oR2Vm
Query track info: 晴天 - Jay Chou
Query artist genres: ['mandopop', 'taiwan pop', 'c-pop', 'zhongguo feng']
-> Find query record used time: 21ms
-> Similar recall used time: 48ms
-> Popular recall used time: 24ms
-> Same artist recall used time: 4ms
-> Related artist recall used time: 5ms
-> Related track recall used time: 123ms
-> Re-ranking used time: 2ms
Result artist genres: ['taiwan indie', 'mandopop', 'chinese viral pop', 'cantopop']
======= Top 20 items =======
index track_id track_name artist_name album_name sources score
1 3Qj9Fy8BPbWmICTiNkuqB7 珊瑚海 Jay Chou 11月的蕭邦 ['same_artist', 'related_track'] 1.0
2 10VuSw48iPN2UK2xX9Y6P0 青花瓷 Jay Chou 我很忙 ['same_artist', 'related_track'] 1.0
3 0IAgufC1FlOg1nZMmRZxRr 突然好想你 Mayday 後 青春期的詩 ['popular', 'related_artist'] 1.0
4 2zB7NKVnzRh7xSUSPLErFr 明明就 Jay Chou 十二新作 ['same_artist', 'related_track'] 1.0
5 5WtMlbTDNZlbN8xZ5zfXva Our Singapore JJ Lin My August 9th - 50 Wonderful Years (2016 Edition) ['similar', 'related_artist'] 1.0
6 5cU1O9P0EDA0rPkPDykhIm 怎麼了 Eric Chou 終於了解自由 (Deluxe) ['popular', 'related_track'] 1.0
7 4daA20tBusVX29bUWgd8Dw 交換餘生 JJ Lin 交換餘生 ['popular', 'related_track'] 1.0
8 1EgGTmmFGtlWuqgXFLrp9x 溫柔 Mayday 愛情萬歲 ['related_artist'] 0.87
9 3ZuyyfGJqx9qhWTVtdMCWz 生命線 - 電視劇《院長爸爸》片頭曲 Bii 生命線 (電視劇《院長爸爸》片頭曲) ['similar'] 0.85
10 3p4UTiSIIpP4LFn0KEyEOj 十面埋伏 Eason Chan Live For Today ['related_artist'] 0.85
11 4lhbajK3dvUcJ0UNEeCdMn 飞鸟和蝉 Ren Ran Ren然 ['related_track'] 0.84
12 3e8uw7YMiKVcIakItBENqm 天天晴朗（蘇打綠版） sodagreen 秋：故事（蘇打綠版） ['similar'] 0.83
13 26O8PmJ32hwAbZnIhbJJwZ 天使 Mayday 為愛而生 ['related_artist'] 0.82
14 3LgoekU3dE5ZMLvuL3NIt9 清醒 (戲劇《淺情人不知》片尾曲) Ariel Tsai 清醒 (戲劇《淺情人不知》片尾曲) ['similar'] 0.81
15 1WnTw4Tzpc5q9dHMjs4aHu 陰天快樂 Eason Chan rice & shine ['related_artist'] 0.8
16 14GFYAUxkeXranhS2qrYIZ 我想要佔據你 告五人 帶你飛 ['related_track'] 0.8
17 1ylx8p71GKQy5g1t4gzuEz 抱歉 Sam Lee 原諒我沒有說 ['similar'] 0.79
18 7fAdinC2UTc0Y9GiKrkTtu 字字句句 卢卢快闭嘴 字字句句 ['related_track'] 0.78
19 1VG8o5rUZQZ0wjs7Bi4siU 最熟悉的陌生人 Elva Hsiao 蕭亞軒 (最熟悉的) ['similar'] 0.77
20 1lM4cYuhJHSsDRfD0ZCRN7 你的背包 Eason Chan 陳奕迅 國語精選 (HQCDII) ['related_artist'] 0.77
Query used time: 0.229s
- GD Music (Live Demo): https://music.gdstudio.xyz
- Bilibili: https://space.bilibili.com/13715770
- Telegram: https://t.me/gdstudio_music
| Scope | License |
|---|---|
| Code, Model Weights | MIT |
| Datasets, Database | CC-BY-NC 4.0 |
Made with ❤️ by GD Studio