HeartMuLa Studio
https://github.com/fspecii/heartmula-studio- Category
- AI Tools
- Rank
- No. 1066Tools index
Previous survey · No. 1071 ·
- Pricing
- Open Source
- Type
- APP
- Builder
- fspecii
- GitHub
- 637 stars
- Date
About
Suno-like AI music-generation studio with reference-audio style transfer, built on HeartMuLa/heartlib.
What it does
HeartMuLa Studio turns prompts, lyrics, style tags, seeds, and optional reference clips into songs or instrumentals. Its browser interface queues generation jobs, reports progress live, plays results, and stores history, likes, and playlists. Lyrics assistance can use a local Ollama model, OpenRouter, or a custom compatible endpoint.
Why it's ranked here
This is a substantial local application, not a thin model wrapper. It combines composition controls, reference sampling, job management, playback, and a persistent library. GPU-aware model swapping makes demanding inference more practical, though the hardware requirements and experimental features narrow its audience.
What's good
Seed control supports repeatable generations. The queue handles multiple requests sequentially, while server-sent events expose startup and job progress. Reference clips get waveform playback, a draggable ten-second selection, and adjustable influence. Docker images, automatic model downloads, persisted audio, and a settings screen reduce routine setup work.
Tradeoffs
Local generation expects an NVIDIA GPU with at least 10GB of VRAM, with 12GB recommended. The container image is roughly 10GB and the first model download roughly 5GB. Reference-audio conditioning and compilation are experimental. AMD compatibility varies, compilation slows the first generation, and INT8 trades speed for lower peak memory use.
How to use it well
Use it for local song ideation, instrumental drafts, lyric experiments, and repeatable style studies on capable NVIDIA hardware. Start with Docker, keep model and audio volumes persistent, then tune memory swapping before trying INT8. Ollama suits private lyric assistance; OpenRouter or a compatible endpoint adds alternatives. It does not yet provide the announced LoRA voice-training workflow.
Technical notes+
README.md documents Docker Compose, prebuilt Docker Hub and GitHub Container Registry images, source installation, GPU tuning, and automatic Hugging Face downloads. backend/app/main.py exposes a FastAPI service with SQLite through SQLModel, permissive CORS, static audio mounts, uploads, generation jobs, history, downloads, cancellation, and SSE. frontend/src/api.ts wraps those HTTP and event interfaces, while frontend/src/App.tsx coordinates startup, queues, playback, likes, playlists, and settings. backend/heartmula/pipeline.py loads HeartMuLa and HeartCodec with mmgp offloading, seed determinism, optional quantized weights, and configurable model paths. backend/quantize_heartmula.py supplies offline INT8 and INT4 conversion paths. backend/app/services/llm_service.py routes lyrics work among Ollama, OpenRouter, and custom OpenAI-compatible services.
Observed
- License
- MIT, identified by the README badge.
- Languages
- Python backend and TypeScript React frontend.
- Install surface
- Docker Compose, prebuilt container images, or source installation with pip and Node.js.
- Interfaces
- Browser application plus a FastAPI HTTP API and server-sent event stream.
- Persistence
- SQLite stores job and library records; generated and reference audio use host directories.
- Platform support
- NVIDIA GPU deployment is documented for Linux, WSL2, and Windows; AMD compatibility varies.
- Core stack
- React, FastAPI, SQLModel, Tailwind CSS, WaveSurfer.js, PyTorch, HeartLib, and mmgp.
Read from README.md, frontend/vite.config.ts, frontend/eslint.config.js, frontend/postcss.config.js, frontend/tailwind.config.js, backend/quantize_heartmula.py, backend/app/main.py, frontend/src/api.ts, frontend/src/App.tsx, frontend/src/main.tsx, backend/app/models.py, backend/heartmula/__init__.py, backend/heartmula/pipeline.py, backend/app/services/llm_service.py.
What it can do
Generate AI music compositions
Text prompts or musical parameters → Audio music files
Transfer audio style from reference tracks
Reference audio file and target composition → Music with applied style characteristics
Create music in specific genres or styles
Genre specifications and musical preferences → Genre-specific audio compositions
Process and analyze reference audio for style extraction
Audio files for style reference → Extracted style parameters and characteristics
Generate music with customizable parameters
Musical structure preferences and composition settings → Customized audio tracks
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.