feat(waveform): precompute and store peaks in DB for instant rendering
Store waveform peaks inline in audio_versions (JSONB columns) so WaveSurfer
can render the waveform immediately on page load without waiting for audio
decode. Adds a 100-point mini-waveform for version selector thumbnails.
Backend:
- Migration 0006: adds waveform_peaks and waveform_peaks_mini JSONB columns
- Worker generates both resolutions (500-pt full, 100-pt mini) during transcode
and stores them directly in DB — replaces file-based waveform_url approach
- AudioVersionRead schema exposes both fields inline (no extra HTTP round-trip)
- GET /versions/{id}/waveform reads from DB; adds ?resolution=mini support
Frontend:
- audioService.initialize() accepts peaks and calls ws.load(url, Float32Array)
so waveform renders instantly without audio decode
- useWaveform hook threads peaks option through to audioService
- PlayerPanel passes waveform_peaks from the active version to the hook
- New MiniWaveform SVG component (no WaveSurfer) renders mini peaks in the
version selector buttons
Fix: docker-compose.dev.yml now runs alembic upgrade head before starting
the API server, so a fresh volume gets the full schema automatically.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Mistral Vibe
136
PLAN_waveform_precompute.md
Normal file
136
PLAN_waveform_precompute.md
Normal file
@@ -0,0 +1,136 @@
|
||||
# Plan: Waveform Pre-computation
|
||||
|
||||
**Branch:** `feature/waveform-precompute`
|
||||
**Goal:** Store waveform peaks in the database during indexing so WaveSurfer renders
|
||||
the waveform instantly (no waiting for audio decode), and show a mini-waveform in
|
||||
the library/overview song list.
|
||||
|
||||
## Background
|
||||
|
||||
WaveSurfer v7 supports `ws.load(url, channelData)` — when pre-computed peaks are
|
||||
passed as a `Float32Array[]`, the waveform renders immediately and audio streams in
|
||||
the background. Currently the frontend calls `ws.load(url)` which blocks until the
|
||||
full audio is decoded.
|
||||
|
||||
The worker already generates a 500-point peaks JSON file (`waveform_url`), but:
|
||||
- It is stored as a file on disk, not inline in the DB
|
||||
- The frontend never reads it (the `peaksUrl` option in `useWaveform` is wired to
|
||||
nothing)
|
||||
|
||||
## Architecture Decision
|
||||
|
||||
Add two JSONB columns to `audio_versions`:
|
||||
- `waveform_peaks` — 500 points, returned inline with version data, passed to WaveSurfer
|
||||
- `waveform_peaks_mini` — 100 points, returned inline, used for SVG mini-waveform in
|
||||
library/song list
|
||||
|
||||
This eliminates a separate HTTP round-trip and lets the UI render the waveform the
|
||||
moment the page loads.
|
||||
|
||||
---
|
||||
|
||||
## Checklist
|
||||
|
||||
### Backend
|
||||
|
||||
#### B1 — DB: Peaks columns + Alembic migration
|
||||
- [ ] Write migration test: after upgrade, `audio_versions` table has `waveform_peaks`
|
||||
and `waveform_peaks_mini` JSONB columns
|
||||
- [ ] Create `api/alembic/versions/0006_waveform_peaks_in_db.py`
|
||||
- [ ] Add `waveform_peaks` and `waveform_peaks_mini` JSONB columns to `AudioVersion`
|
||||
model in `api/src/rehearsalhub/db/models.py`
|
||||
|
||||
#### B2 — Worker: Generate and store both peak resolutions
|
||||
- [ ] Write unit tests for `extract_peaks()` in `worker/tests/test_waveform.py`:
|
||||
- Returns exactly `num_points` values
|
||||
- All values in [0.0, 1.0]
|
||||
- Empty audio returns list of zeros (no crash)
|
||||
- 100-point and 500-point both work
|
||||
- [ ] Update `handle_transcode` in `worker/src/worker/main.py`:
|
||||
- Generate `peaks_500 = extract_peaks(audio, 500)`
|
||||
- Generate `peaks_100 = extract_peaks(audio, 100)`
|
||||
- Store both on `AudioVersion` DB row
|
||||
- [ ] Write integration test: after `handle_transcode`, row has non-null
|
||||
`waveform_peaks` (len 500) and `waveform_peaks_mini` (len 100)
|
||||
|
||||
#### B3 — API Schema: Expose peaks in `AudioVersionRead`
|
||||
- [ ] Write serialization test: `AudioVersionRead.model_validate(orm_obj)` includes
|
||||
`waveform_peaks: list[float] | None` and `waveform_peaks_mini: list[float] | None`
|
||||
- [ ] Update `api/src/rehearsalhub/schemas/audio_version.py` — add both fields
|
||||
|
||||
#### B4 — API Router: `/waveform` endpoint reads from DB
|
||||
- [ ] Write endpoint tests:
|
||||
- `GET /versions/{id}/waveform` returns `{"data": [...500 floats...]}` from DB
|
||||
- `GET /versions/{id}/waveform?resolution=mini` returns 100-point peaks
|
||||
- 404 when version has no peaks yet
|
||||
- [ ] Update `api/src/rehearsalhub/routers/versions.py` — read from
|
||||
`version.waveform_peaks` / `version.waveform_peaks_mini` instead of file I/O
|
||||
|
||||
#### B5 — API: Peaks inline on versions list (verify, no change expected)
|
||||
- [ ] Write test: `GET /songs/{id}/versions` response includes `waveform_peaks` and
|
||||
`waveform_peaks_mini` on each version object
|
||||
- [ ] Confirm no router change needed (schema update in B3 is sufficient)
|
||||
|
||||
---
|
||||
|
||||
### Frontend
|
||||
|
||||
#### F1 — Types: Update `AudioVersionRead` TS type
|
||||
- [ ] Add `waveform_peaks: number[] | null` and `waveform_peaks_mini: number[] | null`
|
||||
to the TypeScript version type (wherever API types live)
|
||||
|
||||
#### F2 — `audioService`: Accept and use pre-computed peaks
|
||||
- [ ] Write unit tests for `AudioService.initialize()`:
|
||||
- With peaks: calls `ws.load(url, [Float32Array])` → waveform renders immediately
|
||||
- Without peaks: calls `ws.load(url)` → falls back to audio decode
|
||||
- Same URL + same peaks → no re-initialization
|
||||
- [ ] Update `AudioService.initialize(container, url, peaks?: number[])` in
|
||||
`web/src/services/audioService.ts`:
|
||||
- Call `ws.load(url, peaks ? [new Float32Array(peaks)] : undefined)`
|
||||
|
||||
#### F3 — `useWaveform` hook: Thread peaks through
|
||||
- [ ] Write hook test: when `peaks` option is provided, it is forwarded to
|
||||
`audioService.initialize`
|
||||
- [ ] Add `peaks?: number[] | null` to `UseWaveformOptions` in
|
||||
`web/src/hooks/useWaveform.ts`
|
||||
- [ ] Forward `options.peaks` to `audioService.initialize()` in the effect
|
||||
|
||||
#### F4 — `PlayerPanel`: Pass peaks to hook
|
||||
- [ ] Write component test: `PlayerPanel` passes `version.waveform_peaks` to
|
||||
`useWaveform`
|
||||
- [ ] Update `web/src/components/PlayerPanel.tsx` to extract and forward
|
||||
`waveform_peaks`
|
||||
|
||||
#### F5 — `MiniWaveform`: New SVG component for library overview
|
||||
- [ ] Write unit tests:
|
||||
- Renders SVG with correct number of bars matching peaks length
|
||||
- Null/empty peaks renders a grey placeholder (no crash)
|
||||
- Accepts `peaks`, `width`, `height`, `color` props
|
||||
- [ ] Create `web/src/components/MiniWaveform.tsx` — pure SVG, no WaveSurfer
|
||||
- [ ] Integrate into song list / library view using `waveform_peaks_mini`
|
||||
|
||||
---
|
||||
|
||||
## Testing Strategy
|
||||
|
||||
| Layer | Tool |
|
||||
|------------------|----------------------------------------------|
|
||||
| Backend unit | pytest, synthetic numpy arrays |
|
||||
| Backend integration | Real Postgres via docker-compose test profile |
|
||||
| Frontend unit | Vitest + Testing Library |
|
||||
| E2E | Playwright — assert waveform visible before audio `canplay` fires |
|
||||
|
||||
---
|
||||
|
||||
## Implementation Order
|
||||
|
||||
1. B1 — migration + model
|
||||
2. B2 — worker (TDD: unit tests → implementation → integration test)
|
||||
3. B3 — schema
|
||||
4. B4 — router
|
||||
5. B5 — verify versions list
|
||||
6. F1 — TS types
|
||||
7. F2 — audioService
|
||||
8. F3 — useWaveform
|
||||
9. F4 — PlayerPanel
|
||||
10. F5 — MiniWaveform
|
||||
Reference in New Issue
Block a user