From 9c995fc4dd1c86be936333e73aa0e9f7d1bda9fa Mon Sep 17 00:00:00 2001 From: Deivid Soto Date: Sun, 31 May 2026 23:23:39 +0200 Subject: [PATCH] feat(stream): bitrate-sized readahead for play-while-download MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The torrent reader used a static 5 MiB readahead — about 1.9s of a 20 Mbps 4K stream — so streaming a torrent while it downloaded outran the download and stalled. anacrolix's reader already prioritises the pieces in the readahead window ahead of the playhead (and re-prioritises on seek); the window was just too small. dynamicReadahead sizes it to ~30s of video (clamped 8-96 MiB, 24 MiB default when bitrate is unknown). The torrent provider probes the bitrate asynchronously so stream start never blocks on ffprobe; readers created after the probe resolves pick up the accurate size. Real 4K (20.7 Mbps) -> 73 MiB. --- Docs/plans/unarr-agent-roadmap.md | 8 +++++- internal/engine/readahead.go | 32 +++++++++++++++++++++++ internal/engine/readahead_test.go | 43 +++++++++++++++++++++++++++++++ internal/engine/stream.go | 4 ++- internal/engine/stream_server.go | 24 ++++++++++++++--- internal/engine/torrent.go | 5 +++- 6 files changed, 110 insertions(+), 6 deletions(-) create mode 100644 internal/engine/readahead.go create mode 100644 internal/engine/readahead_test.go diff --git a/Docs/plans/unarr-agent-roadmap.md b/Docs/plans/unarr-agent-roadmap.md index 9f8dc5f..a2fa220 100644 --- a/Docs/plans/unarr-agent-roadmap.md +++ b/Docs/plans/unarr-agent-roadmap.md @@ -70,7 +70,7 @@ desde la web. Diseño + set de opciones en el estado abajo. - ~~Sin gestión de espacio en disco (`Statfs`)~~ ✅ **Pre-flight de espacio (2026-05-31)** — `CheckDiskSpace` antes de cada descarga (torrent/usenet/debrid) con reserva configurable `downloads.min_free_disk_mb` (default 2048); manager NO hace fallback en disco lleno; aviso web 507 `INSUFFICIENT_DISK` al despachar (torrentclaw). Monitoreo mid-download diferido. Ver estado abajo. - ~~Resume de torrent NO persiste reinicio del daemon~~ ✅ **Auto-resume tras reinicio (2026-05-31)** — `agent.ActiveTaskStore` persiste los `agent.Task` de descargas en vuelo (`active-tasks.json`); el daemon los re-somete al arrancar → los downloaders reanudan los bytes (torrent vía completion DB de anacrolix, debrid vía Range, usenet vía tracker). Dedup en `manager.Submit` (restore + re-despacho web no duplican). `shuttingDown` preserva el entry en apagado limpio (solo terminal genuino lo borra). Ver estado abajo. - Sin seeding/ratio lifecycle (flags existen, nadie los aplica). -- Reproducir-mientras-baja: readahead estático 5MB, sin playhead→prioridad dinámica. +- ~~Reproducir-mientras-baja: readahead estático 5MB~~ ✅ **Readahead dinámico (2026-05-31)** — `dynamicReadahead(bitrate)` = ~30s de vídeo (clamp 8–96 MiB; default 24 MiB sin bitrate) en vez de 5 MiB fijos (~1.9s a 20 Mbps → se atascaba). anacrolix ya prioriza piezas en esa ventana por delante del playhead + en seek; solo faltaba dimensionarla. Bitrate probado async (sin coste TTFF). Ver estado abajo. - ~~HDR→SDR sin tonemap~~ ✅ **Tonemap HDR→SDR (2026-05-31)** — cadena `zscale+tonemap=hable` en el transcode HLS cuando la fuente es HDR y el ffmpeg trae zscale (detectado en runtime, `FFmpegSupportsZscale`; sin zscale → comportamiento actual, no rompe). Verificado en 4K HDR10 real: de lavado/desaturado a colores vívidos. Ver estado abajo. - ~~Sin thumbnails~~ ✅ **Fotogramas bajo demanda (2026-05-31)** — `GET /thumbnail` (ffmpeg 1 frame, `-ss` antes de `-i`, MJPEG) + panel "Características del fichero" (ruta + mediainfo completa + tira de ~5 frames). Sprites/trickplay (scrubber pregenerado) siguen pendientes. Ver estado abajo. - Subtítulos bitmap (PGS/DVB) sin burn-in. @@ -100,6 +100,12 @@ WireGuard endpoint sin pin · sesión única (1 viewer). - ~~**Rutas localizadas unarr 404 (media)**~~ ✅ **ARREGLADO (2026-05-31)**: bajo `NEXT_PUBLIC_BRAND=unarr` el allowlist `UNARR_PAGE_PREFIXES` (paths EN) no reconocía los localizados de next-intl (`/es/biblioteca`, `/es/descargas`, `/es/perfil`) → 404. Fix (web): `enFirstSegmentByLocalized` (mapa localizado→EN derivado de `routing.pathnames`) + `toCanonicalPath()` en `branding/routes.ts` traduce el 1er segmento antes del match. Assertion anti-colisión en el build del mapa (fail-fast si una ruta futura reusa un segmento → no puede colar una ruta denegada). Verificado: 175 entradas, cero crossover; denegadas siguen denegadas. - **Thumbnails — sprites/trickplay (media)**: cerrado solo el camino bajo demanda (N frames en vivo). El scrubber pregenerado (sprite/BIF de toda la timeline, preview al pasar el ratón por la barra) queda como hueco propio: reaprovecharía `/thumbnail` + cacheo en disco del agente. Decidido alcance "solo bajo demanda" con el usuario (2026-05-31). +### Hueco medio — Readahead dinámico (ver-mientras-baja) ✅ CERRADO (2026-05-31) +El lector de torrent usaba un readahead **estático de 5 MiB** (~1.9s de un stream 4K de 20 Mbps) → al reproducir un torrent a medio bajar, la reproducción adelantaba a la descarga y se atascaba. +- `dynamicReadahead(bitrateBps)` (`readahead.go`): ~30s de vídeo, clamp [8, 96] MiB; default 24 MiB cuando el bitrate es desconocido (ya ~5× el viejo 5 MiB). anacrolix (`SetResponsive`+`SetReadahead`) ya prioriza las piezas de esa ventana por delante del read position y re-prioriza en seek — el feedback playhead→prioridad estaba; solo faltaba dimensionar la ventana. +- `torrentFileProvider` lleva `bitrateBps atomic.Int64`, sondeado **async** (`probeMediaInfo` en goroutine vía DataDir+DisplayPath) — sin coste de TTFF; hasta resolverse usa el default, y los readers posteriores (cada range/seek crea uno) cogen el valor preciso. StreamEngine (CLI) → default 24 MiB. +- **Smoke**: ffprobe en 4K real (20.7 Mbps) → readahead **73 MiB** (~28s) vs 5 MiB. Tests del func puro + -race limpio en el probe async. /critico: código sólido, fix aplicado (probe síncrono→async para eliminar 3s de TTFF si falta la cabecera). + ### Hueco medio — HDR→SDR tonemap en transcode ✅ CERRADO (2026-05-31) HDR (HDR10/HLG/DV) transcodificado a SDR salía lavado/desaturado (sin tonemap). Ahora `buildHLSFFmpegArgsAt` inserta `zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv` tras el scale y antes de `format=`, cuando `probe.HDR != "" && Transcode.TonemapHDR`. - **Gate por capacidad**: `FFmpegSupportsZscale(ffmpegPath)` (cacheado, `ffmpeg -filters`) → solo activa si el build trae zscale/zimg. Sin zscale → no se inserta (la fuente sigue reproduciéndose, desaturada — no rompe). `transcoder.go:270` ya advertía que builds sin zimg no pueden tonemapear; el static ffbinaries puede faltarle, pero `/usr/bin/ffmpeg` (distro) y el docker sí lo traen. diff --git a/internal/engine/readahead.go b/internal/engine/readahead.go new file mode 100644 index 0000000..eb48321 --- /dev/null +++ b/internal/engine/readahead.go @@ -0,0 +1,32 @@ +package engine + +// Torrent stream readahead sizing. +// +// anacrolix's Reader (SetResponsive + SetReadahead) already prioritises the +// pieces in a window ahead of the read position and re-prioritises on Seek — +// so the playhead→piece-priority feedback is built in. The problem was the +// window: a static 5 MiB is only ~1.6s of a 25 Mbps 4K stream, so playback +// outran the download and stalled. Sizing the window by bitrate (~30s of video) +// keeps a real buffer ahead of the playhead. +const ( + readaheadSeconds = 30 + minReadahead = 8 << 20 // 8 MiB + maxReadahead = 96 << 20 // 96 MiB — cap so a seek doesn't waste a huge fetch + defaultReadahead = 24 << 20 // 24 MiB — when bitrate is unknown (still ~5x the old 5 MiB) +) + +// dynamicReadahead returns the bytes-ahead window for a torrent reader given the +// stream's bitrate (bits/sec). Unknown/zero bitrate → a generous default. +func dynamicReadahead(bitrateBps int64) int64 { + if bitrateBps <= 0 { + return defaultReadahead + } + ra := bitrateBps / 8 * readaheadSeconds + if ra < minReadahead { + return minReadahead + } + if ra > maxReadahead { + return maxReadahead + } + return ra +} diff --git a/internal/engine/readahead_test.go b/internal/engine/readahead_test.go new file mode 100644 index 0000000..2097464 --- /dev/null +++ b/internal/engine/readahead_test.go @@ -0,0 +1,43 @@ +package engine + +import "testing" + +func TestDynamicReadahead(t *testing.T) { + cases := []struct { + name string + bitrateBps int64 + want int64 + }{ + {"unknown bitrate → default", 0, defaultReadahead}, + {"negative → default", -1, defaultReadahead}, + {"low bitrate clamps to min", 1_000_000, minReadahead}, // 1 Mbps → ~3.75 MiB < 8 MiB + {"mid bitrate scales", 5_000_000, 5_000_000 / 8 * readaheadSeconds}, // 5 Mbps → ~18.75 MiB + {"high bitrate within range", 25_000_000, 25_000_000 / 8 * readaheadSeconds}, // 4K ~25 Mbps → ~93.75 MiB + {"very high clamps to max", 80_000_000, maxReadahead}, // 80 Mbps → 300 MiB > cap + } + for _, c := range cases { + t.Run(c.name, func(t *testing.T) { + got := dynamicReadahead(c.bitrateBps) + if got != c.want { + t.Errorf("dynamicReadahead(%d) = %d, want %d", c.bitrateBps, got, c.want) + } + if got < minReadahead && c.bitrateBps > 0 { + t.Errorf("result %d below min %d", got, minReadahead) + } + if got > maxReadahead { + t.Errorf("result %d above max %d", got, maxReadahead) + } + }) + } +} + +func TestDynamicReadahead_BeatsOldStatic(t *testing.T) { + // The whole point: every result is bigger than the old static 5 MiB that + // stalled HD/4K. + const oldStatic = 5 * 1024 * 1024 + for _, b := range []int64{0, 1_000_000, 8_000_000, 25_000_000, 100_000_000} { + if got := dynamicReadahead(b); got <= oldStatic { + t.Errorf("dynamicReadahead(%d) = %d, not bigger than the old 5 MiB", b, got) + } + } +} diff --git a/internal/engine/stream.go b/internal/engine/stream.go index 1414f15..d2fe05e 100644 --- a/internal/engine/stream.go +++ b/internal/engine/stream.go @@ -235,7 +235,9 @@ func (s *StreamEngine) WaitBuffer(ctx context.Context, progressFn func(buffered, func (s *StreamEngine) NewFileReader(ctx context.Context) io.ReadSeekCloser { reader := s.file.NewReader() reader.SetResponsive() - reader.SetReadahead(5 * 1024 * 1024) // 5MB readahead + // Generous default window (vs the old static 5 MiB that stalled HD/4K). This + // CLI path has no bitrate probe, so dynamicReadahead(0) returns the default. + reader.SetReadahead(dynamicReadahead(0)) reader.SetContext(ctx) return reader } diff --git a/internal/engine/stream_server.go b/internal/engine/stream_server.go index 76615f5..3beb505 100644 --- a/internal/engine/stream_server.go +++ b/internal/engine/stream_server.go @@ -1129,19 +1129,37 @@ func (p *diskFileProvider) FileSize() int64 { } // NewTorrentFileProvider creates a FileProvider from an active torrent file. -func NewTorrentFileProvider(file *torrent.File) FileProvider { - return &torrentFileProvider{file: file} +// dataDir locates the on-disk file for a best-effort bitrate probe that sizes +// the streaming readahead. The probe runs ASYNC so stream start never blocks on +// ffprobe (a missing header would otherwise stall up to the probe timeout); +// until it resolves, readers use the default window, and readers created after +// it resolves pick up the accurate size. +func NewTorrentFileProvider(file *torrent.File, dataDir string) FileProvider { + p := &torrentFileProvider{file: file} + if dataDir != "" { + go func() { + if bps := probeMediaInfo(filepath.Join(dataDir, file.DisplayPath())).bitrateBps; bps > 0 { + p.bitrateBps.Store(bps) + } + }() + } + return p } // torrentFileProvider wraps a torrent.File to implement FileProvider. type torrentFileProvider struct { file *torrent.File + // bitrateBps sizes the reader's readahead window (see dynamicReadahead). + // Set asynchronously by the bitrate probe; 0 until then → default window. + bitrateBps atomic.Int64 } func (p *torrentFileProvider) NewFileReader(ctx context.Context) io.ReadSeekCloser { reader := p.file.NewReader() reader.SetResponsive() - reader.SetReadahead(5 * 1024 * 1024) + // Bitrate-sized window (vs the old static 5 MiB that stalled HD/4K). anacrolix + // prioritises pieces in this window ahead of the read position + on seek. + reader.SetReadahead(dynamicReadahead(p.bitrateBps.Load())) reader.SetContext(ctx) return reader } diff --git a/internal/engine/torrent.go b/internal/engine/torrent.go index b93b6bd..1be8f52 100644 --- a/internal/engine/torrent.go +++ b/internal/engine/torrent.go @@ -642,7 +642,10 @@ func (d *TorrentDownloader) GetStreamProvider(taskID string) (FileProvider, erro return nil, fmt.Errorf("torrent has no files") } - return NewTorrentFileProvider(video), nil + // The provider probes the bitrate asynchronously (to size the streaming + // readahead) — passing DataDir lets it locate the on-disk file without + // blocking stream start. + return NewTorrentFileProvider(video, d.cfg.DataDir), nil } // VideoExts is the canonical set of video file extensions used for file selection.