feat(stream): on-demand frame thumbnails via /thumbnail (hueco medio)

Add GET /thumbnail to the agent stream server: ffmpeg extracts one frame
at a timestamp (-ss before -i, single-frame MJPEG to stdout) for the web's
file-characteristics panel. Auth via a token scoped thumb:<sha256(path)>
(same HMAC scheme as /stream and /hls; the web mints, the agent verifies),
clamped to a real regular file, 404-no-oracle on a bad token, 20s timeout.
ffmpeg path wired into the stream server from the daemon. Version -> 0.13.0.
This commit is contained in:
Deivid Soto 2026-05-31 18:27:22 +02:00
parent 950cdb4efe
commit 2be92516c6
6 changed files with 329 additions and 2 deletions

View file

@ -72,7 +72,7 @@ desde la web. Diseño + set de opciones en el estado abajo.
- Sin seeding/ratio lifecycle (flags existen, nadie los aplica).
- Reproducir-mientras-baja: readahead estático 5MB, sin playhead→prioridad dinámica.
- HDR→SDR sin tonemap (zscale/zimg) → HDR desaturado.
- Sin thumbnails/sprites/trickplay.
- ~~Sin thumbnails~~**Fotogramas bajo demanda (2026-05-31)**`GET /thumbnail` (ffmpeg 1 frame, `-ss` antes de `-i`, MJPEG) + panel "Características del fichero" (ruta + mediainfo completa + tira de ~5 frames). Sprites/trickplay (scrubber pregenerado) siguen pendientes. Ver estado abajo.
- Subtítulos bitmap (PGS/DVB) sin burn-in.
- Audio siempre downmix estéreo AAC (sin passthrough 5.1).
- Mediaserver solo DETECTA Plex/Jellyfin/Emby — no biblioteca navegable propia.
@ -96,6 +96,15 @@ WireGuard endpoint sin pin · sesión única (1 viewer).
- **Refrescar/limpiar streamUrl al re-registrar** (baja): tras reinicio del daemon el secreto cambia; URLs `?t=` ya guardadas en `download_task.streamUrl` quedan stale hasta re-stream. Es auto-curativo, pero el web podría limpiar streamUrl en el re-register del agente.
- **gofmt preexistente** en `internal/agent/types.go` (StreamSession) y `hls.go`/`torrent.go`/`stream_source.go` (no introducido por este trabajo) — chore aparte.
- **Funnel = SPOF CloudFlare** (ya en huecos medios): el funnel sigue siendo trycloudflare; relay propio pendiente.
- **Rutas localizadas unarr 404 (media)**: bajo `NEXT_PUBLIC_BRAND=unarr` el allowlist `UNARR_PAGE_PREFIXES` solo lista los paths EN (`/library`, `/title`, …), pero next-intl sirve los localizados (`/es/biblioteca`, `/es/descargas`, `/es/perfil`) → 404 al navegar la biblioteca en español. Las páginas EN (`/title/<id>`) funcionan. Hallado durante el smoke de "características del fichero" (2026-05-31). Fix: añadir los pathnames localizados al allowlist o derivarlos del mapeo de next-intl. Ajeno a este hueco.
- **Thumbnails — sprites/trickplay (media)**: cerrado solo el camino bajo demanda (N frames en vivo). El scrubber pregenerado (sprite/BIF de toda la timeline, preview al pasar el ratón por la barra) queda como hueco propio: reaprovecharía `/thumbnail` + cacheo en disco del agente. Decidido alcance "solo bajo demanda" con el usuario (2026-05-31).
### Hueco medio — Características del fichero + thumbnails bajo demanda ✅ CERRADO (2026-05-31)
Panel "ver características del fichero" (ruta + mediainfo completa: codec/HDR/bit-depth/tracks audio+subs/tamaño/duración — ya en DB vía ffprobe, solo faltaba surface) + tira de fotogramas extraídos en vivo por el agente.
- **CLI**: `GET /thumbnail?p=&pos=&w=&t=` en el stream server (ffmpeg `-ss <pos>` antes de `-i`, `-frames:v 1`, MJPEG a stdout). Token scope `thumb:<sha256(path)>` (mismo HMAC que `/stream`/`/hls`; web mintea, agente verifica; vector cross-lang Go↔TS pinneado). Clamp a fichero regular, 404-sin-oracle, timeout 20s. `ffmpegPath` cableado en `daemon.go`. Floor `0.13.0`.
- **WEB**: endpoints bajo `/api/internal/stream/` (permitido en unarr; `/api/internal/library` NO) — `file-details` (mediainfo + URLs de frames vía funnel HTTPS) + `owned-files` (lista mínima por contentId, solo items con ffprobe). Lógica pura testeada en `src/lib/stream/thumbnails.ts`. Modal compartido `FileDetailsModal`/`useFileDetails` con skeleton + carga progresiva ("Generando X/N…") + fallback por frame. Gating `supportsThumbnails`/`THUMBNAIL_MIN_VERSION`.
- **Alcance en ambas marcas**: torrentclaw → acción en los 3 builders de menú de biblioteca (`fileInfoMenuItem` compartido). unarr → `UnarrFileDetailsButton` en `/title/<id>` (la biblioteca unarr son estanterías, no `LibraryPage`). Modal reutiliza labels neutrales (namespace `library`, no `torrent`) → marca limpia.
- **Tests/smoke**: Go (token vector, args, 400/404/503, stub-ffmpeg success) + web (resolveThumbnails, parity, version gate, i18n 7 locales). Smoke real contra biblioteca local 4K (Frankenstein, HEVC DV+HDR10): ffmpeg extrae JPEG válido, modal unarr muestra mediainfo + 5 frames vía funnel. /critico 4 revisores → 5 fixes (clipboard promise, dedup posiciones short-clip, tipos compartidos, guard videoInfo, helper menú).
---

View file

@ -310,6 +310,9 @@ func runDaemonStart() error {
// Create persistent stream server
streamSrv := engine.NewStreamServer(cfg.Download.StreamPort)
streamSrv.SetUPnPEnabled(cfg.Download.EnableUPnP)
// Wire ffmpeg so /thumbnail can extract single frames for the web's "file
// characteristics" panel (frames on demand). Empty = thumbnails 503.
streamSrv.SetFFmpegPath(ffmpegResolved)
streamSrv.SetRequireStreamToken(cfg.Download.RequireStreamToken)
// Report the stream-token signing key ONLY when enforcing, so the web's
// "secret present → mint HLS token" signal accurately means "this agent

View file

@ -1,4 +1,4 @@
package cmd
// Version is the CLI version. Overridden by goreleaser ldflags at release time.
var Version = "0.12.0"
var Version = "0.13.0"

View file

@ -90,6 +90,11 @@ type StreamServer struct {
streamSecret []byte
requireToken bool
// ffmpegPath is the resolved ffmpeg binary, used by /thumbnail to extract a
// single frame on demand. Empty = thumbnails disabled (503). Set once before
// Listen() via SetFFmpegPath; read-only thereafter so the handler needs no lock.
ffmpegPath string
lastActivity atomic.Int64
maxByteOffset atomic.Int64 // highest sequential read position (main playback connection)
totalFileSize atomic.Int64
@ -147,6 +152,13 @@ func (ss *StreamServer) SetUPnPEnabled(enabled bool) {
ss.enableUPnP = enabled
}
// SetFFmpegPath sets the ffmpeg binary used by /thumbnail to extract single
// frames on demand. Call before Listen(); empty leaves thumbnails disabled
// (the handler returns 503). Read-only after Listen() — no locking in the handler.
func (ss *StreamServer) SetFFmpegPath(path string) {
ss.ffmpegPath = path
}
// SetCORSAllowedOrigins replaces the operator-supplied extra origins. The
// default allowlist (torrentclaw.com / app.torrentclaw.com / localhost dev
// ports) is always merged in. Call before Listen().
@ -203,6 +215,7 @@ func (ss *StreamServer) Listen(ctx context.Context) error {
mux.HandleFunc("/health", ss.healthHandler)
mux.HandleFunc("/playlist.m3u", ss.playlistHandler)
mux.HandleFunc("/hls/", ss.hlsHandler)
mux.HandleFunc("/thumbnail", ss.thumbnailHandler)
// SO_REUSEADDR allows immediate rebind if the port is in TIME_WAIT (e.g. after agent restart)
lc := net.ListenConfig{
@ -802,6 +815,127 @@ func (ss *StreamServer) handler(w http.ResponseWriter, r *http.Request) {
http.ServeContent(w, r, provider.FileName(), time.Time{}, reader)
}
// thumbnailHandler serves ONE JPEG frame decoded from a file at a timestamp.
// It backs the web's "file characteristics" panel (frames on demand, hueco
// medio): the panel renders a strip of <img> at several positions, each hitting
// this route. Independent of the active /stream — no session, no provider, no
// effect on playback; ffmpeg just seeks the path and emits a single frame.
//
// Auth: a token scoped thumb:<sha256(path)> minted by the web with this agent's
// stream secret. The path travels in ?p= (already client-visible — the library
// UI shows it) and the token's scope binds that exact path, so a tampered p
// fails verification. 404 (not 401/403) on a bad token — no oracle, same as
// /stream. The path is additionally clamped to a real regular file as
// defense-in-depth against a (trusted) web bug pointing ffmpeg at a device/FIFO.
func (ss *StreamServer) thumbnailHandler(w http.ResponseWriter, r *http.Request) {
ss.lastActivity.Store(time.Now().UnixNano())
if ss.writeCORSHeaders(w, r, "") {
return
}
q := r.URL.Query()
rawPath := q.Get("p")
if rawPath == "" {
http.Error(w, "missing path", http.StatusBadRequest)
return
}
if !ss.checkStreamToken(streamScopeThumb(rawPath), q.Get("t")) {
clientIP, _, _ := net.SplitHostPort(r.RemoteAddr)
log.Printf("[thumbnail] rejected from %s — bad/absent token", clientIP)
http.Error(w, "not found", http.StatusNotFound)
return
}
if fi, err := os.Stat(rawPath); err != nil || !fi.Mode().IsRegular() {
http.Error(w, "not found", http.StatusNotFound)
return
}
if ss.ffmpegPath == "" {
http.Error(w, "thumbnails unavailable", http.StatusServiceUnavailable)
return
}
pos := parseThumbPos(q.Get("pos"))
width := parseThumbWidth(q.Get("w"))
// Cap the work: a single keyframe decode is fast, but a corrupt/huge file or
// a seek past EOF could hang ffmpeg. 20s is generous for a keyframe seek.
ctx, cancel := context.WithTimeout(r.Context(), 20*time.Second)
defer cancel()
cmd := exec.CommandContext(ctx, ss.ffmpegPath, buildThumbnailArgs(rawPath, pos, width)...)
var stderr strings.Builder
cmd.Stderr = &stderr
out, err := cmd.Output()
if err != nil || len(out) == 0 {
// A seek past EOF yields no frame — a benign empty output, not an error
// worth alarming on. Log at most a short line for diagnosis.
log.Printf("[thumbnail] no frame (pos=%.1f w=%d path=%q): err=%v %s",
pos, width, rawPath, err, strings.TrimSpace(stderr.String()))
http.Error(w, "thumbnail failed", http.StatusInternalServerError)
return
}
w.Header().Set("Content-Type", "image/jpeg")
// path+pos is stable content; let the browser cache so re-opening the panel
// doesn't re-run ffmpeg. private — it's a frame of the user's own file.
w.Header().Set("Cache-Control", "private, max-age=3600")
w.Header().Set("Content-Length", strconv.Itoa(len(out)))
if _, err := w.Write(out); err != nil {
log.Printf("[thumbnail] write failed: %v", err)
}
}
// buildThumbnailArgs builds the ffmpeg argv that decodes ONE frame at posSec and
// writes a scaled JPEG to stdout. `-ss` BEFORE `-i` does an input (keyframe)
// seek — near-constant time regardless of position — instead of decoding from
// the start. scale=w:-2 preserves aspect with an even height (mjpeg/yuv420
// requires even dimensions). `-an -sn` drops audio/subtitle streams.
func buildThumbnailArgs(path string, posSec float64, width int) []string {
return []string{
"-nostdin",
"-loglevel", "error",
"-ss", strconv.FormatFloat(posSec, 'f', 3, 64),
"-i", path,
"-frames:v", "1",
"-vf", fmt.Sprintf("scale=%d:-2", width),
"-an", "-sn",
"-f", "mjpeg",
"pipe:1",
}
}
// parseThumbPos parses a non-negative seconds offset; defaults to 0 on garbage.
func parseThumbPos(s string) float64 {
if s == "" {
return 0
}
v, err := strconv.ParseFloat(s, 64)
if err != nil || v < 0 {
return 0
}
return v
}
// parseThumbWidth parses the requested width, defaulting to 320 and clamping to
// [80, 640] so a caller can't ask ffmpeg to upscale to an absurd size.
func parseThumbWidth(s string) int {
const def, min, max = 320, 80, 640
if s == "" {
return def
}
v, err := strconv.Atoi(s)
if err != nil {
return def
}
if v < min {
return min
}
if v > max {
return max
}
return v
}
// serveGrowing range-serves a growing remux source (hueco #3 / 3b). Unlike
// http.ServeContent it can't rely on a fixed file size: ffmpeg `-c copy` is
// still writing, and the final byte count isn't known until it exits. So we:

View file

@ -49,6 +49,17 @@ const (
// id means a token minted for one session never validates another.
func streamScopeHLS(sessionID string) string { return "hls:" + sessionID }
// streamScopeThumb is the token scope for a single-frame thumbnail of a
// specific file (the web's "file characteristics" panel). Binding the file
// path's SHA-256 into the scope means a token minted for one file never
// validates a thumbnail request for another — a leaked thumbnail URL exposes
// only the one frame-source it was signed for. The web mints the matching
// scope in src/lib/stream-token.ts (streamScopeThumb), byte-for-byte.
func streamScopeThumb(filePath string) string {
sum := sha256.Sum256([]byte(filePath))
return "thumb:" + hex.EncodeToString(sum[:])
}
// newStreamSecret returns 32 cryptographically-random bytes used to sign stream
// tokens for the lifetime of the daemon. Regenerated each start, so tokens from
// a previous run stop validating (the web re-resolves the URL on demand).

View file

@ -0,0 +1,170 @@
package engine
import (
"net/http"
"net/http/httptest"
"net/url"
"os"
"path/filepath"
"strings"
"testing"
"time"
)
func thumbReq(remoteAddr, query string) *http.Request {
r := httptest.NewRequest(http.MethodGet, "http://stream.test/thumbnail"+query, nil)
r.RemoteAddr = remoteAddr
return r
}
func indexOfArg(ss []string, target string) int {
for i, s := range ss {
if s == target {
return i
}
}
return -1
}
// TestStreamScopeThumb_Vector pins the scope string against the web's
// TypeScript minter (tests/unit/stream-token.test.ts asserts the same vector).
// A token the web mints for a file MUST reduce to the same scope here or the
// thumbnail 404s.
func TestStreamScopeThumb_Vector(t *testing.T) {
got := streamScopeThumb("/movies/Example (2020)/Example.mkv")
const want = "thumb:d3f919154ea48832a0b52e4b4ca3e81185ea5f4e2b9e5fece32c6651908cbdd8"
if got != want {
t.Fatalf("streamScopeThumb mismatch (web parity broken!): got %q want %q", got, want)
}
}
func TestStreamScopeThumb_DistinctPerPath(t *testing.T) {
a := streamScopeThumb("/a.mkv")
b := streamScopeThumb("/b.mkv")
if a == b {
t.Error("distinct paths produced the same thumb scope")
}
if streamScopeThumb("/a.mkv") != a {
t.Error("same path produced a different thumb scope (not deterministic)")
}
if !strings.HasPrefix(a, "thumb:") || len(a) != len("thumb:")+64 {
t.Errorf("scope %q is not thumb:<64 hex>", a)
}
}
func TestBuildThumbnailArgs(t *testing.T) {
args := buildThumbnailArgs("/x/movie.mkv", 123.5, 320)
joined := strings.Join(args, " ")
ssIdx, iIdx := indexOfArg(args, "-ss"), indexOfArg(args, "-i")
if ssIdx < 0 || iIdx < 0 || ssIdx > iIdx {
t.Errorf("-ss must precede -i (fast input seek): %v", args)
}
if args[ssIdx+1] != "123.500" {
t.Errorf("pos arg = %q, want 123.500", args[ssIdx+1])
}
if args[iIdx+1] != "/x/movie.mkv" {
t.Errorf("input arg = %q, want the path", args[iIdx+1])
}
for _, want := range []string{"-frames:v 1", "scale=320:-2", "-f mjpeg", "pipe:1", "-an", "-sn"} {
if !strings.Contains(joined, want) {
t.Errorf("args missing %q: %v", want, args)
}
}
}
func TestParseThumbPos(t *testing.T) {
cases := map[string]float64{"": 0, "abc": 0, "-5": 0, "0": 0, "12.5": 12.5, "600": 600}
for in, want := range cases {
if got := parseThumbPos(in); got != want {
t.Errorf("parseThumbPos(%q) = %v, want %v", in, got, want)
}
}
}
func TestParseThumbWidth(t *testing.T) {
cases := map[string]int{"": 320, "abc": 320, "10": 80, "5000": 640, "200": 200, "640": 640, "80": 80}
for in, want := range cases {
if got := parseThumbWidth(in); got != want {
t.Errorf("parseThumbWidth(%q) = %v, want %v", in, got, want)
}
}
}
func TestThumbnailHandler_MissingPath_400(t *testing.T) {
srv := NewStreamServer(0)
rec := httptest.NewRecorder()
srv.thumbnailHandler(rec, thumbReq("198.51.100.7:40000", ""))
if rec.Code != http.StatusBadRequest {
t.Errorf("missing path: status = %d, want 400", rec.Code)
}
}
func TestThumbnailHandler_BadToken_404(t *testing.T) {
srv := NewStreamServer(0)
rec := httptest.NewRecorder()
// Path present (so we pass the 400 gate) but a bogus token → 404, no oracle.
srv.thumbnailHandler(rec, thumbReq("198.51.100.7:40000", "?p="+url.QueryEscape("/tmp/x.mkv")+"&t=deadbeef.0000"))
if rec.Code != http.StatusNotFound {
t.Errorf("bad token: status = %d, want 404", rec.Code)
}
}
func TestThumbnailHandler_ValidToken_NonexistentFile_404(t *testing.T) {
srv := NewStreamServer(0)
path := "/nonexistent/never-here.mkv"
tok := mintStreamToken(srv.streamSecret, streamScopeThumb(path), time.Now())
rec := httptest.NewRecorder()
srv.thumbnailHandler(rec, thumbReq("198.51.100.7:40000", "?p="+url.QueryEscape(path)+"&t="+tok))
if rec.Code != http.StatusNotFound {
t.Errorf("valid token but missing file: status = %d, want 404 (regular-file clamp)", rec.Code)
}
}
func TestThumbnailHandler_NoFFmpeg_503(t *testing.T) {
srv := NewStreamServer(0) // ffmpegPath left empty
dir := t.TempDir()
path := filepath.Join(dir, "movie.mkv")
if err := os.WriteFile(path, []byte("not really a video"), 0o600); err != nil {
t.Fatal(err)
}
tok := mintStreamToken(srv.streamSecret, streamScopeThumb(path), time.Now())
rec := httptest.NewRecorder()
srv.thumbnailHandler(rec, thumbReq("198.51.100.7:40000", "?p="+url.QueryEscape(path)+"&t="+tok))
if rec.Code != http.StatusServiceUnavailable {
t.Errorf("no ffmpeg configured: status = %d, want 503", rec.Code)
}
}
// TestThumbnailHandler_Success exercises the full success branch with a stub
// "ffmpeg" that writes JPEG magic bytes to stdout — no real ffmpeg/video
// needed. Validates 200 + image/jpeg + the body is passed through verbatim.
func TestThumbnailHandler_Success(t *testing.T) {
srv := NewStreamServer(0)
dir := t.TempDir()
path := filepath.Join(dir, "movie.mkv")
if err := os.WriteFile(path, []byte("x"), 0o600); err != nil {
t.Fatal(err)
}
stub := filepath.Join(dir, "ffmpeg.sh")
// JPEG SOI marker (FF D8 FF) + filler, regardless of args.
if err := os.WriteFile(stub, []byte("#!/bin/sh\nprintf '\\377\\330\\377stub'\n"), 0o755); err != nil {
t.Fatal(err)
}
srv.SetFFmpegPath(stub)
tok := mintStreamToken(srv.streamSecret, streamScopeThumb(path), time.Now())
rec := httptest.NewRecorder()
srv.thumbnailHandler(rec, thumbReq("198.51.100.7:40000",
"?p="+url.QueryEscape(path)+"&t="+tok+"&pos=10&w=200"))
if rec.Code != http.StatusOK {
t.Fatalf("stub ffmpeg: status = %d, want 200 (body=%q)", rec.Code, rec.Body.String())
}
if ct := rec.Header().Get("Content-Type"); ct != "image/jpeg" {
t.Errorf("Content-Type = %q, want image/jpeg", ct)
}
if !strings.HasPrefix(rec.Body.String(), "\xff\xd8\xff") {
t.Errorf("body missing JPEG magic bytes: %q", rec.Body.String())
}
}