Debrid direct links are time-limited; a long playback can outlive the link
the session was created with. When a debrid source dies mid-stream the daemon
now re-resolves a fresh link for the same content and resumes — no torrent
fallback, no playback restart.
- debridFileProvider holds the URL behind a mutex; on an expired-link status
(401/403/404/410) the ranged reader re-resolves via a refresh callback and
retries (bounded: 1 initial + 1 post-refresh attempt). A browser opens
several range connections, so the refresh is coalesced singleflight-style —
N readers hitting the dead link share ONE re-resolution, not N.
- HLS-from-URL: the auto-restart supervisor re-resolves the link before
relaunching ffmpeg (else it just retries the dead URL and burns the retry
budget). The mutable URL lives in s.liveURL under s.mu — restartFromSegment
reads it from the HTTP handler goroutine too (seek-restart), so cfg stays
immutable and the write races nothing.
- agentClient.RefreshStreamURL → POST /api/internal/agent/stream-url.
Cross-source torrent<->debrid swap (the rare "debrid genuinely gone" case) is
intentionally deferred. Reader refresh + coalescing covered by unit tests
(incl. -race); the web endpoint re-resolves against a real AllDebrid account.
Non-browser-native debrid content (mkv/HEVC/…) can now stream: ffmpeg reads
the debrid HTTPS link directly (-i <url>) and transcodes to HLS, instead of
2a's raw direct-play which only works for mp4/m4v.
- HLSSessionConfig gains SourceURL + CacheID; sourceRef() feeds ffprobe,
ffmpeg -i, and subtitle extraction from one place. HTTP-resilience flags
(-reconnect*, -rw_timeout) are added only for a URL source; a seek-restart
re-opens the URL with a Range request (-ss before -i = input seek).
- Segment cache keys by CacheID (the torrent info_hash) for URL sessions so
re-plays hit cache despite the debrid URL changing each resolution
(KeyForID, no filepath.Abs).
- OnStreamSession: the 2a direct-play branch is now gated on PlayMethod != "hls";
a new branch handles DirectURL + PlayMethod=="hls" → HLS-from-URL. The
local-file and both debrid HLS paths share a startHLSPlayback helper.
- ExtractMediaInfo no longer masks a URL probe failure as "file not found"
(surfaces ffprobe's real stderr, e.g. "Protocol not found" on a TLS-less
ffmpeg build).
- Bump 0.11.0 -> 0.12.0 as the HLS-from-URL floor the web gates on.
Validated e2e against real AllDebrid: a cached HEVC x265 mkv transcodes
(h264_nvenc) from the debrid URL and plays 1080p in Chrome via hls.js,
subtitles extracted from the remote mkv.
The daemon can now stream a session straight from a server-resolved debrid
direct URL instead of disk/torrent, delivering the "play instantáneo
cache-fast" promise for cache-confirmed torrents the user never downloaded.
- debridFileProvider: an io.ReadSeekCloser over HTTP Range — network-free
Seek, lazy GET on Read, reopen-on-seek, a HEAD up front for the size, and
a URL-derived name so the served Content-Type is video/mp4 (not
octet-stream) when the web's name lacks an extension.
- OnStreamSession branches on StreamSession.DirectURL before the filePath
checks (no local path, no ffmpeg), builds the provider in a goroutine
(HEAD off the sync loop) and marks the session ready.
- Bump 0.10.0 -> 0.11.0 as the debrid-stream floor the web gates on.
Validated e2e against a real AllDebrid account: a cached mp4 plays 1080p in
Chrome through the agent, including the high-offset seek for a non-faststart
file's moov atom. 2b (HLS-from-URL for mkv/HEVC) + 2c (cache-fast preference
+ mid-stream fallback) remain.
The daemon's baked-in CORS allowlist had the torrentclaw.com family but not
unarr.app — so on the unarr brand the browser dropped every /hls + /stream
response (no Access-Control-Allow-Origin) and the player reported "can't
connect to your agent" even though the agent was reachable. Add unarr.app +
www.unarr.app. (Dev over Tailscale uses cors_extra_origins for the raw IP
origin.) Found while testing the web player from an iPhone over Tailscale.
Hueco #3 / 3c (CLI). NewRemuxSource now copies the video for any
browser-decodable codec: h264, or HEVC/AV1 when the web says the device
decodes them (caps). HEVC is muxed with -tag:v hvc1 (Apple requirement),
and non-aac audio (ac3/eac3/dts) is transcoded to aac while the video is
still copied (ActionRemuxAudio) — this covers the very common h264+ac3 mkv.
Startup instrumentation for time-to-first-frame diagnosis:
- remux branch logs [probe=.. spawn=..]
- transcodeSource logs 'first fMP4 bytes after ..' (ffmpeg → first output)
- serveGrowing logs reads that block >250ms (client seeking ahead of the
live edge) + the first read's offset vs produced/estimated size.
Verified: caps gate (hls without caps, remux with), hvc1 retag (ffprobe of
the /stream output = hevc/hvc1), HEVC playback confirmed on a real iPhone
Safari over Tailscale. LAN timeline: probe 16ms, spawn 1ms, first byte
201ms, no serveGrowing blocks.
Agent side of 3b: serve a growing ffmpeg `-c copy` remux (mkv h264/aac →
fragmented MP4) over /stream with no video re-encode. Dormant until the web
sends PlayMethod="remux" (3b-ii), so this commit changes no live behavior.
- GrowingSource interface + transcodeSource already satisfies it; estimate is
the source file size for copy actions (≈ remux output) vs bitrate×duration
for real transcodes.
- NewRemuxSource: ffmpeg -c copy → growing fMP4 temp, returned as GrowingSource.
- StreamServer.SetGrowingFile + serveGrowing: manual Range responder for a
growing source (http.ServeContent needs a fixed size). 206 with an estimated
total in Content-Range; chunked body while not final (never promise bytes a
running remux might not produce); exact Content-Length once final. Blocks via
ReadAt for not-yet-produced bytes; forward seek waits, backward seek instant.
- daemon OnStreamSession: PlayMethod=="remux" → NewRemuxSource + SetGrowingFile
+ MarkSessionReady (after the ffmpeg check; copy still needs ffmpeg).
- Tests: parseByteRange + serveGrowing (full/offset/bounded/estimate/HEAD/416).
Hueco #3 / 3a (CLI side). StreamSession gains PlayMethod; when the web
sends "direct", the daemon serves the raw file over /stream (HTTP Range,
no ffmpeg) instead of transcoding to HLS — zero CPU, instant seek. Runs
before the ffmpeg-availability check so direct-play works even with
transcode disabled. Legacy/empty PlayMethod keeps the HLS path, so an old
web that never sends "direct" is unaffected.
/stream and /hls were served with no auth (only CORS + rate limit), so a
funnel- or UPnP-exposed daemon leaked active downloads to anyone with the URL.
Bind a short-lived HMAC token (scope + 6h expiry) to every stream URL the
daemon hands out and verify it on each request:
- /stream + VLC playlist: ?t= query, agent-minted, scope "stream"
- /hls: path segment /hls/<session>/<token>/<resource>, web-minted with the
agent's reported secret, scope "hls:<session>" — relative playlist URIs
inherit it with no rewriting
- NO loopback exemption: cloudflared relays public funnel traffic over
localhost, so a loopback source address is not a trust signal
- the agent reports its per-run signing key on register only when enforcing
- require_stream_token config (default true); secret fails hard if rand fails
- /playlist.m3u no longer self-mints a token (was an open token oracle)
Roadmap: Docs/plans/unarr-agent-roadmap.md (hueco #1).
Deploy the web HLS-minting change BEFORE shipping this agent release.
funnel: urlPattern matched api.trycloudflare.com before the real quick-tunnel
URL. Cloudflared logs the control-plane endpoint early, so the agent was
advertising a dead URL. Tighten regex to require at least one hyphen — quick
tunnels are always multi-word (e.g. make-appointments-negotiation-blacks).
Covers with funnel_test.go regression test.
download(oneshot): progress reporter called /api/internal/agent/status with a
synthetic "oneshot-<hash>" task ID that is not a UUID, causing the server to
return 400 every 5 s for the entire download. Pass nil client to
NewProgressReporter for one-shot mode; flush/ReportFinal are no-ops when
reporter == nil so terminal output continues unchanged.
torrent: piece-completion SQLite DB (anacrolix) was created inside the download
dir (DataDir). On NFS/SMB mounts SQLite file locking times out, emitting a
warning and falling back to an ephemeral in-memory DB. Add PieceCompletionDir
to TorrentConfig; the daemon now passes config.DataDir() (agent state dir,
always local) so the DB stays off the network mount. One-shot download leaves
the field empty → harmless in-memory fallback as before.
anacrolix mmap storage (storage.NewMMap) creates completed files with
mode 0000. The download succeeds because the agent keeps its own mmap
handle, but any fresh open — direct streaming (/stream :11818), HLS
ffprobe (:11819), or organize-then-reopen — fails with "permission
denied", surfaced in the web UI as "file not found". Both VLC and the
web player were affected.
makeReadable() relaxes the completed file to 0644 (dirs 0755, recursive
for multi-file torrents) right after download finishes, before organize
moves it, so the readable mode survives the rename.
With `-tune ll` NVENC emits long IDR-less GOPs that ignore
`-force_key_frames`, so ffmpeg's HLS muxer keeps writing into seg-0.m4s
forever instead of closing it at the 2 s boundary. Result:
* seg-0.m4s balloons to the full encoded size (1.2 GB on a 48-min movie)
* seg-1.m4s never appears
* daemon's pollSegments needs seg-N+1 to confirm seg-N is closed → never
advances → `mark-ready: timeout` after 60 s
* web player sits on "preparando sesión" until the user gives up
Verified on ffmpeg 6.1.1 + driver 580 / Ryzen 7 7700X + RTX-class GPU:
without `-tune ll`, the same `-preset p3 -rc vbr` cmd produces 39
discrete segments in 15 s at ~27x real-time (was 1 segment / 9 min of
material with `-tune ll` — encoder kept going on a single output).
Introduced by `3b8d77b feat(hls): faster first-start — probe cache +
tighter encoder presets (0.9.9)`. Dropping `-tune ll` costs ~0.5 dB
PSNR at the same bitrate but restores playback. NVENC first-segment
latency remains under 2 s — well within the player's startup budget.
Adds step 6 to scripts/ship.sh: after smoke checks, list Forgejo
releases and delete any with created_at older than FORGEJO_PRUNE_DAYS
(default 90). Bounded retention prevents the tc-git CPX11 disk from
filling up (each release ≈ 511MB of attachments × 1/week pace).
Skipped silently with a warn if FORGEJO_TOKEN is not exported, so
the step is opt-in via secret presence (no token = no destructive
action). Tunables: FORGEJO_PRUNE_DAYS, FORGEJO_REPO, FORGEJO_BASE,
SKIP_FORGEJO_PRUNE.
Two new pre-flight guards in scripts/release.sh, evaluated right after the
branch check:
1. Reject if HEAD subject matches `(X.Y.Z)` — historical pattern where the
feature commit itself bumped the version (e.g. `feat(...) (0.9.14)`).
Forces every release to land in a dedicated `chore(release): X.Y.Z`
commit so the changelog + tag point at a clean release boundary.
2. Reject if HEAD is already `chore(release): …` — prevents re-running the
script with no new commits since the previous release (would otherwise
produce an empty release on top of itself).
Scope deliberately `chore(scripts)` (not `chore(release)`) so this very
commit doesn't trip guard 2 the next time release.sh runs.
Old copy claimed unarr was a "torrent search" tool. unarr's real job is
downloading (torrent + debrid + usenet), streaming via local HLS, transcoding
with ffmpeg+HW accel, and library management. Search just queries the
torrentclaw.com catalog — secondary feature, not the identity.
- root cobra Short/Long now lead with download/stream/transcode and list the
three backends + WireGuard + Cloudflare Funnel
- README hero + subheading mirror the same positioning
- DOCKERHUB hero updated to match
- "Search & Discovery" group → "Catalog & Discovery" (search still grouped,
but framed as catalog browsing not product identity)
Forgejo runner auto-injects GITHUB_TOKEN; combined with the GITEA_TOKEN we
set explicitly, goreleaser errors with 'multiple tokens'. Unset the GitHub
one inside the run step so goreleaser follows the Gitea/Forgejo release
path defined by .goreleaser.yml's gitea_urls block.
Adds TestBuildHLSFFmpegArgsVAAPIDump alongside the existing assertion
tests. Logs the complete argv buildHLSFFmpegArgsAt emits for a
typical VAAPI session so an operator can paste it into a shell and
reproduce the encode without booting the dev stack — same effect as
`journalctl --user -u unarr-dev | grep ffmpeg`, no daemon needed.
Verified locally against AMD Raphael iGPU on this dev box: the
dumped argv encoded a 5 s 4K source → 720p in 3.1 s wall, produced
3 HLS segments + init.mp4 that decode cleanly under ffprobe.
goreleaser v2 dropped `release.gitea_urls`; the key is now top-level
on its own. With the old nested form `goreleaser release` failed with
`yaml: unmarshal errors: line 67: field gitea_urls not found in type
config.Release` before even starting the build.
Re-anchor to v0.9.14 so the ship pipeline can produce binaries.
Closes QW2. Validated against the dev box's AMD Raphael iGPU
(/dev/dri/renderD128, radeonsi/mesa 25.2.8). The "proper" full-GPU
path via scale_vaapi triggers a known mesa 25 + Raphael bug
("Cannot allocate memory" per session start, encode still succeeds
but logs are spammy) — hybrid CPU scale → format=nv12 → hwupload
→ h264_vaapi encode delivers GPU surfaces to the encoder without
poking the broken scaler.
Three concrete changes in buildHLSFFmpegArgsAt:
1. New `case "h264_vaapi"` adds `-vaapi_device /dev/dri/renderD128`.
Multi-GPU hosts (this dev box has NVIDIA on renderD129 + AMD on
renderD128) need it so the encoder doesn't bind to a non-VAAPI
render node — without it the encoder fell back to NULL device
in manual smoke testing.
2. Filter chain branches on codec: VAAPI uses
`scale=…,format=nv12,hwupload` while libx264 / NVENC / QSV
keep the existing `scale=…,format=yuv420p,setparams=…` shape.
The setparams color metadata block is dropped on VAAPI because
VAAPI surfaces don't expose VUI fields and the encoder writes
its own.
3. Two new unit tests lock the argv shape so a future refactor
doesn't accidentally merge the paths back together:
TestBuildHLSFFmpegArgsVAAPI asserts the new flags + the
ABSENCE of scale_vaapi; TestBuildHLSFFmpegArgsLibx264NoRegression
verifies the software path keeps yuv420p + setparams + has
none of the VAAPI extras.
Manual ffmpeg validation on the dev box:
hybrid encode of 5 s 4K → 720p: 0.66 s wall, 472 % CPU, 268 KB
output — no errors logged. scale_vaapi variant in comparison
spammed "Cannot allocate memory" while emitting valid output.
GitHub torrentclaw org is shadow-banned and the CI lives at git.torrentclaw.com
now. Forgejo Actions is enabled cluster-wide; this moves the workflows into the
runner's natively-watched .forgejo/workflows/ tree and adapts each step so the
existing Forgejo runner ('docker', 'ubuntu-latest' labels) can execute them
without leaning on GitHub-only tooling.
- ci.yml: drop actions/setup-go (use container: golang:1.25), replace
golangci-lint-action with the upstream install.sh, drop codecov-action
(third-party, can re-add later with a Forgejo-compatible variant).
- release.yml: drop goreleaser-action (install via curl), wire GITEA_TOKEN +
the new release.gitea_urls block in .goreleaser.yml so goreleaser publishes
to Forgejo. Sign step swaps 'gh release upload' for curl against the Forgejo
releases API (via the in-cluster forgejo:3000 hostname). VirusTotal job
dropped — depended heavily on 'gh release' wiring; can be reimplemented
against the Forgejo API later if we re-enable it.
- docker-rebuild.yml: drop docker/login-action + docker/build-push-action,
use raw 'docker' commands with manually-installed buildx + qemu. Same
weekly schedule (Mon 04:17 UTC) and same 'latest' refresh behaviour.
- pages.yml: deleted — install.sh / install.ps1 are already served from the
Hetzner releases volume at torrentclaw.com/install.sh, so the GitHub Pages
copy was redundant even before the shadow-ban.
.goreleaser.yml: add release.gitea_urls (api=forgejo:3000, download via the
public Forgejo URL) + prerelease:auto. ship.sh uses '--skip=publish' so local
runs aren't affected by the new release block.
Closes the deferred bajo-priority item from the fase 3.3b critico.
Without this the watcher kept polling a torn-down HLSSession for up
to 60 s — fine in current code paths (Close always pairs with ctx
cancel which makes the select{} branch fire), but the function's
correctness then leaned on a caller invariant rather than its own
state check. Adding IsClosed() as a public wrapper around the
existing isClosed() lets the watcher detect any future
session-shutdown path (registry replace, idle sweep, internal kill)
without touching the unexported helper.
Critico flag: rctx was rooted at context.Background() instead of the
session's hlsCtx, so a tab close / session cancel mid-POST left the
goroutine blocking on the in-flight webhook for up to 10 s. Switched
to a child of hlsCtx — the same scope the watchSessionReady loop
already respects via the outer ctx.Done() select.
Idempotent webhook means a now-orphan session getting marked ready
is cosmetic; the savings here are goroutine pinning + a slow webhook
on a torn-down session.
Closes Fase 3.3b. Daemon now tells the server the moment a session's
first HLS segment + init.mp4 land on disk; the web side flips
streaming_session.ready_at = NOW(), which its SSE endpoint pushes to
subscribed players so the loading UI flips from "Preparando…" to
"Stream listo" without polling HEAD on the segment URL.
Surface:
- New Client.MarkSessionReady(ctx, sessionId) HTTP method →
POST /api/internal/agent/session-ready.
- New engine.HLSSession.ReadyCount() + FromCache() accessors so the
watcher goroutine doesn't reach into private state.
- New cmd.watchSessionReady(ctx, client, hsess, sessionId) goroutine
polls ReadyCount every 200 ms with a 60 s deadline + short-circuits
for cache-HIT sessions (ready the moment StartHLSSession returns).
- Daemon callback spawns it right after streamSrv.HLS().Register so
the watcher's lifecycle matches the session's.
Best-effort: a transient network failure on the webhook is logged + the
goroutine exits — the player's existing HEAD-probe retry path still
discovers ready state independently. The webhook is an acceleration,
not a hard dependency.
DetectHWAccelDiagnostic spawns subprocess calls; an unexpected panic
(broken ffmpeg binary, OOM mid-exec) would otherwise leave the
WithTimeout context dangling until natural expiry. defer keeps the
goroutine + timer reachable until runDaemonStart returns, but on a
long-lived daemon that's the process lifetime anyway — same effective
cost, with the safety guarantee.
Daemon now runs engine.DetectHWAccelDiagnostic at startup (instead of the
lighter DetectHWAccel) and ships the full picture — ffmpeg version,
resolved binary path, HW encoders compiled in, device files / drivers
detected — up to the server in the RegisterRequest payload.
Why: the most common cause of slow first-play is a software-only ffmpeg
build. Surfacing the diagnostic in the web AgentsTab "Diagnose
transcoder" modal lets a user see *why* their backend landed on libx264
(e.g. brew's default formula ships without --enable-nvenc, or the
container is missing /dev/nvidia0) without SSHing in to run `unarr
probe-hwaccel` manually.
Also emits a single `[transcode]` startup log line summarising the same
data — convenient for `journalctl --user -u unarr | grep transcode`.
Bounded by a 10 s context so a hung ffmpeg binary can't stall daemon
startup forever.
Mirrors the slash command added in torrentclaw-web/.claude/commands.
With the global ~/.gitignore excluding .claude/ by default, the
gitignore override is required for project-shared commands/agents/hooks
to be checked in (settings.local.json and projects/ stay local).
/publish documents the full unarr release flow (bump + tag + binaries +
Hetzner + Docker Hub + smoke) as a single command, while GitHub Actions
remains unavailable for the torrentclaw org.
GitHub Actions release.yml + docker job currently doesn't fire (org
shadow-ban). ship.sh replicates the CI pipeline locally so releases
keep landing on Hetzner + Docker Hub without depending on CI:
1. Sanity checks: clean tree, tag at HEAD, version.go match
2. goreleaser release --skip=publish (build dist/*)
3. publish-cli-release.sh (rsync to Hetzner + flip version.txt)
4. docker buildx --push multi-arch (amd64 + arm64)
5. Smoke: torrentclaw.com/version + docker run image version
6. Optional --push to git-push tag to GH
Exposed via make targets: ship, ship-dry, ship-push.
Cinematic widescreen content (1920×804 at 2.39:1, 3840×1600 21:9, etc.)
was being misclassified: a 1080p source presented as 1920×804 fell to
720p because 804 < 900. Same shape for 2160p sources letterboxed below
2000px tall.
ResolveResolution now takes (width, height) and picks the larger of the
width-derived and height-derived buckets, so anamorphic/letterboxed
sources land in the right bucket.
First-frame latency drops by another 1-2 s on cold-cache plays:
1. HLS segment duration halved from 4 s to 2 s. seg-0 lands in ~half
the wait time — the player paints the first frame as soon as it
arrives. Software encodes on 4K go from ~3 s wait to ~1.5 s; HW
encoders shave ~0.5 s. Trade-off: 2× segment count per source
(~3600 segments for a 2 h movie instead of ~1800), but each is
half the size on disk. Within HLS spec — Apple recommends 6 s, but
2 s is valid; LL-HLS uses 1-2 s.
2. Cache from 0.9.9 self-heals: cached entries used 4 s segments;
VerifyComplete now expects a different highest segment index and
invalidates them, triggering a re-encode on next play. No manual
cleanup needed.
3. OnStreamSession daemon callback now runs StartHLSSession in a
goroutine. Sync HTTP responses return immediately (~50 ms instead
of waiting for the ~0.3-1 s ffprobe). Other pending actions in
the same sync cycle (new tasks, deletes) no longer wait for the
transcoder warmup. Browser HEAD probes already have a 30 s retry
budget that covers the brief gap between playerSessionRegistry.add
and streamSrv.HLS().Register.
Helpers added (engine.segmentDurationFor / segmentStartSec /
segmentCountForDuration) so a future short-first-segment variant or
non-uniform layout can slot in without touching every call site.
Internal: -hls_init_time was investigated but discarded — ffmpeg's
implementation treats it as a min duration, not a target, so it
couldn't deliver a uniformly 2 s first segment on top of a 4 s
steady state. Uniform 2 s is simpler and gets the same first-frame
win.
