# Images

Render real images — PNG, JPEG, GIF, WebP, BMP — inside a TUI. The same model code displays Kitty graphics in a Kitty terminal, halfblocks in Livebook, and adapts gracefully when the audience's terminal support is unknown.

Built on [ratatui-image](https://github.com/ratatui/ratatui-image).

## Quick start

```elixir
def mount(_opts) do
  {:ok, picture} = ExRatatui.Image.new(File.read!("priv/slides/cover.png"))
  {:ok, %{picture: picture}}
end

def render(state, frame) do
  area = %Rect{x: 0, y: 0, width: frame.width, height: frame.height}
  [{state.picture, area}]
end
```

`ExRatatui.Image.new/2` decodes once and returns a stateful widget handle. Hold it in the model and reuse it across renders — re-encoding only happens when the protocol changes or the rect resizes. Image format is auto-detected from the bytes; no extension or content-type hint required.

Bad bytes return `{:error, {:decode_failed, message}}` rather than raising, so a placeholder can be rendered gracefully.

## Options

`ExRatatui.Image.new/2` accepts a keyword list:

| Option | Values | Default | What it does |
|---|---|---|---|
| `:protocol` | `:auto`, `:halfblocks`, `:kitty`, `:sixel`, `:iterm2` | `:auto` | Which terminal protocol to render with. `:auto` resolves at render time against the transport (see [resolution table](#protocol-resolution)). Explicit protocols are honored except over `CellSession` where `:halfblocks` is forced. |
| `:resize` | `:fit`, `:crop`, `:scale` | `:fit` | `:fit` preserves aspect inside the rect (anchored top-left if smaller). `:crop` preserves aspect, fills the rect, crops the overflow. `:scale` stretches to fill (no aspect preservation). |
| `:background` | `nil`, any `t:ExRatatui.Style.color/0`, or raw `{r, g, b}` | `nil` | Color used to fill transparent pixels / unused area for halfblocks. |

To change options later, build a new handle — the widget struct just wraps the resource ref.

## Protocol resolution

Each transport stamps its own capability hint. Widget-level `:auto` resolves against it:

| Transport | `:auto` resolves to | Explicit `:kitty` etc. |
|---|---|---|
| Local terminal (no probe) | `:halfblocks` | Honored — emits raw escapes |
| Local terminal (after probe) | The detected protocol | Honored |
| SSH (`image_protocol: :kitty`) | `:kitty` | Honored, default `(8, 16)` font size |
| SSH (`image_protocol: :kitty` + `image_font_size: {10, 20}`) | `:kitty` | Honored with accurate scaling |
| SSH (no opts) | `:halfblocks` | Honored |
| Distributed (`image_protocol: :kitty`) | `:kitty` | Honored, default `(8, 16)` font size |
| Distributed (`image_protocol: :kitty` + `image_font_size: {10, 20}`) | `:kitty` | Honored with accurate scaling |
| Distributed (no opts) | `:halfblocks` | Honored |
| `CellSession` (Livebook / Kino) | `:halfblocks` | **Forced to `:halfblocks`** (escape sequences can't survive cell diffing) |

This means **the same model code is portable**: a slide deck that renders pixel-perfect Kitty graphics in a local Kitty terminal will silently fall back to halfblocks when the same `ExRatatui.App` is driven from a Livebook cell — no branching.

Image widgets work over **every** transport, including `ExRatatui.Distributed` — image bytes are snapshotted into the render tree, costing roughly the source file size per frame on the wire. Fine for stills; watch the bandwidth when animating large images.

### Probing the local terminal

`ExRatatui.Image.auto_local_protocol/1` writes a query escape sequence and waits for the terminal's reply. On success the result is cached on the terminal reference; on no-TTY / no-reply the cache stays empty and `:auto` falls back to halfblocks. Either way it's safe to call from any environment.

There are two ways to wire it in.

**Inside `ExRatatui.App`:** return `probe_image_protocol: true` from `mount/1`. The runtime calls `auto_local_protocol/1` right after mount, on the `:local` transport only (CellSession forces halfblocks; SSH / Distributed use the session-level `:image_protocol` opt instead):

```elixir
@impl true
def mount(_opts) do
  {:ok, initial_state, probe_image_protocol: true}
end
```

The probe is automatically skipped under `test_mode: {w, h}` so headless tests don't accidentally write probe escapes.

**Outside `ExRatatui.App`** (e.g. `ExRatatui.run/1`):

```elixir
ExRatatui.run(fn terminal ->
  ExRatatui.Image.auto_local_protocol(terminal)
  # ... rest of the app
end)
```

To make an independent decision based on the probe, `ExRatatui.Image.probe_terminal/0` returns `{:ok, %{protocol: atom, font_size: {w, h}}}` or `{:error, reason}` without touching any cache.

### Telling SSH / Distributed what protocol the client supports

An SSH or Distributed client terminal can't be probed, so the audience declares it at start time. Pair `:image_protocol` with `:image_font_size` to get accurate Kitty/Sixel/iTerm2 scaling — without the font size, the encoder falls back to `(8, 16)` cell pixels which mis-scales on most modern terminals (Kitty/Ghostty are closer to `(10, 20)`):

```elixir
# SSH daemon
ExRatatui.SSH.Daemon.start_link(
  mod: MyApp.TUI,
  port: 2222,
  image_protocol: :kitty,
  image_font_size: {10, 20}
)

# Distributed attach
ExRatatui.Distributed.attach(:"app@host", MyApp.TUI,
  image_protocol: :kitty,
  image_font_size: {10, 20}
)
```

Per-image explicit choices (`ExRatatui.Image.new(bytes, protocol: :sixel)`) are always honored, regardless of the session-level hint.

## Font-size caveat

Cells aren't pixels. The render pipeline needs the terminal's cell-pixel dimensions to scale Kitty / Sixel / iTerm2 payloads correctly. The default is `(8, 16)`; `auto_local_protocol/1` replaces it with the real value reported by the terminal. If Kitty graphics look mis-scaled, run the probe.

## Examples

* [`examples/images/image_demo.exs`](https://github.com/mcass19/ex_ratatui/blob/main/examples/images/image_demo.exs) — interactive viewer with `p` to cycle protocol, `r` to cycle resize mode, and a live status panel showing the render output dimensions. Runs on every transport via the same script:

  ```sh
  mix run examples/images/image_demo.exs                # local terminal
  mix run --no-halt examples/images/image_demo.exs --ssh
  elixir --sname app --cookie demo -S mix run --no-halt \
    examples/images/image_demo.exs --distributed
  ```

* [`examples/cell_session/headless_image.exs`](https://github.com/mcass19/ex_ratatui/blob/main/examples/cell_session/headless_image.exs) — fetch a photo, render through `CellSession`, dump the cell grid to stdout with ANSI fg/bg colors. The Livebook / Kino path; safe to run anywhere (no TTY required).

Both accept an `IMAGE_PATH` env var, default to fetching from `picsum.photos` once at startup, and fall back to an embedded 1×1 PNG if the network is unreachable. The SSH demo also honors `IMAGE_PROTOCOL` / `IMAGE_FONT_W` / `IMAGE_FONT_H` env vars.

## Telemetry

Each `ExRatatui.Image.new/2` call emits a `[:ex_ratatui, :image, :decode]` span:

* `:start` — `%{format: atom, bytes: non_neg_integer}` (format sniffed from magic bytes: `:png`, `:jpeg`, `:gif`, `:webp`, `:bmp`, or `:unknown`).
* `:stop` — adds `:width` and `:height` on success, or `:error` (reason) on failure.

Per-render encode timing (Kitty / Sixel / iTerm2 payload generation) is rolled into the existing `[:ex_ratatui, :render, :frame]` span — they happen inside the same NIF render pass.

## `:fit` and `:crop` do not upscale — this is intentional

This is the single most common point of confusion, so it gets its own section.

Both `:fit` and `:crop` clamp output to the **source image's natural pixel size**. They never upscale. Given a 400×300 picsum photo in a render area sized 800×500 target pixels:

| Mode | Render output | Visible result |
|---|---|---|
| `:fit` | 400×300 | Image at natural size, anchored top-left, ~50% empty area |
| `:crop` | 400×300 | Identical to `:fit` here — both clamp to source dimensions |
| `:scale` | ~640×480 (aspect-preserving fill) | Image fills the area |

The difference between `:fit` and `:crop` only manifests when the **source is *larger* than the target** on at least one axis: `:fit` shrinks to fit (whole image visible, letterboxed); `:crop` keeps natural size and shows a window into the source corner.

This is upstream ratatui-image behavior, not something we layer on. We expose `ExRatatui.Image.render_size/4` to predict what each mode will do for a given combination of source dims, cell area, and font size — useful for status panels, layout decisions, or just understanding what's on screen. The `examples/images/image_demo.exs` example uses it to surface the render output dimensions live while cycling modes.

For "fill the area regardless of source size," use `:scale`. That's the only mode that upscales.

## Known limitations (v1)

* **No animated GIFs.** First frame only. Frame-by-frame animation needs render-loop integration that isn't here yet.
* **No SVG.** The underlying `image` crate doesn't include an SVG decoder.
* **No streaming / progressive decode.** Bytes are decoded all at once at `new/2`.