# 3D Rendering

Render real 3D scenes — lit meshes, a movable camera, software rasterization or ray tracing — inside a TUI. The same model code draws into any true-color terminal, blitting pixels to cells with half blocks, supersampled half blocks, or an ASCII ramp.

Built on [ratatui-3d](https://github.com/limlabs/ratatui-3d), vendored in-tree.

> #### True-color required {: .info}
>
> Every blit mode emits 24-bit colors. A terminal without true-color support renders the scene as muddy approximations or not at all.

## Quick start

```elixir
alias ExRatatui.ThreeD.{Camera, Light, Material, Mesh, Object, Scene}
alias ExRatatui.Widgets.Viewport3D

def render(_state, frame) do
  area = %ExRatatui.Layout.Rect{x: 0, y: 0, width: frame.width, height: frame.height}

  viewport = %Viewport3D{
    scene: %Scene{
      objects: [%Object{mesh: Mesh.cube(), material: %Material{color: {100, 150, 255}}}],
      lights: [
        Light.ambient({255, 255, 255}, 0.15),
        Light.directional({-1.0, -1.0, -1.0}, {255, 255, 255})
      ]
    },
    camera: %Camera{position: {3.0, 2.5, 4.0}, target: {0.0, 0.0, 0.0}}
  }

  [{viewport, area}]
end
```

`Viewport3D` is pure data, like every other widget: the scene and camera are rebuilt and rendered every frame. Animation and camera movement live in the model between frames — see the camera recipe below.

## The scene

A `ExRatatui.ThreeD.Scene` is a flat list of objects and lights with a background:

```elixir
%Scene{
  objects: [...],          # list of %Object{}
  lights: [...],           # list of %Light{}
  background: {8, 8, 16},  # {r, g, b}, channels 0-255
  sky: nil                 # optional gradient, sampled by the raytrace pipeline
}
```

The scene itself is flat: every object carries its own world-space transform. Hierarchies (such as an articulated model) are composed before building the scene — by hand, or with the `ExRatatui.ThreeD.Node` scene-graph helper described in [Scene graph](#scene-graph).

Colors are `{r, g, b}` integer tuples (0-255). Vectors are `{x, y, z}` float tuples.

## Objects, materials, transforms

```elixir
%Object{
  mesh: Mesh.cube(),
  material: %Material{color: {200, 80, 80}, diffuse: 0.8, specular: 0.5, shininess: 32.0},
  transform: %ExRatatui.ThreeD.Transform{
    position: {1.0, 0.0, 0.0},
    rotation: {:euler_xyz, {0.0, 0.7, 0.0}},
    scale: {1.0, 1.0, 1.0}
  },
  visible: true
}
```

`ExRatatui.ThreeD.Material` is a Phong material: a base `color` plus `ambient`, `diffuse`, `specular`, and `shininess` coefficients.

Rotation accepts three forms:

| Form | Meaning |
|---|---|
| `{:euler_xyz, {rx, ry, rz}}` | intrinsic X then Y then Z, in radians |
| `{:axis_angle, {ax, ay, az}, angle}` | `angle` radians about a non-zero axis |
| `{:quat, {x, y, z, w}}` | a raw quaternion |

## Meshes

Built-in primitives carry no geometry data:

```elixir
Mesh.cube()
Mesh.sphere(16, 24)   # stacks, slices
Mesh.plane()
Mesh.cylinder(24)     # radial segments; radius 0.5, height 1.0, axis +Y
```

Custom meshes carry vertices and a flat list of triangle indices. Normals are computed natively when omitted:

```elixir
Mesh.new(
  [{-0.5, -0.5, 0.0}, {0.5, -0.5, 0.0}, {0.0, 0.7, 0.0}],
  [0, 1, 2]
)

# with explicit normals and texture coordinates:
Mesh.new(vertices, indices, normals: normals, uvs: uvs)
```

Triangles are wound counter-clockwise when viewed from the front; back faces are culled.

## Scene graph

`ExRatatui.ThreeD.Node` builds an articulated model as a tree of local transforms and bakes it into the flat scene. A node has a local `transform`, an optional `visual` object rendered at that node, and `children`:

```elixir
alias ExRatatui.ThreeD.{Node, Object, Mesh, Transform}

arm =
  %Node{
    transform: %Transform{rotation: {:axis_angle, {0.0, 1.0, 0.0}, base_angle}},
    visual: %Object{mesh: Mesh.cylinder(24), transform: %Transform{scale: {0.5, 0.3, 0.5}}},
    children: [
      %Node{
        transform: %Transform{position: {0.0, 0.3, 0.0}, rotation: {:axis_angle, {0.0, 0.0, 1.0}, elbow}},
        visual: %Object{mesh: Mesh.cube(), transform: %Transform{position: {0.0, 0.45, 0.0}, scale: {0.18, 0.9, 0.18}}}
      }
    ]
  }

scene = Node.to_scene(arm, lights: lights, background: {10, 12, 18})
```

`ExRatatui.ThreeD.Node.flatten/1` returns the world-space objects; `ExRatatui.ThreeD.Node.to_scene/2` wraps them with `:lights`, `:background`, and `:sky`. Frames compose via `ExRatatui.ThreeD.Transform.compose/2`.

**Contract:** keep intermediate nodes rigid (`scale: {1.0, 1.0, 1.0}`) and put scale only on leaf `visual` objects. Then every baked object is exactly one `Transform` with no shear. A non-uniform scale on a node that has rotated children is unsupported (the result is approximate).

## Render modes and pipelines

```elixir
%Viewport3D{render_mode: :auto, pipeline: :rasterize}
```

`render_mode` chooses how the render reaches the terminal. **Pixel-graphics modes** render the scene as an image at native terminal resolution (crisp, non-blocky) on capable terminals:

| `:render_mode` | Notes |
|---|---|
| `:auto` (default) | best protocol the terminal supports, else falls back to `:braille` |
| `:kitty` | Kitty graphics protocol (Ghostty, WezTerm, Kitty) |
| `:sixel` | Sixel graphics (WezTerm and others) |
| `:iterm2` | iTerm2 inline images |

Pixel modes engage only on a real terminal that probes as graphics-capable. Under `ExRatatui.App` on the `:local` transport, return `probe_image_protocol: true` from `mount/1` (or `init/1`) so the runtime runs the capability probe right after startup — otherwise `:auto` cannot detect the protocol and the scene renders through the `:braille` fallback (and at a default font size, not scaled to the pane):

```elixir
def init(_opts), do: {:ok, %{...}, probe_image_protocol: true}
```

Over `CellSession`/Livebook, distributed/SSH without graphics passthrough, or unsupported terminals, pixel modes **fall back to `:braille`**. **Cell-blit modes** pack the render into character cells and always work:

| `:render_mode` | Pixels per cell | Notes |
|---|---|---|
| `:half_block` | 1x2 | one `▀`, fg/bg are the upper/lower pixel |
| `:braille` | supersampled | anti-aliased `▀`; the cell-mode fallback for pixel modes |
| `:ascii` | 1x2 | a shaded ASCII ramp with colored characters |

> #### Continuous animation flickers in pixel modes {: .tip}
>
> Pixel-graphics modes re-transmit the whole image every frame, so a scene that animates continuously (a `Subscription.interval` spinning a mesh) can visibly flicker on some terminals — the image flashes off between deleting the previous frame and placing the new one. This is inherent to terminal image protocols, which target mostly-static images, not animation at frame rate. The cell-blit modes (`:braille`, `:half_block`) diff through ratatui normally and stay flicker-free, so they are the better choice for continuous motion; pixel graphics shine for static or interactive scenes, where frames only change on input. The examples default to `:auto` and bind `m` to cycle modes — press it to drop to `:braille` if motion flickers.

| `:pipeline` | Notes |
|---|---|
| `:rasterize` (default) | scanline rasterizer with depth buffering; fast |
| `:raytrace` | CPU ray tracing with shadows; slower, more realistic |

`pipeline` is orthogonal to `render_mode` — both pixel and cell modes can rasterize or ray-trace. The GPU pipeline upstream is intentionally not exposed: a GPU context inside a NIF can take down the runtime.

## Lights

```elixir
Light.ambient({255, 255, 255}, 0.15)
Light.directional({-1.0, -1.0, -1.0}, {255, 255, 255})
Light.point({2.0, 3.0, 2.0}, {255, 220, 180})
```

`directional/3` and `point/3` accept an `:intensity` option (default `1.0`).

## Driving the camera

`ExRatatui.ThreeD.Camera` is data; `orbit/3` and `zoom/2` are pure helpers that return a new camera. Keep the camera in the model and update it from key events:

```elixir
def handle_event(%Event.Key{code: "left", kind: "press"}, state),
  do: {:noreply, %{state | camera: Camera.orbit(state.camera, 0.15, 0.0)}}

def handle_event(%Event.Key{code: "i", kind: "press"}, state),
  do: {:noreply, %{state | camera: Camera.zoom(state.camera, -0.5)}}
```

`orbit/3` takes yaw and pitch deltas in radians (pitch is clamped to avoid gimbal lock); `zoom/2` moves along the view direction, clamped to a small minimum distance.

## Performance

Primitive meshes are cheap to rebuild every frame. Large custom meshes re-cross the NIF boundary each frame, so keep per-frame geometry modest. The `:raytrace` pipeline is CPU-bound and scales with the rendered area — prefer `:rasterize` for interactive frame rates and reserve ray tracing for smaller viewports or still frames.

## Examples

- [`viewport3d_cube.exs`](https://github.com/mcass19/ex_ratatui/blob/main/examples/widgets/viewport3d_cube.exs) - a spinning lit cube
- [`viewport3d_scene.exs`](https://github.com/mcass19/ex_ratatui/blob/main/examples/widgets/viewport3d_scene.exs) - a multi-object scene with an orbiting camera
- [`viewport3d_controls.exs`](https://github.com/mcass19/ex_ratatui/blob/main/examples/widgets/viewport3d_controls.exs) - live camera, render-mode, and pipeline controls
- [`viewport3d_custom_mesh.exs`](https://github.com/mcass19/ex_ratatui/blob/main/examples/widgets/viewport3d_custom_mesh.exs) - a custom mesh built from vertices and indices
- [`viewport3d_articulated.exs`](https://github.com/mcass19/ex_ratatui/blob/main/examples/widgets/viewport3d_articulated.exs) - an articulated arm built from a `Node` scene-graph