charles/loom
1
0
Fork 0
Code Issues 4 Pull requests Projects Releases Packages Wiki Activity Actions

tui: image rendering pipeline — Kitty output, caching, and multi-backend groundwork #110

New issue
Closed
opened 2026-04-12 18:09:50 +00:00 by claude-desktop · 0 comments
claude-desktop commented 2026-04-12 18:09:50 +00:00
Collaborator
Copy link

User story

As a user running loom-tui in Kitty (or any graphics-capable terminal), I want generated images to render correctly and efficiently in gallery detail view, generate preview, and entity reference images, so that the TUI is a viable alternative to the GTK frontend for browsing and reviewing images.

Problem

Images are currently broken in gallery detail view despite the Kitty renderer being detected. Root causes identified:

1. Output ordering (partially fixed)

The Kitty renderer was writing escape sequences directly to stdout inside the ratatui terminal.draw() closure. ratatui buffers its own output and flushes after the closure returns, so Kitty commands were interleaved with TUI content.

Status: The deferred output buffer (pending_output + flush_pending()) was merged in the test harness PR, but needs verification on a real Kitty terminal. The escape sequences may still need synchronization markers (\x1b[?2026h / \x1b[?2026l synchronized output mode) to avoid flicker during the flush window.

2. Render guard (partially fixed)

render_detail() was calling ctx.render_image() every frame tick (~250ms). A DetailState.last_rendered cache guard was added, but the guard only prevents redundant calls — the initial render still needs to work correctly.

3. Image path may not exist

Gallery items created by the generate screen point to paths in pending_dir(). If the image was moved to the gallery folder during save, the original path is stale. render_detail() checks item.image_path.exists() but the path stored in the DB may not be the final location.

4. PNG-only limitation

The Kitty renderer's transmit_png() sends f=100 (PNG format). If the source image is JPEG or WebP (common for AI outputs), the renderer silently fails or sends malformed data. Needs format detection and transcoding to PNG before transmission.

5. No image cleanup on navigation

When the user leaves detail view, the Kitty image placement persists on screen until overwritten. Should emit a=d,d=i delete commands when leaving detail view or switching images.

Acceptance criteria

Kitty backend

  • Images render correctly in Kitty terminal (gallery detail, generate preview)
  • No flicker or visual corruption during render (use synchronized output mode)
  • Image placements are cleaned up when leaving detail view
  • JPEG/WebP sources are transcoded to PNG before transmission
  • Cache prevents redundant re-transmission of the same image at the same size
  • Render guard prevents redundant placement commands on static frames

Multi-backend groundwork

  • ImageRenderer trait is sufficient for Sixel, Chafa, and Halfblock backends
  • flush_pending() default no-op works for inline renderers (halfblock, chafa)
  • detect_renderer() correctly falls back through the priority chain
  • Protocol auto-detection respects tui.toml override (image_protocol field)

Generate preview

  • Preview pane renders the latest generated image after LoomEvent::ImageGenerated
  • Preview updates on each progress frame (if backend supports preview images)

Tests

  • MockRenderer records all render/delete calls for assertion
  • Integration test verifies render guard skips redundant calls
  • Unit test verifies Kitty pending_output buffering
  • Unit test verifies PNG chunk framing

Out of scope

References

## User story As a user running loom-tui in Kitty (or any graphics-capable terminal), I want generated images to render correctly and efficiently in gallery detail view, generate preview, and entity reference images, so that the TUI is a viable alternative to the GTK frontend for browsing and reviewing images. ## Problem Images are currently broken in gallery detail view despite the Kitty renderer being detected. Root causes identified: ### 1. Output ordering (partially fixed) The Kitty renderer was writing escape sequences directly to `stdout` inside the ratatui `terminal.draw()` closure. ratatui buffers its own output and flushes after the closure returns, so Kitty commands were interleaved with TUI content. **Status**: The deferred output buffer (`pending_output` + `flush_pending()`) was merged in the test harness PR, but needs verification on a real Kitty terminal. The escape sequences may still need synchronization markers (`\x1b[?2026h` / `\x1b[?2026l` synchronized output mode) to avoid flicker during the flush window. ### 2. Render guard (partially fixed) `render_detail()` was calling `ctx.render_image()` every frame tick (~250ms). A `DetailState.last_rendered` cache guard was added, but the guard only prevents redundant *calls* — the initial render still needs to work correctly. ### 3. Image path may not exist Gallery items created by the generate screen point to paths in `pending_dir()`. If the image was moved to the gallery folder during save, the original path is stale. `render_detail()` checks `item.image_path.exists()` but the path stored in the DB may not be the final location. ### 4. PNG-only limitation The Kitty renderer's `transmit_png()` sends `f=100` (PNG format). If the source image is JPEG or WebP (common for AI outputs), the renderer silently fails or sends malformed data. Needs format detection and transcoding to PNG before transmission. ### 5. No image cleanup on navigation When the user leaves detail view, the Kitty image placement persists on screen until overwritten. Should emit `a=d,d=i` delete commands when leaving detail view or switching images. ## Acceptance criteria ### Kitty backend - [ ] Images render correctly in Kitty terminal (gallery detail, generate preview) - [ ] No flicker or visual corruption during render (use synchronized output mode) - [ ] Image placements are cleaned up when leaving detail view - [ ] JPEG/WebP sources are transcoded to PNG before transmission - [ ] Cache prevents redundant re-transmission of the same image at the same size - [ ] Render guard prevents redundant placement commands on static frames ### Multi-backend groundwork - [ ] `ImageRenderer` trait is sufficient for Sixel, Chafa, and Halfblock backends - [ ] `flush_pending()` default no-op works for inline renderers (halfblock, chafa) - [ ] `detect_renderer()` correctly falls back through the priority chain - [ ] Protocol auto-detection respects `tui.toml` override (`image_protocol` field) ### Generate preview - [ ] Preview pane renders the latest generated image after `LoomEvent::ImageGenerated` - [ ] Preview updates on each progress frame (if backend supports preview images) ### Tests - [ ] `MockRenderer` records all render/delete calls for assertion - [ ] Integration test verifies render guard skips redundant calls - [ ] Unit test verifies Kitty `pending_output` buffering - [ ] Unit test verifies PNG chunk framing ## Out of scope - Sixel renderer implementation (charles/loom#13) - Chafa renderer implementation (charles/loom#14) - Halfblock renderer implementation (charles/loom#14) - Mouse-based image interaction (see mouse support ticket) ## References - Kitty Graphics Protocol spec: https://sw.kovidgoyal.net/kitty/graphics-protocol/ - Synchronized output: https://gist.github.com/christianparpart/d8a62cc1ab659194571ec44513272ac4 - `crates/loom-tui/src/image/kitty.rs` — renderer implementation - `crates/loom-tui/src/image/mod.rs` — `ImageRenderer` trait - `crates/loom-tui/src/screens/gallery.rs` — `render_detail()` call site - `crates/loom-tui/src/app.rs` — post-draw `flush_pending()` call
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
fix/prompt-enhance-timeout
tui/onboarding-wizard
tui/inpaint-canvas
tui/polish-batch
tui/gallery-grid-images
ci/simplify-workflows
v1.1.3
v1.1.2
v1.1.1
v1.1.0
v1.0.0
Labels
Clear labels   
area:agents
Agent types, pool, config

  
area:ai
AI features (prompt enhancement, tags, face detection, etc.)

  
area:config
Configuration files & loading

  
area:dashboard
Dashboard UI + observability

  
area:design
UI/UX mockups — routes to designer

  
area:design-review
Design review — routes to design-reviewer

  
area:devtools
Log / debug overlays

  
area:entities
Entities screen

  
area:gallery
Gallery screen

  
area:generate
Generate screen

  
area:image
Image rendering subsystem (kitty/sixel/chafa)

  
area:infra
Scaffolding, CI, packaging

  
area:meta
Tracking, scaffolding, setup

  
area:model-browser
Model browser screen

  
area:navigation
Sidebar, status bar, palette, keybinds

  
area:presets
Presets screen

  
area:security
Security — routes to reviewer-security (opus)

  
area:sessions
Session store, SDK resume

  
area:settings
Settings screen

  
area:sharing
Sharing overlay

  
area:test
Automated tests

  
area:ux
UX polish, notifications, error handling

  
area:webhook
Webhook routing + handlers

  
area:workdir
Clone cache, worktrees, git identity

  
type:bug
Bug

  
type:chore
Chore

  
type:meta
Tracking / decisions, not implementation

  
type:user-story
User story ticket

No labels
area:agents
area:ai
area:config
area:dashboard
area:design
area:design-review
area:devtools
area:entities
area:gallery
area:generate
area:image
area:infra
area:meta
area:model-browser
area:navigation
area:presets
area:security
area:sessions
area:settings
area:sharing
area:test
area:ux
area:webhook
area:workdir
type:bug
type:chore
type:meta
type:user-story
Milestone
Clear milestone
No items
No milestone
loom-tui v0.1.0
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
charles/loom#110
No description provided.