Show images mentioned in chat as inline thumbnails

Pasted screenshots and image files added to the prompt were plain
@/tmp/eca-screenshot-XXX.png text, as were their mentions in sent
messages, so there was no way to check which image went out.

Image mentions now show inline as thumbnails in the prompt, the
context line, sent messages and the compose buffer, and RET toggles
them back to text. RET is handled in eca-chat--key-pressed-return
since font-lock strips keymap text properties in chat buffers. An
after-change hook shows them however they land in the prompt (paste,
completion, prompt history, yank), and the context line keeps each
chip state across its redraws. Terminal frames and missing or remote
files stay as text. See eca-chat-image-show-thumbnails and
eca-chat-image-thumbnail-size.

Text typed right after a context chip inherited its properties, so
it looked glued to the chip, was dropped from the sent message, and
backspacing it deleted a chip-length chunk that could cross the
prompt start, moving the > prefix to the context line. Pasting into
an empty prompt left point right at the chip end, making it easy to
hit. Chip properties are now non-sticky, deleting a chip uses its
bounds and stops at the prompt start, and items added to the prompt
are spaced from the word before them.

Author: ericdallo

CHANGELOG.md

@@ -2,6 +2,8 @@
 
 ## Unreleased
 
+- Image mentions show inline as thumbnails: screenshots pasted in the prompt, image files added as context and images mentioned in sent messages. `RET` toggles them back to text. Customize with `eca-chat-image-show-thumbnails` and `eca-chat-image-thumbnail-size`.
+- Bugfix: text typed right after a context chip in the prompt became part of it: it took the chip color, was dropped from the sent message, and backspacing it could break the prompt, moving the `>` to the context line. Pasted images and contexts added to the prompt are now also spaced from the word before them.
 - Bugfix: questions asked in parallel (several `ask_user` tool calls) stopped reacting to RET once one was answered, and an option could answer another question. Each question is now answered on its own: point moves on to the next pending one, a typed answer goes to the topmost question accepting one, and stopping cancels them all.
 - Bugfix: `eca-open-global-config` edited a file the server never reads on Windows when `HOME` is unset (#321), since Emacs defaults `HOME` to `%APPDATA%` while the server uses the user profile directory. The path now comes from the server's `initialize` response (`globalConfigPath`), translated through TRAMP for remote sessions; with older servers the local fallback resolves the home directory like the server does.
 - Bugfix: new chat buffers now get the session's first workspace folder as `default-directory` (#323). It was inherited from whichever buffer was current when the buffer got created, which for a new session is the async `initialize` response, so `(let ((default-directory dir)) (eca))` wrappers ended up with the chat rooted in the previous buffer's directory.

README.md

@@ -99,6 +99,7 @@ Chat
 - `eca-chat-clear-prompt`: Clear written prompt in chat
 - `eca-chat-repeat-prompt`: Repeat a previously sent prompt
 - `eca-chat-copy-at-point`: Copy the code block or assistant response at point
+- `eca-chat-toggle-image-thumbnail`: Toggle the thumbnail of the image mention at point, also `RET` on it: a screenshot pasted in the prompt, an image file added as context, or an image mentioned in a sent message
 - `eca-chat-stop-prompt`: Stop a running prompt in chat
 - `eca-chat-tool-call-accept-all`: Accept all pending tool calls in chat
 - `eca-chat-tool-call-accept-all-and-remember`: Accept all pending tool calls in chat and remember for session
@@ -198,6 +199,8 @@ Chat
 - `eca-chat-tool-call-prepare-update-interval`: When using `smart` throttle, process every Nth `toolCallPrepare` update.
 - `eca-chat-tool-call-approval-content-size`: Face height used for tool call approval UI text.
 - `eca-chat-save-chat-initial-path`: Default initial path to save chats.
+- `eca-chat-image-show-thumbnails`: Whether image mentions show inline as thumbnails right away (default `t`). When `nil` they show as text until `RET` toggles their thumbnail.
+- `eca-chat-image-thumbnail-size`: Max width and height in pixels of image mention thumbnails (default `200`).
 
 Completion
 

eca-chat-compose.el

@@ -32,9 +32,17 @@
   "Save clipboard image DATA of mime TYPE and insert an @file mention.
 Writes the image to a temporary eca-screenshot file, like the eca
 chat buffer does, and inserts \"@/path/to/file \" at point so the
-server picks it up as a file context when the prompt is sent."
+server picks it up as a file context when the prompt is sent.  RET
+on the mention toggles the image thumbnail."
   (when-let* ((output-path (eca-chat-media--save-clipboard-image type data)))
-    (insert eca-chat-context-prefix output-path " ")
+    (unless (memq (char-before) '(nil ?\s ?\t ?\n))
+      (insert " "))
+    (let ((start (point)))
+      (insert (eca-chat--propertize-image-mention
+               (concat eca-chat-context-prefix output-path)
+               output-path))
+      (eca-chat--add-image-link-overlay start (point)))
+    (insert " ")
     (eca-info "Image added, size: %s"
               (file-size-human-readable (file-attribute-size (file-attributes output-path))))))
 
@@ -81,6 +89,9 @@ eca chat buffer behavior.
               (cons '(eca-capf (styles basic substring))
                     completion-category-defaults))
   (setq-local completion-ignore-case t)
+  (eca-chat--setup-item-stickiness)
+  (add-hook 'after-change-functions
+            #'eca-chat--auto-show-image-thumbnails-after-change nil t)
   ;; Paste image from clipboard support, mirroring eca-chat-mode: drop
   ;; the handlers inherited from markdown-mode (which insert markdown
   ;; image links) and register the @file mention handler.

eca-chat-context.el

@@ -16,6 +16,7 @@
 (require 'eca-util)
 (require 'eca-api)
 (require 'eca-chat-expandable)
+(require 'eca-chat-image)
 
 ;; Forward declarations for eca-chat.el core
 (defvar eca-chat-mode-map)
@@ -323,36 +324,63 @@ If STATIC? return strs with no dynamic values."
                                                                          (format ":L%d-L%d" start end)))
                                    'font-lock-face 'eca-chat-context-buffer-face)))
              (_ (concat eca-chat-context-prefix "unknown:" type)))))
-    (propertize context-str
-                'eca-chat-item-type 'context
-                'eca-chat-item-str-length (length context-str)
-                'eca-chat-context-item context)))
+    (eca-chat--propertize-image-mention
+     (propertize context-str
+                 'eca-chat-item-type 'context
+                 'eca-chat-item-str-length (length context-str)
+                 'eca-chat-context-item context)
+     (and (equal type "file") (plist-get context :path)))))
 
 (defun eca-chat--filepath->str (filepath lines-range)
   "Convert FILEPATH and LINES-RANGE to a presentable str in buffer."
   (let* ((item-str (concat eca-chat-filepath-prefix
                            (eca-chat--context-presentable-path filepath)
                            (-when-let ((&plist :start start :end end) lines-range)
                              (format "(%d-%d)" start end)))))
-    (propertize item-str
-                'eca-chat-item-type 'filepath
-                'eca-chat-item-str-length (length item-str)
-                'eca-chat-expanded-item-str (concat eca-chat-filepath-prefix
-                                                    filepath
-                                                    (-when-let ((&plist :start start :end end) lines-range)
-                                                      (format ":L%d-L%d" start end)))
-                'font-lock-face 'eca-chat-context-file-face)))
+    (eca-chat--propertize-image-mention
+     (propertize item-str
+                 'eca-chat-item-type 'filepath
+                 'eca-chat-item-str-length (length item-str)
+                 'eca-chat-expanded-item-str (concat eca-chat-filepath-prefix
+                                                     filepath
+                                                     (-when-let ((&plist :start start :end end) lines-range)
+                                                       (format ":L%d-L%d" start end)))
+                 'font-lock-face 'eca-chat-context-file-face)
+     filepath)))
+
+(defun eca-chat--context-chips (start end)
+  "Return (CONTEXT . IMAGE) for each context chip between START and END.
+IMAGE is the thumbnail the chip shows, or nil."
+  (let ((chips '())
+        (pos start))
+    (while (< pos end)
+      (let ((next (next-single-property-change pos 'eca-chat-context-item nil end)))
+        (when-let* ((context (get-text-property pos 'eca-chat-context-item)))
+          (push (cons context (-some-> (eca-chat--image-thumbnail-at pos next)
+                                (overlay-get 'display)))
+                chips))
+        (setq pos next)))
+    chips))
 
 (defun eca-chat--refresh-context ()
-  "Refresh chat context."
+  "Refresh chat context.
+Redrawn chips keep their thumbnail, or its absence when hidden with
+RET, while new ones follow `eca-chat-image-show-thumbnails'."
   (save-excursion
     (-some-> (eca-chat--prompt-context-field-ov)
       (overlay-start)
       (goto-char))
-    (delete-region (point) (line-end-position))
-    (seq-doseq (context eca-chat--context)
-      (eca-chat--insert (eca-chat--context->str context))
-      (eca-chat--insert " "))
+    (let ((chips (eca-chat--context-chips (point) (line-end-position))))
+      (delete-region (point) (line-end-position))
+      (seq-doseq (context eca-chat--context)
+        (let ((start (point))
+              (chip (assoc context chips)))
+          (let ((eca-chat--inhibit-auto-thumbnails t))
+            (eca-chat--insert (eca-chat--context->str context)))
+          (cond
+           ((cdr chip) (eca-chat--show-image-thumbnail start (point) (cdr chip)))
+           ((not chip) (eca-chat--auto-show-image-thumbnails start (point)))))
+        (eca-chat--insert " ")))
     (eca-chat--insert (propertize eca-chat-context-prefix 'font-lock-face 'eca-chat-context-unlinked-face))))
 
 (defun eca-chat--add-context (context)
@@ -476,9 +504,8 @@ DATA is the binary image data as a string."
         (eca-chat--select-window)
         (if (eq 'system eca-chat-yank-image-context-location)
             (eca-chat--add-context context)
-          (progn
-            (eca-chat--insert-prompt (concat (eca-chat--context->str context 'static) " "))
-            (goto-char (+ (point) (+ 2 (length output-path))))))
+          (goto-char (eca-chat--insert-prompt
+                      (concat (eca-chat--context->str context 'static) " "))))
         (eca-info "Image added, size: %s" file-size)))))
 
 (defun eca-chat--clipboard-image-p ()
@@ -602,6 +629,34 @@ file or directory."
   (when (eq last-command-event ?\s)
     (eca-chat--maybe-finalize-context-token)))
 
+(defconst eca-chat--item-properties
+  '(eca-chat-item-type eca-chat-item-str-length eca-chat-context-item
+    eca-chat-expanded-item-str eca-chat-image-path)
+  "Text properties making a prompt item, like a context chip.")
+
+(defun eca-chat--drop-inherited-item-face ()
+  "Drop the item face a char typed right after an item inherited.
+Item properties don't stick to typed text, see
+`eca-chat--setup-item-stickiness', but faces do, since the query
+typed after the context line `@' takes its face."
+  (let ((pos (1- (point))))
+    (when (and (> pos (point-min))
+               (get-text-property (1- pos) 'eca-chat-item-type)
+               (not (get-text-property pos 'eca-chat-item-type)))
+      (with-silent-modifications
+        (remove-text-properties pos (point) '(font-lock-face nil))))))
+
+(defun eca-chat--setup-item-stickiness ()
+  "Keep the text typed right after a prompt item out of it.
+Otherwise the typed text inherits the item properties: it looks
+glued to the item, gets deleted along with it and is replaced by
+the item when the prompt is sent."
+  (setq-local text-property-default-nonsticky
+              (append (mapcar (lambda (prop) (cons prop t))
+                              eca-chat--item-properties)
+                      text-property-default-nonsticky))
+  (add-hook 'post-self-insert-hook #'eca-chat--drop-inherited-item-face nil t))
+
 (declare-function dired-get-marked-files "dired")
 (declare-function treemacs-node-at-point "treemacs")
 (declare-function treemacs-button-get "treemacs")
@@ -917,12 +972,19 @@ Calls CB with the resulting message."
    ((when-let ((item-type (get-text-property (point) 'eca-chat-item-type)))
       (when-let ((item-str (get-text-property (point) 'eca-chat-expanded-item-str)))
         (when-let ((face (get-text-property (point) 'font-lock-face)))
-          (funcall cb (format "%s: %s"
-                              (pcase item-type
-                                ('context "Context")
-                                ('filepath "Filepath"))
-                              (propertize item-str 'face face)))
-          t))))))
+          (funcall cb (concat (format "%s: %s"
+                                      (pcase item-type
+                                        ('context "Context")
+                                        ('filepath "Filepath"))
+                                      (propertize item-str 'face face))
+                              (when (get-text-property (point) 'eca-chat-image-path)
+                                " (RET: toggle thumbnail)")))
+          t))))
+   ;; Image mentions of sent messages
+   ((when-let* ((path (get-text-property (point) 'eca-chat-image-path)))
+      (funcall cb (concat "Image: " (abbreviate-file-name path)
+                          " (RET: toggle thumbnail)"))
+      t))))
 
 ;;;; Completion-at-point function