intro⌗

What is SSE? SSE stands for server sent events which is a feature that enables HTTP Servers to send real-time events to the client.

For a web app to be closer to a native app, it has to implement a mechanism to support real-time updates. It highly increases how good the user experience is.

how does sse work⌗

You probably heard of WebSockets. If you haven’t, WebSockets is a bidirecitonal TCP connection that can send and receive text or binary streams. It is very widely supported in most browsers and specifically targets the web.

WebSockets works by starting an HTTP connection that has status code 101 Switching Protocols then negotiates the right conditions for this WebSockets connection and if all goes well the HTTP connection is turned to WebSockets.

SSE is much more simpler. SSE is a normal HTTP request with content-type ’text/event-stream’. When an SSE request start, it waits for any events to start and then notifies the event source. No protocols being upgraded or negotitating happening, just an HTTP request that doesn’t close automatically.

ok, but how though⌗

Ok, so, say you want to implement SSE support in your favorite programming language. Well, you have to do these things in-order:

  1. Set the content-type header to ’text/event-stream'
  2. Send those headers
  3. Write to the body the events that you want in SSE format.
  4. Profit

The SSE format is rather straight forward, each field is seperated by a semicolon and a space. Each field surrounded by another one has to be started with a newline. Lastly, each message is seperated by two newlines.

Here is an example:

event: timer
data: 10

event: timer
data: 9

event: timer
data: 8

benefits⌗

SSE trumpets WebSockets in my opinion for 3 simple reasons:

  1. Easy to use
  2. Accessible
  3. Easy to implement

Nearly all web browsers support server-sent events with a uniform javascript api which is as easy as starting a connection and subscribing to specific or all events. Both from a client side and server side it is easier to support SSE than WebSockets because it builds on HTTP requests.

hunchentoot⌗

Okay, let’s move onto the implementation. Hunchentoot supports sending headers and getting a writable stream. However, to write plain text to the stream requires extra steps…

send-headers is a function that sends the headers returns a writable stream. But, before you send the headers, you should set the content type via (setf (hunchentoot:content-type*) "text/event-stream").

Afterwards, you can technically write whatever you’d like to write. Since SSE requires text and that text be utf-8, you have to convert all subsequent messages from Common Lisp strings to binary. To convert from string to binary, you need to use the function (string-to-octets "Hello World!").

The message that you want to send must be an SSE event. To create SSE events on the fly we could use the format function like so (format "event: ~a~%data: ~a~%~%" "name" "data"). Finally, write them using (write-sequence) and flush using finish-output.

Let’s wrap the writing process into a function so it’s a lot prettier.

(defun write-event (wstream event data)
  "write-event writes a SSE event to a stream"
  (declare (stream wstream) (string event data))
  (let ((octets (string-to-octets (format nil "event: ~a~%data: ~a~%~%" event data))))
    (ignore-errors
      (write-sequence octets wstream)
      (finish-output wstream)
      t)))

So, now if we glue the hunchentoot handler together with write-event, we should have something similar to this:

(hunchentoot:define-easy-handler (sse :uri "/sse") ()
  (setf (hunchentoot:content-type*) "text/event-stream")
  (let ((wstream (hunchentoot:send-headers)))
    (write-event wstream "hello" "world"))
  nil)

waiting for events⌗

The above code works perfectly fine however it quits as soon as it finishes. You need to implement a way to wait for future messages or quit once the client disconnects.

Unfortunately, currently there is no way for hunchentoot to notify you if the client has disconnected so effectively you have to write to the stream to know that it is closed.

What I currently use is a library called chanl which supports go-like channels for sending and receiving data. You can send via (chanl:send ch "hello") and block until you receive via (chanl:recv ch). My function is like this:

; sse
(defvar *ch* nil)
(defvar *ch-mtx* (bt:make-lock))
(defun add-ch ()
  (bt:with-lock-held (*ch-mtx*)
    (let ((ch (make-instance 'chanl:channel)))
      (setf *ch* (append *ch* (list ch)))
      ch)))
(defun remove-ch (ch)
  (bt:with-lock-held (*ch-mtx*)
    (setf *ch* (remove ch *ch*))))

(defun send-to-ch (name value)
  (bt:with-lock-held (*ch-mtx*)
    (loop for ch in *ch* do
      (chanl:send ch `(,name ,value)))))

(hunchentoot:define-easy-handler (sse :uri "/sse") ()
  (setf (hunchentoot:content-type*) "text/event-stream")
  (let ((wstream (hunchentoot:send-headers)) (ch (add-ch)) (open t))
    (loop while open do
      (let ((val (chanl:recv ch)))
        (unless (write-event wstream (nth 0 val) (nth 1 val))
          (setf open nil))))
    (remove-ch ch))
  nil)
;; send events like this:
;; (send-to-ch "event-name" "event-value")

2023-10-30 Update: Added a mutex to the code so that no race condition occur. 2023-11-01 Update: Fixed the “broken pipe” error

bonus: htmx example⌗

To connect this sse endpoint with htmx, you could use this page as a template.

(defparameter *txt-content* "  <script src='https://unpkg.com/htmx.org@1.9.6'></script>
<script src='https://unpkg.com/htmx.org/dist/ext/sse.js'></script>
  <div hx-ext='sse' sse-connect='/sse' sse-swap='eventName'>
      Contents of this box will be updated in real time
      with every SSE message received from the chatroom.
  </div>")

(hunchentoot:define-easy-handler (index :uri "/") ()
  (setf (hunchentoot:content-type*) "text/html")
  *txt-content*)
;; send events with
(send-to-ch "eventName" "123")
(send-to-ch "eventName" "qweqwewq")
;; you can change the event name from the htmx and
;; then send different events

I am currently building a tiny closed-source project in Common Lisp and surprisingly it is extremely enjoyable despite how bad Common Lisp libraries are. Hunchentoot is the defacto standard but it only supports HTTP 1.1 not HTTP 2.0 or 3.0.

I plan to make more Common Lisp content as I extremely enjoy the language and documenting it.