what the?⌗

Yeah, I was writing a project for a client to create 16personalities like surveys for them. Essentially, I was stubborn to write this project in Common Lisp even though:

  1. I know how to use React, Svelte, et cetera
  2. I know that Common Lisp could be a time sink
  3. I had no way to strip tailwind css

Any avid reader of this blog knows I ramble way too much about Lisp and that was one of my biggest wishes; to write a website backend and frontend entirely in Common Lisp. The reasons were that it was more elegant.

One of my previous posts, an HTML dsl in Common Lisp is a 20~30 line function. Next was assessing which classes were used and weren’t and that was easy to find out since Common Lisp allows you to do that.

digging into function bodies⌗

Common Lisp allows any developer to just dig into the source code of any interpetted function. Just do:

(function-lambda-expression (fdefinition '->html))

Cool, eh?

but how do I get the classes tho?⌗

Just dig into the list like a real mole. Or, without any romantic language, just LOOP, find any :class followed by a string and collect that string, finally if the argument is a list call the function by itself.

(defun collect-attributes (list)
  (loop
    with ret = `()
    for i from 0 below (length list)
    for y = (1+ i)
    for item = (elt list i)
    do
    (cond
      ((consp item) (setf ret (append ret (collect-attributes item))))
      ((keywordp item)
        (if (> (length list) (1+ i))
          (let ((next-item (elt list (1+ i))))
            (when (stringp next-item)
              (incf i)
              (setf ret (append ret (list next-item))))))))))

okay, but how do I parse fast?⌗

Instead of answering this question, I’ll answer the opposite of it. Namely, how do I parse slow?

  1. Don’t cache the file
  2. Use classes instead of structures
  3. Use lists instead of vectors
  4. Make your buffers as small as possible and re-read consistently
  5. Don’t cache the lexens
  6. Make the data structures as complex as possible.
  7. Use errors to signal EOF for a refill
  8. Don’t use a state hash-table.

If you avoid these mistakes, you’ll parse really really fast. You don’t need to make sure that you can do refills properly when you know that the content is under 16MB. Also, you don’t need to run the lexer again if you know that the content does not change.

The lexer should use a very basic data structure. It should consist of three or four fields: the index of the last character, the start index of the first character, the name of the lexen and the content of the lexen.

Essentially, when did we start parsing, when we stopped, what we collected and what we think this lexen resembles.

After that, one could recollect all the lexens to re-create the CSS file. The HTML classes can made into a compiled regular expression before hand so that the execution is faster.

Here’s a snippet on how to turn a class into a tailwind compatiable regular expression:

(defun regex-replace-all (val &rest args)
  (loop
    with str = val
    for i from 0 below (length args)
    do
      (incf i)
      (setf str (ppcre:regex-replace-all (elt args i) str (elt args (1+ i))))
    finally
      (return str)))

(defun escape-html-class (str)
  (format nil "(^\\.~a([:]+[a-zA-Z0-9]+)?$)"
    (regex-replace-all str ":" "\\\\\\:" "/" "\\\\\\/")))

Then, join all the strings using "|" and you have one regular expression that matches ALL classes that were used in your HTML functions.

hey, but how do I recognize which functions export HTML to the web?⌗

What I did was prefix all functions that export any form of HTML with ->. So, the index page became ->/ and I made component functions like ->button and ->card which took care of knowing what to export and what not to export.

Again, Common Lisp makes it easy to find all functions in a file. Checkout the following snippet:

(let ((pkg (find-package :my-package)))
  (do-all-symbols (x pkg)
    (when
      (and
        (fboundp x)
        (eql (symbol-package x) pkg)
        (equalp (ignore-errors (subseq (format nil "~a" x) 0 2)) "->"))
      (print x))))

The above snippet finds all symbols in a package, makes sure it is a function and prints it if it starts with "->". Nifty.

you forgot the parser and lexer⌗

Yes, I did intentionally since they span over some 250 lines and they’re tailored to my specific use case. However, I shared them here in this GitHub gist.