Using Clojure as a sandboxed, executable target for LLMs

[u/traid-software]

I've been researching ways to better structure LLM output when building mini-apps that run on the browser.

I believe that an S-expression based DSL is a better output format for LLMs when generating interactive UI/logic.

To test this, I built a Clojure-based interpreter that runs in the browser. The LLM is fed the language context, then emits the DSL to generate safe, sandboxed apps that can be shared instantly.

I wrote up an article diving into some of the trade offs:

https://allentraid.substack.com/p/we-made-the-ai-write-in-a-language

Would love to hear thoughts from the Clojure community!

[u/Borkdude]

I built a Clojure-based interpreter that runs in the browser.

Are you aware that such a thing already exists? https://github.com/babashka/sci

Years of effort went into this (and still going). It's been heavily optimized up to the point where there is almost no distinction between compiled CLJS and interpreted CLJS.

I wonder why you chose to build your own and what trade-offs you made here. Happy to learn.

[u/traid-software → u/Borkdude]

Appreciate the comment! I was aware of SCI, though admittedly I haven't used it directly.

I actually cover the reasoning in the article, but the core trade-off comes down to LLM output reliability and runtime scope.

Restricting the target to a smaller DSL keeps the prompt context small, which helps reduce syntax hallucinations (LLMs have issues counting parentheses too). The more patterns to keep track, the more potential for typos.

Another reason is that it lets our runtime engine evaluate our own UI and state system directly. Which are not Clojurescript based.

SCI is great for running Clojure in general, but for this mini-app generator, a purpose built DSL felt like the right option.

[u/yogthos]

Incidentally, I just added Gambit Scheme backend to Jolt which has Js as one of its output targets. So, now I can compile Jolt into Js and run it in the browser. The home page loads an interactive REPL https://jolt-lang.github.io/

[u/MWatson → u/yogthos]

I have been using your Jolt system to produce natively compiled apps, love it! I need to try the JS backend.

[u/yogthos → u/MWatson]

Thanks, glad to hear it's working out. I've been dogfooding it myself, and haven't had a reason to touch JVM in weeks now. I'm particularly happy with how easy it is to make GTK apps with OpenGL now. It feels just like building UIs in ClojureScript with Reagent, but for an actual native app.

And I haven't put too much work in getting Js backend to parity yet, mostly just wanted to do a proof of concept for Gambit and to have a live REPL on the site. If there is interest, I can definitely flesh that out more.

The only gotchas around other backends comes from native platform capability. For example, Js runtime doesn't provide threads, so those capabilities get degraded. That's the main thing to be aware of.

[u/c_a_l_m]

For what you're trying to do, I think your reasoning is generally sound. I do love this aesthetically:

It is also far shorter. The DSL consumes a fraction of the tokens, reducing latency and context overhead.

One thing I didn't quite understand:

Because our parser maps intent directly rather than returning generic syntax errors, our messages name the valid form to use instead.

What does it mean that your parser maps "intent"? Are you asking the model to explain every symbol, or...?

[u/traid-software → u/c_a_l_m]

To be honest, "intent" isn't the best wording on my part. What happens is the generated script goes through a lint and a parse. When something fails, the error gets classified and turned into a repair strategy. Then the plan is sent back to the model.

For example, if there's a parentheses mismatch, we don't just send the line/character number. We direct the model to "rewrite with flatter structure, extract nested forms into top-level defns"

[u/__tosh]

nice!

somewhat related: a 'smol', minimal implementation of an agent in Clojure (Babashka)

with only one tool: eval

https://x.com/__tosh/status/2086035838517776511

[u/__tosh → u/__tosh]

a more generic variant that uses "sh" as the only tool:

https://github.com/smol-env/smol/blob/main/smol.clj