I have a penchant for trying to build practical things in obscure programming languages. Scheme isn't that obscure, but it's used pretty infrequently for "real work" (unfortunately). I'm quite a fan of it, so I decided to give it a shot for writing a web app, something that I do pretty often. Since there's so many implementations, I tried a few. I also took a holistic approach to the entire system and reevaluated my choice of text editor for the task.
Web applications are oftentimes repetitive. Ruby on Rails is probably the world's most efficient way to make a generic CRUD app. React makes it trivial to make a web frontend quickly. But both of those tools are easy, not simple. If you want to do something off the beaten path, you will need to learn how a lot of the magic works under the hood.
I, for one, had had enough of stitching together Medium tutorials and hundreds of thousands of Node modules (seriously, you get over 900,000 by running
npx create-react-app ___). Instead of spending hours learning the intricacies of common web application frameworks, I decided that it would be more fun and more educational to build my own setup.
Like Paul Graham noticed back in the 90s, you can write a (server-rendered) web app in any language you please. Unlike when he was starting Viaweb, most people today use frameworks that already do a lot for you. Gone are the days of generating HTML by concatenating strings in C++. Now everyone uses fancy frameworks on the frontend and just pushes JSON from the server. That means that writing your own web framework from scratch in a relatively obscure programming language isn't as huge of a boon as it was for PG.
That said, I'm not starting a startup here. Once I get something built, my intimate familiarity with my own code will mean that I'll be very productive. Everyone else will have a bigger y-intercept, but I will have a greater slope (hopefully).
Unlike Common Lisp, where there are many compatible implementations, and Clojure, where there is one common implementation (not including ClojureScript, which I regard as a different but related language), Scheme is much more loosely defined, meaning that there are a lot of incompatible implementations. At first, this might seem like a bad thing. It's already a small community, so the mutual incompatibility leaves each individual implementation with a very small number of users. That means fewer libraries and more friction in getting started.
On the other hand, it allows for much tighter communities and more "convention over configuration". CHICKEN Scheme, for instance, has a website with all of the canonical documentation for everything, including third-party packages. It has a single package manager. Unlike CL, you don't have to install Quicklisp and ASDF after the fact. There's also no compatibility issues with projects that use Roswell and projects that don't (a lot of people dislike it for being mostly written in C instead of CL).
Not all Scheme implementations have as rich of a community or as many well-integrated libraries as CHICKEN does. Many were built as a passion project or a school assignment and didn't make it further than there. That said, there are a lot of Scheme and Scheme-like languages worth investigating to create an efficient web framework.
A short list:
- Racket (not a Scheme, but close enough)
- Clojure (absolutely not a Scheme, but worth considering)
- Common Lisp (same deal)
Now, time for a little Survivor. I eliminated implementations and languages until I was left with Guile (sorry, but I already spoiled it in the title).
CHICKEN is the Scheme I've spent the most time with overall. It has a great library ecosystem and some nice facilities for developing web apps. The standard CHICKEN web framework is called Awful. It definitely feels like the sort of framework that I'm writing, but it scratches the author's itches instead of mine.
Unfortunately, I think that Awful hasn't been used for a new app in a while. Some issues that I ran into:
- The web REPL "fancy editor" (with indentation and paren-matching) is broken.
- The AJAX facilities are built around a very old (1.x) version of jQuery.
- A few snippets of code in the documentation reference functions that no longer exist (like
(remote-address)for restricting access to the debugging interfaces).
- Some of the examples only work on CHICKEN 4, the previous major version. The split between versions 5 and 4 kind of feels like the split between Python 3 and 2.
- You run Awful applications through an
awfulexecutable, not the standard compiler or interpreter. I wanted to spawn a REPL in another thread (so that there would be a REPL in my terminal after typing
awful main.so), but I can't control the execution of Awful itself when it starts through its own executable. There might be a solution for this problem, but I didn't find one before deciding to move on.
While none of these issues are real deal-breakers, I think that getting around them would be a lot of work. They're all signs that nobody has written a brand-new web application with Awful in a long time.
I debated whether to try to build a web application straight from Spiffy, the CHICKEN web server. However, I decided I should try other languages first. Had I stuck with CHICKEN, my framework built on Spiffy would have looked a lot like the framework I eventually started building on Guile.
I much prefer the consistency of Scheme, not to mention how much easier it is to internalize the whole spec. But CL was designed for Real Work (TM) and so has a number of industrial-grade implementations. SBCL rivals C for speed on some benchmarks.
Ultimately, the decision to not use CL came down to my personal preference regarding the language itself. You might find that CL is the perfect language for a very similar application, so don't let my opinion keep you from giving it a shot.
Just like how people on comp.lang.lisp debated whether Scheme was a Lisp for a very long time, the general Lisp community is reluctant to accept Clojure as one of its own. It uses a very different concept of lists that feels more Lisp-inspired than like an actual Lisp derivative. While that alone didn't stop me from giving it a try, I really like the semantics of Scheme better. I also found that I needed more experience with Java and the JVM than I had, particularly when it came to debugging my proof-of-concept application.
Again, I decided I wanted to use a real Scheme, not another kind of Lisp. As with CL, you might find Clojure perfect for your use-case.
Gerbil, Gauche, and Chibi🔗
Both Gauche and Gerbil do have web servers/frameworks written in them (Gauche-makiki and the Gerbil httpd, respectively). However, neither language has nearly as mature of a package ecosystem as CHICKEN. Gerbil seems very cool, but is also somewhat immature. Chibi had no web functionality that I could find.
Guile is the GNU extension language, which means that it is used as an embedded scripting language in a variety of GNU programs. It was also at one point intended to replace the existing Emacs Lisp interpreter in GNU Emacs, but as far as I understand that never happened. Because of its intended use as an extension language, Guile is probably already installed on your system. It's maintained by Andy Wingo, whose blog I am a fan of.
Guile has a web framework called GNU Artanis (with an impressively, amusingly bad website, or at least font choice). I decided against using it because this was around the time when I made the decision to implement my own web framework from scratch. However, having it around means that I will be able to see how they implemented certain things. Additionally, the web server that Artanis is built around (along with an SXML converter and various other tools) is included in Guile itself, no third-party packages required.
Guile is also pretty fast and has good documentation. They recently released version 3.0, which includes a JIT compiler. For some reason, the Arch Linux people haven't gotten around to packaging the latest version in the official repositories as of this writing. Maybe some of the software that uses it as an embedded scripting language needs the older version. For now I'm fine with 2.2.6, but for running my app in production I might want a newer, faster version.
Another "killer feature" for me is the
--listen flag. With it, you can telnet to port 37146 (or whatever you specify) and get a REPL. This allows for nice incremental development, something that I found much more difficult with CHICKEN. Now I don't need to actually make a better web REPL since a perfectly functional one was available over telnet. In production, I'll just make it so that the REPL is only accessible to 127.0.0.1 so that I can just SSH in and start debugging.
Editors and other tooling🔗
Since Emacs is primarily written in its own dialect of Lisp, it's the standard for writing code in any of the Lispy languages. That said, there's plenty of Lispers out there who hate Emacs, so Vim and Neovim are well-supported too. I've made at least three solid attempts to learn Emacs now, starting from a vanilla configuration or a fancy distribution like Spacemacs. I've never had a positive out-of-the-box experience like I have with Neovim. Of course, your mileage may vary (and I don't really want to engage in a flamewar with anyone).
Paredit is frequently hailed as the magic trick that Emacs has for writing S-expressions. Instead of acting on the text like it's text, you treat it like data. It's impossible to get your parenthesis wrong. I think that, with enough practice, you could surely be more efficient using all of the Emacs tooling than you would with a minimally-configured Vim. But in my experience, setting up that Emacs configuration is really painful. Even if Emacs were to improve my peak productivity, it noticeably decreases my average productivity since I spend so much time troubleshooting keybinding collisions and performance issues.
Given that Emacs has been such a pain (for me), I decided to configure my Neovim for a good balance between productivity and robustness (i.e. not breaking on me and requiring lots of troubleshooting). As far as plugins go, here's a few that I found indispensable:
tpope/vim-sensible: better defaults
luochen1990/rainbow: makes matching parenthesis easier
From there, Neovim already knows how to indent Scheme code the same way that Emacs does. Hit
gg=G to format the entire buffer if you messed up something.
A lot of Emacs users (especially Common Lispers) cite SLIME/SLY as one of the defining features of their experience. While it's powerful (and actually pretty stable), regular REPLs wrapped in
rlwrap (which adds Readline support for things like paren matching and keybindings) are plenty productive for me. The decreased setup and work required to get a nice environment makes it worth it for me to use my simpler Neovim config.
I'm really enjoying the process of building my own web framework from "scratch" instead of gluing together existing components. It's been a good learning experience and a good way to make something cool in the process.