At Cognician, we've started building our next generation of administrative tooling. We chose the following stack:
- Trapperkeeper, a toolkit for binding disparate components together, using Stuart Sierra's Component for lifecycle management and Prismatic's Graph for dependency management. Includes a configuration service.
- Pedestal, a toolkit for handling HTTP, including routing, async capability (particularly, HTTP SSE) and great security defaults.
- Datomic, an immutable database.
- Onyx, a Storm successor for distributed event stream processing.
- Transit, a self-describing wire format built on top of JSON/MessagePack, for client/server transfer.
- Om, an opinionated ClojureScript wrapper over Facebook's React library.
We like this stack particularly because it's 100% Clojure, from browser through web server and event processing to database. One toolset, one syntax, one paradigm, all with immutable persistent data structures. Nice.
We don't use lein-cljsbuild for two basic reasons:
- Cljsbuild doesn't support Google Closure's modules capability; where we have a single codebase producing multiple output files. We need this so that we can put all the common libraries we use into one 'core' file and then each app file has only its implementation code.
Instead, we use shadow-build, which is ridiculously fast, and allows us to produce a core library file and several app files from a single codebase.
Given all of that, today I'll show you how we've put this system together, with working code!
I'll show you how we compose four Trapperkeeper services into two systems - two for production and two for development:
- App: Datomic and Pedestal
- Dev: Cljx and Shadow-build
Here's the code so you can follow along:
These run in both development and production contexts. Only the app services run when you issue
lein tk at the command line, which makes use of the
bootstrap.cfg file to determine how to satisfy dependencies at startup. You can learn more in the Trapperkeeper bootstrapping documentation. In fact, I heartily recommend reading all their documentation!
This service manages our Datomic database connection, and ensures that we have a database, and that all the schema we have defined is present.
It depends on the
ConfigService (provided by Trapperkeeper) to discover all the configured Datomic database uris.
We ensure that schema is loaded by using the handy Conformity library, which will only affect the database if there are actually changes to make.
Once initialised, this service provides access to Datomic database connections and to database values.
This service manages the lifecycle of the Pedestal web server. Here we configure all the interceptors, notably injecting config and Datomic connections and current database values into each Pedestal request context.
This allows us to simply read from the provided database value in most web requests, and ensures that - by default - our database queries use a common time basis.
Of course, we have the connection as well, which means we can easily create new database values, submit transactions, and so on.
These services only ever run on a developer machine. When started, both of these services watch the file-system for changes and recompile their output as needed. As this automated reloading is in place once the services are started, there's rarely a need to reset them.
The other thing is that these services are not in the main
/src source tree; instead they are in
/dev which is only added to the source path for the
dev Leiningen profile. This prevents any of this code from loading into memory in production, where all this output would already have been compiled.
Pretty simple - compiles all .cljx files to both .clj and .cljs files as they change. We make use of Juxt's dirwatch library to watch the file-system recursively, and only invoke
cljx/cljx-compile when we know a
.cljx file changed.
This one is interesting. Shadow-build is quite low-level to use (which is great), so we added a thin layer over the top to support our own use-case.
First of all, we assume that Facebook's React will always be used, so it's hardcoded as a preamble to all output, and its externs are used for advanced compilation.
- Writes to
- Uses unminifed React.
- Output is not optimised and is pretty printed.
- Writes to
- Uses minifed React.
- Output is
:advancedoptimised and is not pretty printed.
Finally, we configure this service with data at the
:shadow key in project.clj:
:public-path: the path from which the app will load when included in a HTML file, as an absolute web path.
:target-path: the path to which output should be written, relative to the project root.
:core-libs: a vector of all the root namespaces to include in the
core.jsoutput (so that individual app modules don't have to). Dependent namespaces will be automatically included.
:externs: a vector of the externs files to use when building
:modules: a vector of maps describing each app module, each with keys:
:id: a keyword to name the module. Used to produce the filename.
:main: the root namespace for the app module.
Each module defined in
:modules will exclude any code that is already present in
Composing services as systems
Ok! So now that we understand what these services do, we can finally see how to compose them into two systems. It's actually rather simple - it's all explained in the comments:
So, to actually start all of this up, you'd follow these steps:
git clone https://github.com/robert-stuttaford/tk-app-dev cd tk-app-dev lein repl
And once the REPL is up, start the dev system:
And start the app system:
After all the dust settles, you should be able to visit http://localhost:8080 and see the
Hello! You're in debug mode! message rendered by the Om app.
Now, when you change any
.cljx file, you'd use
(user/reset) to reload your changes and reset the app system (which restarts the web server and ensures that any new schema is present in your database)!
And, when you change any
.cljs file, your ClojureScript code will be automatically recompiled - usually quickly enough for it to be ready by the time you refresh your browser.
Caveat: an escape hatch for cljx
Sometimes you might end up with a code-path from
dev/user that depends on forms present in the
.clj output of the code in
.cljx files, which, when missing, prevents the REPL from starting up.
In case you do, you can simply use this command to compile the cljx via Leiningen:
lein with-profile cljx cljx
That'll invoke cljx in its own Leiningen profile which doesn't load the
dev/ code at all.
No Clojure how-to is complete without a snippet of Emacs lisp to add to your dotfiles. Here are two simple keybinds for resetting the app and dev systems respectively. The second one uses
shift+r instead of just
(defun cider-repl-reset () (interactive) (save-some-buffers) (with-current-buffer (cider-current-repl-buffer) (goto-char (point-max)) (insert "(user/reset)") (cider-repl-return))) (global-set-key (kbd "C-c r") 'cider-repl-reset) (defun cider-repl-dev-reset () (interactive) (save-some-buffers) (with-current-buffer (cider-current-repl-buffer) (goto-char (point-max)) (insert "(user/dev-reset)") (cider-repl-return))) (global-set-key (kbd "C-c R") 'cider-repl-dev-reset)
And that's it for today!
I hope you found this post useful. I don't have comments on here yet, but please feel free to reach out on Twitter!