About Me

Principal Software Engineer at LonoCloud

Probably best known for creating noVNC and websockify

Connect:

Introducing ClojureScript-in-ClojureScript

A (hopefully temporary) fork of ClojureScript: https://github.com/kanaka/clojurescript

A Port of ClojureScript to ClojureScript

More specifically: A port of the Clojure parts of the ClojureScript compiler to ClojureScript.

Why?

The best reason I can give is to show examples of what is possible

Web Examples

Node.js Examples

Clojure-in-Clojure

ClojureScript -> ClojureScript-in-ClojureScript

Standard ClojureScript
Input
Program
Runtime
Program.cljs
ClojureScript.clj
JVM
---
program.js
Node.js/Browser
ClojureScript-in-ClojureScript
Input
Program
Runtime
ClojureScript.cljs
ClojureScript.clj
JVM
Program.cljs
ClojureScript.js
Node.js/Browser
---
program.js
Node.js/Browser
Self-hosted ClojureScript-in-ClojureScript
Input
Program
Runtime
ClojureScript.cljs
ClojureScript.clj
JVM

1X
ClojureScript.cljs
ClojureScript.js
Node.js/Browser
Program.cljs
ClojureScript.js
Node.js/Browser
---
program.js
Node.js/Browser

Porting Steps

A chronological rather than functional overview of the process.

Review: ClojureScript Compilation Process

Step 1. Minimal web REPL

The beginning: a Clojure/conj hackathon with chouser.

  • Copy src/clj/cljs/{compiler,analyzer}.clj to src/cljs/cljs/*.cljs
  • Static snapshot of the analyzer namespaces atom.
  • Move macros out and pull in using :use-macros
  • Stub out a bunch of stuff
  • Rewrite emit-constant multimethods as EmitConstant protocol (ClojureScript protocol oriented rather than type/class oriented).
  • Bootstrap HTML/JavaScript file:
    • load the static namespaces snapshot
    • setup the analyzer environment
  • RESULT:
    • very simple web REPL
    • invoke cljs.core functions
    • specials (fn*, if, def, let*)
    • no macros (defn, let, etc)

Step 2. Node.js REPL Support

  • Add Node.js file loading to Google Closure (goog.js)
  • node.js launch script (run.js):
    • load the static namespaces snapshot
    • provide a basic REPL using Node.js readline module.
  • RESULT:
    • very simple Node.js REPL
    • equivalent to the simple web REPL

Step 3. Dynamic namespaces data

  • The namespaces atom: the analyzer's namespace and var data for def forms being compiled.
  • Lookup vars during compilation (only macros are resolved to compiler's namespace vars).
  • Normally discarded after compilation, but cljs-in-cljs needs it!
  • Old and busted:
    • Write entire namespaces content to a file
    • Manually add "(set! cljs.analyzer.namespaces DATA)"
  • RESULT (New hotness):
    • Emit namespaces data to end of each compiled file
    • Just for the namespace being compiled
    • Swapped into to build up the namespaces as each file is loaded.

Step 4. Complete the reader

  • Port syntax quote/unqoute from Java code. Syntax quote/unquote is a key component of macros.
  • Port the anonymous function and function argument reader
  • Activate comment reader
  • RESULT:
    • Full reader (modulo bugs)

Step 5. defmacro

Step 6. clj-defmacro and core macros

  • Add clj-defmacro to core_macros.clj and pull it in using :use-macros so that it is a Clojure (JVM) side version of defmacro.
  • Use clj-defmacro to port macros from Clojure core.clj and ClojureScript core.cljs into cljs-in-cljs core.cljs file.
  • Port destructure function for use by let, loop, fn macros.
  • RESULT:
    • Much of core Clojure is now available
    • Starting to feel like a real REPL.

Step 7. Types, protocols and first-class symbols

  • Definition and extension of protocols uses metadata on symbols to pass around information.
  • In ClojureScript, symbols are just prefixed strings.
  • Problem: no custom attributes on strings in JS.
  • Change symbols to real things that support metadata.
  • Port reify, extend-type, deftype, defrecord, and defprotocol macros and supporting functions from ClojureScript core.clj. Port extend-protocol from Clojure core.clj.
  • RESULT:
    • New types, protocols, records, etc can be defined at "runtime" (more specifically at cljs-in-cljs compile time).

Step 8. Namespaces as a value

  • In Clojure, the *ns* symbol refers to the mutable Namespace object of the current namespace.
  • The namespaces data in ClojureScript is a normal Clojure persistent datastructure in an atom.
  • Problem: modifying a namespace in the namespaces data (via a def*) will not automatically update the copy of it in *ns*.
  • Drop *ns* for cljs-in-cljs and replace with *ns-sym*: a dynamic "var" with the namespace symbol.
  • Update *ns* uses to use *ns-sym* and deref namespaces atom as appropriate.
  • Port some namespace related functions
  • RESULT:
    • No stale *ns*.
    • Namespace creation and other functions.
    • More correct macro and symbol resolution.

Step 9. File I/O Support

Step 10. Miscellaneous

  • Port defmulti and defmethod macros.
  • Output "streams" (*out*, *err*, *rtn*)
    • Rendering functions rather than file streams
    • *rtn* for rendering of return value
  • Port closure.clj
  • Line numbering pushbackreader
  • Bug fixes: / symbol, keywords, regexes, unicode, etc, etc
  • Tests
  • REPL functionality/cleanup
  • RESULT:
    • Useful enough to hack up cool examples for a Clojure conference.

Step 11. Self-hosting

On going work

Most things compile except for some tricky macro/var/namespace interactions in core.clj/core.cljs

The hack: copy the already compiled core.js file and the rest will compile.

Finishing Up

Future Work

  • Require/Use/CLASSPATH issues
  • Enhance debug repl
    • Vision: like firebug but for (and in) ClojureScript
    • Make it a compile/lein option
  • Self-hosting (get all of core.clj to compile)
  • Merging upstream ClojureScript (branched in Nov 2012)
  • Push to upstream ClojureScript
  • Greater use of Node.js
    • npmjs.org is the fastest growing language module repository
    • Create and push Node.js package (npm)
    • Node.js library interop
  • Performance work
    • V8 profiling
    • asm.js: asmjs.org (not cljs-in-cljs specific)
  • Cool new applications
    • cljs-in-cljs + catnip + js-git
    • LightTable Light (no backend needed)
    • All the cool things I can't think of, that all of you will create.

Thanks:

  • Chouser: the initial bootstrap and other help.
  • Fogus: Himera design used in ClojureScript.net
  • Irakli Gozalishvili: bug fixes
  • Arthur Edelstein: jq-console/editor for ClojureScript.net
  • Mike Anderson: surprise-ending example.
  • Game of Life demo: https://github.com/sw1nn
  • David Nolen for saying "Wait, this isn't right" ... there really is no better encouragement than that.
  • Alan Dipert for thinking (and saying repeatedly) that this is cool.
  • Aaron Brooks for being a sounding board.
  • Supporting colleagues at LonoCloud
  • Alex Miller for putting on an awesome conference.
  • Others I've forgotten

Questions?

Use arrow keys to navigate