How to map clojure code to and from JSON?

Question

I have a crazy idea, which involves putting some clojure code into CouchDB and writing views that query it. I don't want to store the clojure code as plain text, because then I would have to worry about parsing it in the views. Formatting and comments don't need to be preserved, but the code should be able to go in and out of the database without changing in structure. Keywords, symbols, and strings should all stay in their native type. Additionally, I want the code to look elegant and be efficient.

I'm thinking of representing things as follows:

• Symbols as strings that start with '
• Keywords as strings that start with :
• Strings unmodified, except when they start with ' or :, in which case they're escaped with a backslash.
• (parens) as an array
• [brackets] as an array with "_[]" as the first element
• maps ({}) as an object
• sets (#{}) as an object with the values set to 1 and "_#{}" included.

Critiques, experiences, and ideas are appreciated.

Edit: Here's what happens if I try reading and writing JSON code using json functions from clojure.contrib:

user> code
((ns bz.json.app (:use (ring.middleware file))) (defn hello [req] {:status 200, :headers {"Content-Type" "text/plain"}, :body "Hello World!"}) (def app (wrap-file hello "public")))
user> (read-json (json-str code))
[["ns" "bz.json.app" ["use" ["ring.middleware" "file"]]] ["defn" "hello" ["req"] {"body" "Hello World!", "headers" {"Content-Type" "text/plain"}, "status" 200}] ["def" "app" ["wrap-file" "hello" "public"]]]

There's a fair bit that needs to be done for line 4 of the above to be exactly like line 2. It appears that it's a library project, unless there's a function somewhere that does it that I don't know about.

With such a library, here's what calling it might look like:

user> (= (json-to-code (read-json (json-str (code-to-json code)))) code)
true

Answer 1

clojure.contrib.json has been superseded by clojure.data.json:

(require '[clojure.data.json :as json])

(json/write-str {:a 1 :b 2})
;;=> "{\"a\":1,\"b\":2}"

(json/read-str "{\"a\":1,\"b\":2}")
;;=> {"a" 1, "b" 2}

You might also like to use cheshire which has a nice API and support for various extensions such as custom encoding and SMILE (binary JSON):

(:require [cheshire.core :as json])

(json/encode {:a 1 :b 2})
;;=> "{\"a\":1,\"b\":2}"

(json/decode "{\"a\":1,\"b\":2}")
;;=> {"a" 1, "b" 2}

Answer 2

As mikera suggested, clojure.contrib.json/write-json will convert not only primitive types, but Clojure's ISeqs and Java's Maps, Collections and Arrays, too. This should cover most of your code (seen as data), but in the event you want to write anything fancier, it's easy to extend the JSON writer, by mimicking out Stuart Sierra's source code (see here):

(defn- write-json-fancy-type [x #^PrintWriter out]
    (write-json-string (str x) out)) ;; or something useful here!

(extend your-namespace.FancyType clojure.contrib.json/Write-JSON
    {:write-json write-json-fancy-type})

This is assuming you don't need to store computed bytecode, or captured closures. This would be an entirely different game, significantly harder. But since most Clojure code (like most Lisp's) can be seen as a tree/forest of S-Expressions, you should be OK.

Parsing out JSON back to the data can be done with clojure.contrib.json/read-json (take a short time to look at the options on its definition, you may want to use them). After that, eval may be your best friend.

Answer 3

For completeness sake there is also clj-json which uses Jackson underneath to parse the JSON.

Answer 4

I think your idea is sound, but I'd simplify the handling of collections by using tagged arrays (["list", …], ["vector", …]) instead. Apart from that, I wouldn't change the implementation strategy.

I like your idea and to code in Clojure, so I took a stab at implementing your code-to-json (with the above suggestion incorporated) at https://gist.github.com/3219854.

This is the output it generates:

(code-to-json example-code)
; => ["list" ["list" "'ns" "'bz.json.app" ["list" ":use" ["list" "'ring.middleware" "'file"]]] ["list" "'defn" "'hello" ["vector" "'req"] {":status" 200, ":headers" {"Content-Type" "text/plain"}, ":body" "Hello World!"}] ["list" "'def" "'app" ["list" "'wrap-file" "'hello" "public"]]]

json-to-code is left as an exercise for the reader. ;)

Answer 5

With version 0.1.2 of clojure.data.json the whole thing could look like this I guess:

(require ['clojure.data.json :as 'json])

(defn- json-write-date [s ^java.io.PrintWriter out escape-unicode?]
  (.write out (str "\""
    (.format (java.text.SimpleDateFormat. "yyyyMMddHHmmssZ") s)  "\"")))

(extend java.util.Date clojure.data.json/Write-JSON {:write-json json-write-date})

(json/json-str { :example (java.util.Date.)})
"{\"example\":\"20120318182612+0100\"}"`

Answer 6

If you want to use JSON as a representation, I'd strongly suggest using clojure.contrib.json, which already does the job of converting Clojure data structures to JSON pretty seamlessly.

No point reinventing the wheel :-)

I've used it pretty successfully in my current Clojure project. If it doesn't do everything you want, then you can always contribute a patch to improve it!