Clojure at a Bank – Clojure Code Immaturity

November 4, 2012 § 13 Comments

I’ve posted recently about a team at an investment bank wanting to make the transition from Java to Clojure. Here I want to write about some of the issues around our Clojure code being ‘immature’. Before I do though it’s only fair I state up front that not all of our early code was terrible, Clojure is indeed a pragmatic language where you can write decent and understandable code relatively easily. Still..

Comments and LOC

Most of the devs on our team have a TDD/DDD/BDD background with half having once plied a trade as XP consultants. Our approach to writing beautiful Java code was to make it flow and to tell a story. Expressive names for classes, methods and variables, each chosen to convey clarity and meaning to the fortunate reader.

Therefore when we jumped both feet into Clojure we unconsciously brought with us the belief that comments just weren’t needed. Add in to the mix that we gave our args the shortest possible names – most of the time just single characters – one could argue that we purposefully went about trying to obfuscate what we were writing.

Then to make our code more fugly, we executed the common newbie sin of not really knowing what’s in Clojure.core but churning out bucket-loads of FP code anyway. For example we had a brave early attempt to get around assoc-in not being able to work with nested vectors as well as maps when it actually could (assoc-in m [:key1 0 :bar]). This led to some funky code existing in functions with interesting sounding names like ‘weave-in-vectors’ – the choice of naming being a sure fire smell that there must be a more idiomatic way of doing things. Then there’s the little stuff: ‘if’ and ‘let’ vs ‘if-let’, ‘let’ as its own form rather than embedded into a ‘for’ as ‘:let’. Then there’s zipmap, juxt, mapcat, group-by… a considerable list of helper functions that avoids us having to write our own cruft.

I also have to own up to having a personal fetish for a low LOC wanting it to compare ever so favourably to the old Java stack that preceded it. The cost was that some people wondered wtf some of my code did but at least there were few lines of it. There’s got to be some prize for that, right?

We matured past these issues by communicating amongst ourselves as we found better ways of doing things and thankfully we had a team where criticism was generally well received. Clojure itself is an opinionated language and when you’re coding in a more idiomatic way the pieces of the language tend to fit more easily together. Idiomatic = more graceful/simplistic. Don’t say to a colleague: “Your code is shit”, do say: “There’s a more idiomatic way of doing this”. StackOverFlow and blog posts are full of examples of how to write more idiomatic code for particular use-cases and the Clojure Google Groups are good also.

Namespaces

In Java/.NET development we now have extremely powerful tools that help you to navigate your way around a large code-base – i.e. Eclipse/Intellij for Java. As the amount of class files inexorably grows it never really seems to matter and you just get used to it. (Here’s a controversial Recursivity blog post entitled “IDEs Are a Language Smell”).

In FP a single namespace will nearly always contain much more logic than compared to the average OO class. Since you’ll be spending more focused time in fewer files this then creates a need for namespaces to be presentable. Comments at the top can be helpful, they should have fewer public functions, and they should be split up if they get too large – we’ve occasionally split out the cruft from a namespace into an accompanying ns-utils.clj to make the main one clean. We’ve also reapplied various bits of DDD and OO to model namespaces around business domain concepts and to keep them well encapsulated.

Then there’s (:require) vs (:use). (:require) is much better as each dependency usage is clearly marked with a prefix so you can clearly see where dependencies are used. This is kind of obvious but in the early days we used (:use) in most places – without only – and now we’re having to go back over and correct. Note that we did play around with using lein-slamhound for optimising our namespace declarations but then we found that the kind of namespaces you typically want to use this on need restructuring anyway.

Macros, Protocols, Defrecords

Having fun with macros is a right of passage. Some people passionately detest them whilst others enjoy using complicated solutions to solve complicated problems. I’ve learned that if I’m going to build a macro then it helps to keep it minimal and to delegate out the logic into a separate function relatively quickly. We once had some special code eval’ing deftest forms to generate tests based on some data that we had saved up. The idea was that the auto-generated tests would then play nicely with lein-test and consequently our build server. The trouble is that arguments to macros have to be serialized and you’re limited in by this, not to mention that the code can become that much harder to follow. By looking under the hood at what the deftest macro actually did – basically registering test functions – we replaced this little mini-framework of macros and evals without much fuss and it gave better performance in return (we stopped reloading the same data twice, once to register the tests and once to run them). Macros are helpful and powerful but they come with a cost.

Protocols are like having an extremely awkward member of the team around; someone who can get stuff done with an air of awesomeness yet at the same time you’re wondering if there is just isn’t a simpler way. They have some quirks setting them up and it’s added complexity, but on the whole I have to say that Protocols have been good for us. We had some hairy areas of the codebase where we were doing concurrent operations passing around a lot of functions, partially built up or otherwise. Introducing Protocols allowed us to pass a family of related functions around together as one thing with some immutable state. In a different area of the code-base they also laid down some hard interface definitions. We didn’t strictly need them for this purpose but enforcing a little of compiler-checked OO felt like a good thing.

Defrecords were primarily introduced for performance reasons. Where we were creating lots of little map instances we switched to using simple Defrecords instead. I would also argue that they forced us to think a bit harder about our modeling of data and use of simple data structures, leading to cleaner code in some areas.

Wrap Up

In my opinion figuring how to write Clojure code that is more idiomatic and simplistic is what makes Clojure fun, along with the fact that Clojure is blistering pragmatic. The learning never stops.

§ 13 Responses to Clojure at a Bank – Clojure Code Immaturity

  • I liked this post a lot. It highlights the importance of not feeling comfortable with “just knowing” the basics of the language, but also taking the time to review the core libraries.

    This happened to me a few days ago with merge-with, I wrote something about it here: http://www.plusastra.com/2012/10/implement-but-remember-to-check-api.html

    You can run into this also if you don’t know about the ecosystem and even further if you mix Clojure with “plain JAR files”.

  • bobx says:

    So you have hardcore XP guys and you wrote code with one letter arguments? Seems like some standards got dropped when you moved to clojure. What I would have liked to see in this article is what you actually accomplished by doing this other than getting paid to code like a schoolgirl by a major corporation?

  • Tinco says:

    Yes, because using one letter arguments is the gravest sin you can commit as a hardcore XP guy?

    What does ‘code like a schoolgirl’ even mean?

    He didn’t post any code, does that mean you think clojure is a language for schoolgirls, not worthy of major corporations?

  • geek42 says:

    i am interesting of the following things:

    1, which dirven your team to migrate from java to clojure in transtion(as i know, its very important domain)?

    2, have you ever tested how much time a newbie guys needs to understand the code and began to maintain it

  • I totally agree with “we executed the common newbie sin of not really knowing what’s in Clojure.core”. I laughed when I read this article : http://java.dzone.com/articles/replacing-common-code

    “If you’ve written a fair amount of Clojure code and aren’t familiar with clojure.set, then chances are you’ve probably reinvented a few functions that are already available in the standard library.” << Yeah, no, my sample project is way too simple for that to happen… Oh wait what is that clojure.set/index …?

  • Jeremie Pelletier says:

    I get the feeling you looked at Clojure as a dialect of Java rather than what it truely is; a dialect of Lisp. This only confirmed what I already suspected; banks still don’t hire good programmers.

  • Jon Pither says:

    Hi geek42,

    WRT 1 – See my other post about why we chose to make the move to Clojure – http://www.pitheringabout.com/?p=693.
    WRT 2 – Everything depends on the individual. Some guys are productive after 3-4 weeks, others take longer depending on their interest.

  • Jon Pither says:

    bobx,

    ‘Standards got dropped when you moved to Clojure’. As I said at the start I’m not blasting all of our early code in this post – far from it. I’m picking on some examples to highlight some of our learnings. If the odd standard was compromised set against our previous work in Java then frankly so what? Many other standards were raised far higher (rapid feedback, far more concise and purposeful code, testing that makes sense). To your ‘school-girl’ point where you say you want to see what we actually accomplished, I’m looking to detail this in a separate post.

  • Jon Pither says:

    Hi Jeremie,

    Would you be able to backup your ‘feeling’ as to why we looked at Clojure simply as a ‘dialect of Java’ rather than Lisp? Can you be more specific as to what this means? I guess you’re saying that if we’d been experienced Lisp programmers then we wouldn’t have had to go through some of the learning hoops in this post – this is quite an obvious point isn’t it? As for your sweeping generalisation that ‘banks don’t hire good programmers’ I can’t see any point in replying to it.

  • Michael Cohen says:

    Thanks for sharing your experiences. I think there’s a lot of shops out there right now looking at Clojure and it’s still early enough days that these perspectives can be quite valueable.

  • anonymous says:

    “Rite of passage.”

  • I realize these two things don’t make sense together, but I bet in a year you’ll agree with me: one letter arguments are a feature and not commenting is really silly.

    Last time I looked, Clojure didn’t have a good formatting system the way Lush does, so there is an excuse. This is a tragic waste, but I got other stuff to do than reproduce Lush helptool.

    Thanks for an amusing read; I know I need to go look at clojure.core more the next time I reach for the alien technologyl

  • [...] Clojure at a Bank – [Our] Clojure Code Immaturity – experiences with going from Java to Clojure: 1) too few comments, too short names => hard to learn the code; 2) not knowing clojure.core well enough => reimplementing (if-let, juxt, …); 3) structure, comment, split up your namespaces well, navigating more complicated then in Java IDEs; 4) reasonably used Macros, Protocols, Defrecords payed off; [...]

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

What’s this?

You are currently reading Clojure at a Bank – Clojure Code Immaturity at Pithering About.

meta