Friday, December 23, 2011

Does Clojure fix "Fundamental problems with the Common Lisp language (citation)"?

I was looking for some Common Lisp libraries to implement an idea of mine. For lots of reasons I was considering not using Python or Clojure and going directly with Common Lisp (in fact, I think it is going to be Scheme... but it is hard to tell).

As it often happens when following semi-random links on Google, I stumbled on something quite interesting:
Fundamental problems with the Common Lisp language

Nothing extremely new, indeed. The only point that really surprised me was the "hard to compile efficiently" thing. I would have said that in general SBCL is a pretty fast environment. Not as fast as C++, perhaps not even as Java (but I believe that this depends from the specific benchmarks used), but still fast.

However I was mostly interested in the other claimed problems:

  1. Too many concepts; irregular
  2. Hard to compile efficiently
  3. Too much memory
  4. Unimportant features that are hard to implement
  5. Not portable
  6. Archaic naming
  7. Not uniformly object-oriented
May seem like a lame argument... but I think that clojure actually addresses all the problems but number 2 and 3. Please notice that I'm not claiming that clojure is a memory hog or that it is slow. Simply put, right now my impression is that common lisp is still faster than clojure. I believe that this is due to Clojure being an additional layer over Java. Java is itself probably marginally faster than SBCL (though your mileage may vary). With marginally faster I mean that really depends on what you are doing and one or the other may result faster.

Alioth benchmarks are, like all benchmarks, not extremely relevant. Still, this is my general impression. SBCL does a wonderful job, in that CL is much higher level than Java and there are many more engineers optimizing the JVM. However, Clojure takes its toll, in that, according to my tests (and to alioth as well) there is quite a lot of work to do to make it catch up with Java.

Probably for high level stuff or specific problems (where in Java there would be essentially some part of clojure runtime/libraries to be re-implemented) they are on par.

About the memory, my feeling is that JVM is a rather memory intensive business, and Clojure can't do much about it. E.g., Python programs usually run using much less memory when confronted with similar data-sets.

I'm not saying that we should drop CL and switch to clojure. However, I believe that clojure addressed some of the problem that many (some?) in the CL community feel CL has.

Thursday, December 15, 2011

Better toys for us programmers

Today PyCharm 2.0 is out. A couple of days ago, the last version of IntelliJ (11) was released as well.

I'm somewhat reluctant to discuss the matter here; I'm not a free software integralist by any means (like for example posting from a beautiful MacBook Air, with OS X), still when discussing commercial software I feel a vague sense of guilt because I feel like I am doing advertisement. This is especially true as most of the times there it does not involve only reporting facts (which would be acceptable, as the truth tends to be true, even if in favor of a commercial entity) but impressions, which could make me seem biased.

These are the most interesting improvements I found in PyCharm/IntelliJ (it should be clear when stuff applies only to one of the two -- and amazingly enough, the what's new page on PyCharm is more detailed):

  • Support for pypy (this is going to be immensely important for me, as I'm planning to move part of my development environment on pypy)
  • Support for ipython (more on this later)
  • Cython support (which may be something I'll be using soon enough)
  • Git graphs (been a bit of a PITA lately to remember the proper log options to have them in the console, my memory ain't what it used to be)
  • Gist support (I love that)
There is more. I did not even realize that before it was not there... but for example now PyCharm completes the keyword arguments in Python stuff. Which is extremely nice, in my opinion. And also the refactorings seem to work more accurately.

Eventually, I want to point out a feature I was sorely missing in all environments I tried, i.e., choosing the method to step into when debugging. First, I'm not the kind of guy that spends lots of time in the debugger (see., that would be a clear smell on the quality of my unit tests). But when I do, I often feel rather boring having to walk step by step irrelevant code.

Consider this:

my_object.that_is_some_method(
    ILuvDIP(...), self.that_is_interesting(), self.may_be_a_property)

and suppose that I feel the bug is in that_is_interesting. Now, it may well be that my Python debugging skills are not excellent. Afterall only recently I stopped instrumenting my code with prints and always use a proper debugger. Before that I relied almost only on unittests and prints.

Before PyCharm 2, it was hard to step into that_is_interesting and not in ILuvDIP. I believe that the same logic also applies to Java and IntelliJ and hopefully to Clojure. Then, back to us.

I really ain't lots of problems with IntelliJ. I feel that an IDE is a very valuable asset when developing Java. In fact, I would say it is a PITA to do without. Perhaps with Emacs and some modules like JDEE. Still, I don't know... I try to avoid Emacs these days (as I'm finding vi more and more natural to me).

The question is more interesting regarding Clojure and Python (and Ruby...). Probably if I would use Django a lot, PyCharm would be a clear winner. Support is awesome and you have to work in a frameworkish way in any case. There is nothing wrong with that, of course.

These days I'm mostly writing library/algorithmic code in Python. And I feel like ipython+vim is a great tool here. It's got an almost Mathematica/Matlab vibe that is nice for what I'm doing. I also try using that approach with Clojure more often than not. Tests as documentation and specification, REPL as an integrated development environment. It is possible that with ipython builtin in PyCharm I could just move that workflow to PyCharm itself.

There is however, the issue of code complete. Emacs fares pretty well to complete Clojure, but as far as I remember is not so good regarding completing Java (perhaps I should have installed JDEE). As a consequence, I used IntelliJ a lot even with Clojure.

Recently, I started exploring Clojure+vim too. And it is a wonderful world. I have most of what I need and it is extremely lightweight. I have to investigate the issues further. However, IntelliJ remains a solid environment for Clojure development.

Now the essential question is... I quite need IntelliJ for Java. And having it working with Clojure is a big plus (even if maybe not a strict necessity). But should I buy a separate PyCharm or just rely on IntelliJ plugin?

Monday, December 12, 2011

Erlang and OTP in Action (review)

First time I got into Erlang, it was with "Programming Erlang: Software for a Concurrent World" (Joe Armstrong). It is a very nice book, in my opinion, and I enjoyed immensely reading it. That was like 4 years ago or something. Back then, the functional revolution was just at the beginning: no widespread Scala, no Clojure at all, essentially no F#. Back then it looked like OO was going to rule the industry for years and years, with no contender of sort. Rails was fresh, Django was fresher (back then the APress book was just being released).

I bought the book because I wanted to see this "brand new" technology (20 year old, but just going to make it through in the circles I did frequent). And really, the language looked like 20 years old. Full of Prolog legacy, Unicode who's that guy? and so on. However, it no other piece of software I knew could as easily. Massive concurrency, hot swapping code. Wow.

The language, I did not especially like. The runtime… WOW! As a language, I love Python or Clojure because the way apparently distant functionalities work together and create something even more beautiful. Erlang does not have that at language level. It has at a framework level.

Think about hot swapping code in an object oriented software. First, I somewhat believe that object oriented modules are somewhat more tangled that functional equivalents. Partly because of the object reuse OO promotes (that can really be against you if you want to swap code). Then, there is the whole problem of references vs. addresses. Addresses have an additional level of indirection that makes it far easier to swap a process than it is to swap an object.

But the very idea that state is in the function parameters and that process linking and easy restarting thing are at the very root of how easy it is to swap code in Erlang. But then… back to the books.

The essential problem was that after seeing a bit of Erlang, I thought OTP was not a big deal. Yes, it is easier to use. But also plain Erlang is. And I convinced myself that should I need Erlang, I could just use plain Erlang. Than things changed: I read some OTP using code and I understood not so much of it. That was this year. What I understood, was that it could be helpful. I had to write a concurrent prototype and I welcomed the idea not to write as much code as possible.

In the meantime, I forgot many things about Erlang. In this situations, instead of reading the same book twice, I buy another book to gain perspective. So I decided to buy another book. One of the candidates was "ERLANG Programming" (Francesco Cesarini, Simon Thompson). It also had excellent reviews. Essentially I believe it contains more material than Armstrong's book and is also a bit more recent. However, as far as I understand, is still a bit terse on OTP.

As a consequence, I bought "Erlang and OTP in Action" (Martin Logan, Eric Merritt, Richard Carlsson) instead. And I'm very happy of this choice. It complements Armstrong's book well and extensively covers OTP. In fact, I also believe that the approach is very interesting. Introduce OTP first and learn to use it, then when you know what it can do, you are going just to use that. Then, learn plain Erlang in order to extend OTP when your use case is not covered. And a nice plus was a detailed description of JInterface, which I could need as well.

In fact, I do think that as it may make sense to introduce objects as early as possible in a book on an object oriented language, starting with OTP is a very big plus from a learning perspective. Then perhaps the point is that I did not need to get into a functional mindset (which I think Armstrong book does with more attention.

If the question is however just "learn to think functionally" I believe that "The Joy of Clojure: Thinking the Clojure Way" (Michael Fogus, Chris Houser), LYHFGG or Land of Lisp are probably better alternatives. Another interesting one is "Functional Programming for Java Developers: Tools for Better Concurrency, Abstraction, and Agility" (Dean Wampler), even if it has a whole different perspective.

Wednesday, December 7, 2011

Good and not so good reasons to learn Java (or any other language) [part 2.5]

So this is the third part after the not so good reasons to learn Java and the other good reasons to do it.


Java could be fun


Really. There is some stuff that is really nicely done. I like Akka, for example. And I find the idea of hacking with assertion really funny. It is a totally different approach to meta-programming that I really liked. I also like Antlr... after that every other library for imperative/oo-languages to build parsers seemed primitive.

There is some nice stuff in the Python world as well (and yeah, in Lisp you have Lisp and Haskell has wonderful stuff to). But I can tell: implement a language in Java and in C++. In Java you will be finished so much earlier... of course, other languages are faster too. But your team may not include other Haskell hackers.

Libraries, libraries, libraries

In Java there is a library for everything. Quite often, they are very well done, even if somewhat over-engineered. Probably my idea of over-engineering is a bit extreme (it comes from having seen lots of lean languages). But really, they are robust and well tested.

Moreover, Java is usually efficient enough to be a decent contendant for more demanding tasks.. Maybe C/C++ can be avoided for your application (and Java libraries are usually easier to use than C++ ones).

Java as an intermediate level platform

Say you are interested in language design. As far as I can see, you have few choices.

  • Write your own runtime, vm, etc. That was the "old" approach (Python, Ruby)
  • Implement your language on the top of a Lisp[0] or Prolog[1] interpreter
  • Use LLVM
  • Use JVM
  • Use CLR/Mono
I would rule out the latter, because I don't do any windows. Between LLVM and JVM as far as I understand it may depend on the language. JVM has a few quirks (who said lack of TCO?), but has plenty of available documentation, real world examples (Scala & Clojure), a large number of developers working for the well-being of the plarform, and a huge amount of libraries, libraries, libraries you may want to use.

LLVM is probably going to be faster, though. And has lots of optimizations and stuff for static languages. In any case, to make an informed choice, you need to know Java

Tools (IDEs, Maven, Ant)

IDEs are the bless and the curse of Java. After having used for sometime IntelliJ I really feel that the amount of functionality that IDEs for other languages offer is puny. The possible exception is Emacs for Lisps.

The funny thing is that I'm not an IDE gui. However, really, when you do refactoring (even simple stuff like moving functions and files around) it is an invaluable time-saver. I miss vim as a text manipulating programming language, but still... for Java IDEs are almost necessary. And not only bridge part of the gap with other languages, they have lots of useful stuff.

Regarding Maven... well, I just like it. I also like the fact that I'm able to build IntelliJ projects from Maven scripts is wonderful. And also Ant is a very good tool (although rather over-engineered). My humble opinion is that Ant is far easier to use than the whole autotools company. This may also have something to do with the fact that the whole Java deploy process is easier.

In any case, the point here is not how cool is Maven. Developers from other platform may want to know what is boiling up in Java-land. Sometimes we have sub-par tools and we do not even know it. Of course quite often Java tools fix "Java problems" that are different from the ones we have in other languages. Sometimes not.

E.g., the design of Maven could inspire similar tools for the other platform. Both for its strengths and for its weaknesses (to avoid them, of course).

Learn "classical" threads

Ah, this is weak. However, other languages have a "threading" module that is heavily inspired by the way Java does threading. I don't particularly like it, but being familiar with it may be a very good idea. As far as I can tell, is also what most people have in mind when thinking about threading. Who am I to say that they are wrong? I can just tell them there is better stuff.

And yes, pthreads are even more a PITA.

Find Java in other languages. Sometimes.

This is basically the extended version of the older argument. Java is hugely popular. Many "new" things are created in Javaland, and then are ported to other platforms. Or perhaps are not created but became first popular inside the Java community.

For example, xUnit libraries are available everywhere, but I think that most people first started working with it in Java. Some of the authors of the original Smalltalk library actively work on JUnit, etc etc etc.


Conclusion


I do not think that Java is particularly good. Not particularly bad, either. It is not the kind of enlightening language Scheme is. It is not easy to use and predictable like Python. However, its popularity may make it unwise not to learn it. Especially for communication reasons:

  • Books
  • Other developers (talking about OOP, libraries)
  • New ideas brewed in Javaland

wow... that's it. ;)

---
[0] Scheme, Clojure, Common Lisp...
[1] Erlang was created this way, even if now it has its own (wonderful) runtime

Saturday, December 3, 2011

Good and not so good reasons to learn Java (or any other language)[part 2]

So this is basically the second part of "Good and not so good reasons to learn Java".

And here I will discuss the good reasons to learn Java. Some of them...
I decided to split this in two posts. The next one will be published in a few days.

If you know me a bit, you probably know that I do not like Java particularly. In fact I found out that the essential problem is with Java being presented as something modern or "superior". It is not. However, it did something very good in the software environment (a part from electing over-engineering as a form of art): many technology that were outside the mainstream and was dubbed slow (garbage collector) are now widely accepted and that is partly because of Java. This is no reason to learn Java, though.

Moreover, my opinion is that Java is not a good first language. It is not a good second language either. However, after learning a high level language like Python or Ruby, a couple of functional languages (maybe Clojure and Haskell or Scheme and Scala) and something lower level, such as C, well... learning Java it could be a very good idea.

Lots of interesting stuff runs on the JVM

Yes... I'm talking about Clojure and Scala. And perhaps JRuby too. Sometimes understanding the underlying platform helps understanding some design choices. Moreover, there is stuff (e.g., in Clojure) that has not yet a fully clojurized variant and we have to use Java stuff.

Knowing Java is a plus, of course.

Lots of interesting books deal with Java

I'm not talking here about the wonderful Effective Java or Java Puzzles. They are great Java books, of course. And some suggestions may also apply to other languages. Still, if you are not interested into Java, you may miss them.

No, I'm talking about other great books...


Some of them are not entirely in Java and some have non-Java alternatives. Still reading Java is a huge plus in reading those books. I believe part of the reason is that:

Java is like a common-denominator object oriented language


Forget the Java platform. Consider just the language. Essentially Java is really a common-denominator of other object oriented language. Other languages typically offer more. Are more dynamic, offer more powerful type-systems, more expressive constructs.

Just thing about the OOP building blocks. Here we are talking about the mis-interpreted OOP as a matter of types vs. a matter of messages. That is what most people think about when referring to OOP. Unfortunately, as I said.

Java has those. But does not have much more. If you want to design OO stuff, Java gives you the building block, but does not stand above offering higher level construct. Think about Python or Ruby or Lisp... their OO facilities are so superior (plus they typically offer stuff outside the OOP model -- well, Lisp is so much more than an OO language...) that you are probably going to design stuff differently. Probably you are not going to over-engineer stuff.

Modifications are cheap. Abstractions are easy. And of course you can go outside the OO model when it makes sense.

On the other hand in Java you cannot. You can just get back at a procedural level, which is clearly inferior. The best thing you can do in Java is try to be as OO as possible: other choices usually do not pay.

As a consequence:

  1. Java is a great language to expose those "ill-conceived OOP" concepts that are used in every other language where ill-conceived OOP techniques are used.
  2. Java is a great language to truly learn to program like an object-oriented zealot

Please notice that I consider the latter a very good thing. It is a very good exercise that lets you understand the merits and the drawbacks of the paradigm. Probably when you think that something really sucks in Java, you hit a limit of OOP in a static language. Other languages may make that easier, but the wall is still there.

Java is good to learn "good OOP" too

Yes! As I already said there is a lot of smart people working with Java. They have already discovered and taught how to do "good" OOP in Java. You should learn it too.

Then, when using a different language things could only be easier. Some of the techniques you learned may be useless, because the language offers better abstractions. Still you will have a pretty clear understanding of what such abstractions are doing and perhaps you may develop a feeling for when not to use them.

As a matter of fact, too much magic is bad, even if it may seem cool. While sticking too much to simple things may actually complicate the design in the long run (meta-programming leads to code that you do not write and that is the only kind of code that needs no debugging or testing or maintenance, as it does not exist), too much magic has the same effect at the opposite position of the spectrum.

Avoid writing Java in other languages

If you know Java, at a certain point you may discover you would be writing much the same code if you were using Java. If the language you are using is C++, you probably have done something wise: chosen a restricted subset of C++ and used that one (than we may argue if you actually left out good stuff).

However, if you are using Python or Ruby, or, worse, Clojure, then you are writing awful code. $x code is not meant to be structured like Java. If it does, you are not using the language well. This is basically a side effect of "Java is like a common-denominator object oriented language".