Friday, May 11, 2012

Software developers are not carpenters: on "Why do web sites andsoftware take so long to build? And why is it so hard?"

This evening it happened to me to read this quite interesting article about why software is so hard to write. In the article, if you don't care or do not want to read it, the blogger reports the decision of a friend of his quitting his (probably highly paid) job as software developer because he got tired of it. The motivation he (the one who quit) suggested are that software is complex to build and is the only large-scale human artifact that is mostly built by hand. Hand-made stuff, he argues, is more fragile and error prone that machine-made stuff. He also points out that good software developers do not agree on the right way to do things, etc. Eventually, he arguments that in civil engineering architects (or engineers) design stuff, politicians and administrators regulate it and carpenters build it. That thing is not possible with software.

So I have to say that: I'M FUCKING TIRED OF PEOPLE COMPARING BUILDING SOFTWARE TO BUILDING HOUSES. It is not the same thing. Moreover, I believe that comparisons with other forms of engineering has been detrimental for software development and lots of wrong practices and costly mistakes came from this illusion. I also think that people who really believe that building software is like building houses should be building other stuff. Neither software or houses (not houses I'm going to live in). Am I saying Rational F*cking Unified Process?

Humans have been building houses and bridges for millennia (yeah) and software since 50 years. It is quite natural that techniques have been developed. Software engineering is young. Really. So please, have patience. We will surely agree on the principle. That will happen earlier if people realize that no matter the process, unskilled developers are not going to build a working and maintainable piece of software.

Regarding the objection of software development being too manual, last time I checked, gcc and javac are exactly machine building software, not a bunch of underpaid third world workers. So, we do have machines building software. Compilers, build systems... testing infrastructures. We have lots of machines that help to build software, much in the same way that we use machines to build other things. We do not use machines to design most non trivial stuff.

And we continuously create new kind of machines doing stuff for us. Moreover, in the last 50 years we have been rising the abstraction level: since CPUs basically compute the same stuff, it means that we have more "machines" at work. People do not write opcodes. We write Python (Lisp? Clojure? Ruby?) code that is compiled and/or interpreted by a machine that runs on a machine called operating system that offers uses the services provided by a machine called kernel that uses relatively high-level services provided by the hardware. I don't tell the computer to write a 0 in some position of the hard disk!

Of course, we don't have machines that convert pointed-haired-boss-level babbling and unclear requirements to executable software. In other words, software developers are not skilled in divination. Perhaps haruspicy [0] or augury should be added to CS curricula. The problem is people do not even know exactly what they want. The best we can do is to keep them involved in the development process and use continuos feedback to deliver the product they expect.

"Name one other thing in the world, he said, that is used by so many people and which is created entirely by hand"

Well, many things come to my mind. Books. Laws. Theorems. You don't have machine writing books (well, in fact IA tried to build machines performing some limited "artistic" tasks, with mixed results -- the more constrained the piece of art, the better the chances --). I do not think we have machines writing laws. Human have been creating laws for millennia. We *wrote* them down. And still, there is not consensus on what is right. Laws are bugged and there are people exploiting bugs in them (and that is not even closely illegal: it is the highly paid profession of the lawyer). We don't have nor want machines writing laws for us. Experts do not agree in what is a right law. Laws are continuously changed, modified, patched, deployed (in Italy we have the concept of "decreto attuativo" which I'm not even sure it has a clear translation in English -- the idea is that the law is so vague that you need some "lower level" laws to actually explain how the law should be implemented). Laws are unjust. And nobody fucking complains that the Romans could build bridges lasting millennia (I could have a roman bridge right under my feet right now) and we can't have unambiguous laws. Well, romans were pretty good to write laws as well...

So essentially the point is that some people think that software should be like houses, because it is a product. I think that software should be like poetry, because it is an idea.

----
[0] The problem is that if you believe write requirements based on sheep guts then maybe you should start the noble profession of PHB.
[1] [0]s/sheep guts/birds flight/

16 comments:

Τραμπούκος said...

The argument of the original article is deeply flawed. We don't have machines building anything. We don't have machines building chairs or wheels we don't even have machines building simple stuff like nails and nuts and bolts. Someone always painstakingly designs that stuff and then we get machines replicating it. In software this kind of replication is the easiest thing in the world. You can make infinite copies of your application after having designed it.

Never the less software engineering is currently in its infancy compared to other engineering disciplines. As alan kay said in one of his talks we are starting to build cathedrals they collapse and then we are building pyramids on top of the rubble.

Unknown said...

I couldn't agree more. The ease of replication is an interesting point.

However, as far as I know we have machine that build chairs (at least some kind of chair).

Τραμπούκος said...

Do you mean we have machines designing chairs or just machines that take a chair blueprint designed by a human and replicate the design ?

Machines that would design a chair from scratch and then produce it really sound interesting.

Unknown said...

Now I get your objection. Yes, we have machine building stuff, not designing it. Not that I'm aware of.
However, I think that even "implementing" software is extremely different from building a house.

Unknown said...

I like your article. Especially your last sentence reminded me of what I'm telling people asking me, why I like to be a software developer.

Developing software is my personal dream. I like it very much. I always compare developing software with art painting. I believe that doing software projects is and should be a very creative process. When I start a new software project then I don't have a clear idea on how things should be done. The canvas is blank. You will need phantasy, a vision. Then you need to experiment, you might change your mind a few times (continous painting over).

Does this mean that writing software is an art?

Henrik Ebbeskog said...
This comment has been removed by the author.
Henrik Ebbeskog said...
This comment has been removed by the author.
Henrik Ebbeskog said...

Cool! I just wrote about the same thing a couple of days ago: http://kodkreator.blogspot.se/2012/05/developers-are-not-carpenters.html

Cícero said...

Great post Enrico!

Τραμπούκος: I talked about this difference between designing and manufacturing things this week with some colleagues. People tend to compare the development of a new software to the production line of a factory, where a bunch of machines is just replicating some design that was elaborated firs by a human and probably took a lot of time to be completed.

You can make an analogie with the iPhone, today you can assemble a lot of iPhones per day, but obviously to achieve the first model it took some time for the guys at Apple to think about design, features and the stuff. You can think the same way about software development: it takes some time to design the software but after that the replication has almost cost zero.

Unknown said...

@ Klaus: I strongly believe that software development is at least a craft, in the sense it is a semi-artistict production. Some artisans are part artists and part manifacturers, and I think it applies to software as well.

We have a language (like writers). We can write stuff like an accountant writes a letter or we can write stuff like a writer writes a novel. I think.

In fact art is about communication and creativity within a set of rules. We know the rules. We only have to be creative.

Besides... read Programming Pearls (both the functional and non functional variants). It is hard not to think to haiku.

Unknown said...

@Herik: nice post!

Unknown said...

@Cicero: I rather agree. I think that it is a problem that a some people spread the idea that designing software is a "nobler" craft than writing stuff.

It is part of feeling "better" because it is what they do. However, in my personal experience, the distinction is extremely blurred. You design when other people need to use what you write, you implement otherwise.

With agile methods, the distinction is almost gone. It is more about the role you interpreting in a specific moment. Perhaps even the focus you give to your activity more than anything else.

Unknown said...
This comment has been removed by the author.
Unknown said...

I am impressed with the effort you have so obviously put into this content. I am also impressed with your point of view on this topic, especially since you have made your points so clear.

Unknown said...

The site was absolutely fantastic! Lots of great information and inspiration, both of which we all need!b Keep 'em coming... you all do such a great job at such Concepts... can't tell you how much I, for one appreciate all you do!

AimIT Software - Software Companies

Unknown said...

I can say that creating your web site and software takes artistic touch too. Pointing to that because you must create a good looking website to attract clients, not just in looks but also in contents. Even incredible software enhancements takes time to be designed and finished.