Orwell Dev-C++: Dev-C++ 5.2.0.3 Released

Not a big fan of Windows C++ IDEs.... well I'm not a big fan of Windows, I'm not a big fan of IDEs and I don't particularly like C++ either. As far as I'm concerned, if you really need a C++ IDE for Windows you should have started using CodeBlocks or Eclipse. However, if you really want to stick with Dev-C++, at least grab this new version.

The importance of using a modern C++ compiler should not be overlooked.

Orwell Dev-C++: Dev-C++ 5.2.0.3 Released: Time for another pile of bug fixes. I've also added a few features, like an updated set of built in compiler options and full file path hint...

Complex and Social Network Analysis (europython 2012)

Complex and Social Network Analysis in Python

View more presentations from rik0.

On language level

When historians started to name periods of time, they roughly divided human history in three main periods: Ancient History, Middle Ages and Modern History. Although there is not general consensus on the actual subdivision (I remember something about Burdach and Burckhardt), that is the general idea.

Modern history essentially starts with the 15th century (again, this is debated) and goes on. Then there is the idea of contemporary history, which is a moving target and regards the last 80 years. I somewhat believe that perhaps we need a better name to refer to what happened in the 20th century that is going to last after it is not contemporary anymore. Still, it is not my job.

Something similar occurred with Art History, where Modern Art goes from 1860 to 1970. Then we have Contemporary or PostModern. So in a few years, a new name will be needed... like Post^2Modern. Don't care.

And we basically have the same trouble. In 1960 Fortran was a high level language. So every non machine level language is a high level language. And we call some languages very high level.
In a few years will we have "overwhelmingly high level languages"?

Game Theory Slides

Game theory

View more presentations from rik0.

Social Choice Theory Slides

Social choice

View more presentations from rik0

Worst bug ever: random and concurrent software, but the bug was somewhere else!

Today I finally pinpointed the worst bug I ever met. The bug itself manifested perhaps a week ago when I decided to plot the distribution of a given attribute in a simulator. Essentially, the distribution should have been a power-law but a different behavior occurred. Varying some parameters changed the shape of the curve, but not the fact that it was not a power law.

Here, in red the "expected curve", while the other three colors are the plots of the distributions I found (blue the original one, green and cyan variants with different parameters). The essential problem is that finding the exact cause of this was a hell. This kind of bugs is typically not found with unit-testing. In fact, the very problem is randomized and even mocking the random number generator does not really help. In fact, each individual event occurred apparently "right" and only the whole process showed a wrong behavior.

Since the simulation is concurrent, I spent the first days analyzing if the actual order of execution did somehow cause the problem (the red curve is obtained with a non concurrent model). Turns out that I did things right: event order did not influence the problem.

Eventually, I reviewed the drawing code (in the sense of urns, not in the sense of painting!). After a couple of days I figured out that perhaps I should have mathematically proved that the code implementing the draw had the same distribution of the "red" variant. My wrong assumption was that sampling a graph edge and taking one end-point was equivalent to an extraction from an urn where each node is placed n times, where n is its degree. Essentially the question is that even though the graph was undirected, the computer did ordered the edges end-points.

This is the distribution obtained taking the other endpoint:

And here, eventually, what we got when I randomize between the two endpoints:

I just have to rewrite the code implementing the drawing. And I also have to make it fast (my former procedure was quite fast, given the fact that the whole graph is distributed).

About doing other people homework (funny functional programming in Python)...

Some time ago there was a kid who asked us to perform his homework. The assignment was to write a program that took 10 numbers from the user and print sum and average. It was a matter of utmost urgency, according to him.

Unfortunately enough, back then I decided that doing his homework was bad. I think it would have been far funnier if he would give his teacher this program:

(lambda it, sys: (
    lambda n: (
        lambda F, nxt: nxt(
            map(
                lambda dummy: (F(int) * F(raw_input))(),
                it.takewhile(lambda m: m < n, it.count()))
            ))(
                type('F+', (object, ), {
                    '__init__' : lambda self, f: setattr(self, 'f', f),
                    '__mul__': lambda self, g:
                        type(self)(lambda *stuff: self.f(g(*stuff))),
                    '__call__': lambda self, *stuff: self.f(*stuff)}),
                lambda seq: sys.stdout.write(
                    "%d %f" % (
                        sum(seq),
                        sum(seq)/float(len(seq))))))(10))(
                                __import__('itertools'),
                                __import__('sys'))

Next time, I will surely do this sort of things. By the way, what about Python not being functional enough?
Yeah... it is not really PEP compliant. Don't think it matters, though.

Software developers are not carpenters: on "Why do web sites andsoftware take so long to build? And why is it so hard?"

This evening it happened to me to read this quite interesting article about why software is so hard to write. In the article, if you don't care or do not want to read it, the blogger reports the decision of a friend of his quitting his (probably highly paid) job as software developer because he got tired of it. The motivation he (the one who quit) suggested are that software is complex to build and is the only large-scale human artifact that is mostly built by hand. Hand-made stuff, he argues, is more fragile and error prone that machine-made stuff. He also points out that good software developers do not agree on the right way to do things, etc. Eventually, he arguments that in civil engineering architects (or engineers) design stuff, politicians and administrators regulate it and carpenters build it. That thing is not possible with software.

So I have to say that: I'M FUCKING TIRED OF PEOPLE COMPARING BUILDING SOFTWARE TO BUILDING HOUSES. It is not the same thing. Moreover, I believe that comparisons with other forms of engineering has been detrimental for software development and lots of wrong practices and costly mistakes came from this illusion. I also think that people who really believe that building software is like building houses should be building other stuff. Neither software or houses (not houses I'm going to live in). Am I saying Rational F*cking Unified Process?

Humans have been building houses and bridges for millennia (yeah) and software since 50 years. It is quite natural that techniques have been developed. Software engineering is young. Really. So please, have patience. We will surely agree on the principle. That will happen earlier if people realize that no matter the process, unskilled developers are not going to build a working and maintainable piece of software.

Regarding the objection of software development being too manual, last time I checked, gcc and javac are exactly machine building software, not a bunch of underpaid third world workers. So, we do have machines building software. Compilers, build systems... testing infrastructures. We have lots of machines that help to build software, much in the same way that we use machines to build other things. We do not use machines to design most non trivial stuff.

And we continuously create new kind of machines doing stuff for us. Moreover, in the last 50 years we have been rising the abstraction level: since CPUs basically compute the same stuff, it means that we have more "machines" at work. People do not write opcodes. We write Python (Lisp? Clojure? Ruby?) code that is compiled and/or interpreted by a machine that runs on a machine called operating system that offers uses the services provided by a machine called kernel that uses relatively high-level services provided by the hardware. I don't tell the computer to write a 0 in some position of the hard disk!

Of course, we don't have machines that convert pointed-haired-boss-level babbling and unclear requirements to executable software. In other words, software developers are not skilled in divination. Perhaps haruspicy [0] or augury should be added to CS curricula. The problem is people do not even know exactly what they want. The best we can do is to keep them involved in the development process and use continuos feedback to deliver the product they expect.

"Name one other thing in the world, he said, that is used by so many people and which is created entirely by hand"

Well, many things come to my mind. Books. Laws. Theorems. You don't have machine writing books (well, in fact IA tried to build machines performing some limited "artistic" tasks, with mixed results -- the more constrained the piece of art, the better the chances --). I do not think we have machines writing laws. Human have been creating laws for millennia. We *wrote* them down. And still, there is not consensus on what is right. Laws are bugged and there are people exploiting bugs in them (and that is not even closely illegal: it is the highly paid profession of the lawyer). We don't have nor want machines writing laws for us. Experts do not agree in what is a right law. Laws are continuously changed, modified, patched, deployed (in Italy we have the concept of "decreto attuativo" which I'm not even sure it has a clear translation in English -- the idea is that the law is so vague that you need some "lower level" laws to actually explain how the law should be implemented). Laws are unjust. And nobody fucking complains that the Romans could build bridges lasting millennia (I could have a roman bridge right under my feet right now) and we can't have unambiguous laws. Well, romans were pretty good to write laws as well...

So essentially the point is that some people think that software should be like houses, because it is a product. I think that software should be like poetry, because it is an idea.

----
[0] The problem is that if you believe write requirements based on sheep guts then maybe you should start the noble profession of PHB.
[1] [0]s/sheep guts/birds flight/

Lion, brew and gcc

For me, the transition to lion was relatively painless. Painless here basically means that I patched up the couple of things that gave me problems . In the specific situation, the issue is that Apple's new developer's tools do not include regular gcc anymore. Instead there is a version working with llvm backend which is great but has some issues with some packages that have not been updated yet.

Another problem is that is closely related is that I had a Python 2.7 installed with Python main website package. I did this because older OS Xs did not have Python 2.7. That Python was built with the older gcc-4.0 apple shipped with SL. Thus, new libraries I install with pip still want that compiler, which apple moved in /Developers-3.2.x/... Thus, I lived so far adding that directory to the PATH and happily compiling.

What I should have done was getting rid of my beloved python and either use EPD or Apple built-in 2.7 (shipping with Lion) or use a brew-ed python or see if Python.org distributes a Python compiled with the new compiler (which, as far as I know, is perfectly capable of building Python). The question is: are there any python extension I need that need the older gcc? But this is not something I'm going to discuss here: not yet ready to make the transition.

I just want to point out that I installed the old gcc from homebrew-alt, and now I can just brew install --use-gcc for packages that need the old gcc compiler. This is easy to use and nice. I also removed the old Developers and hope everything's fine.

% git clone https://github.com/adamv/homebrew-alt.git /usr/local/LibraryAlt
% brew install /usr/local/LibraryAlt/duplicates/apple-gcc42.rb

An older version of gfortran I installed conflicted, but the commands above worked like a charm after removing it.

Clojure: writing tail recursive functions without using recur

Once upon a time, in the land of the Clojure, there was a brilliant student who enquired the nature of things and he for that he was greatly loved and appreciated by his teachers, for he was brilliant and asked about the nature of things and they could explain him the world. He learned about macros and first order functions and actors and software transactional memory and he was happy. But then he began to focus on the nature of recur and he felt that something was amiss and the way trampolines and functions interact made him wonder.

During a short holiday he was in Schemeland, and he saw that they did not use recur. They named the functions and called them with their name in tail position and that was the way they did. And he felt it was good. So he asked his Master:

"Why can the schemers call the functions in tail position and their stacks never end?"

And the Master told him that it was because their soil is fertile, while the land of the Clojure is just an oasis in the wastes of Javaland.

"We are lucky," he added, "for our land still gives us food spontaneously and the air does not drives us mad. And our spring is natural and not a framework. And if you do not like recur, you can always map, for or loop. Higher order functions and macros can help you to hide what you do not like, for you are the master of your own language."

For a while the student was content, still he had a recurring thought: functions can do everything and in Church he did not find recur but only functions. And so he learned Y. But Y is a demanding beast, for it requires functions to be defined awkwardly. And the student wanted to simply write:

(defn factorial [n acc]
  (if (= n 0)
    acc
    (factorial (- n 1) (* n acc))))

His Master saw him troubled and in pain, and one day a big application they were creating exploded with a StackOverflowError and he knew that unless the student was cured he could not be writing code with the others. So one evening, he told his student that if he really wanted to learn the secrets to make tail-call recursive functions run with constant stack usage, he shall go to the Land of Snakes, where they eat only Spam and Eggs, to look for some wise men and have them teach the secrets of creating tail recursion at language level. And he warned the student that the travel was dangerous.

The young student was frightened, because the Land of Snake is far away and he was barely aware of the perils that lurked in the shadow. However, his resolve was strong and he packed his things, an editor and few jars to survive the wastes of Javaland, and ventured forth. He travelled through the dull wastes where everything is private or protected and he has to ask things to do things for him. But as most people grew up in the land of Clojure, he knew that objects are but closure in disguise and he new how to bind them to his will with the power of the dot form.

After three days and three nights he eventually reached the Mines of Ruby and where massive gates blocked his way. He spoke the words as his master instructed (password: mellon), but the gate remained closed, for someone monkey-patched it and the door now spoke english, but the student did not know it and with failure in his heart he left the place, because he was trained in the ways of Lambda and could not cope with such a stateful abomination.

After months of wandering, he eventually reached Schemeland, where he at least could tail recur without recur. He spent another month drowning his pain into first class continuations and losing hope to ever come back to the land of Clojure, until one day he overhead the men telling tales at the tavern about a pythonista adventuring to Schemeland. The student sought the pythonista and eventually he found him and he asked him about the secrets of recursion and the pythonista showed him code and the student was happy because he knew classes are another word for closures and he was a master of closures. But he also understood that state was the missing element and he was saddened, because he also knew that state is treacherous.

Nonetheless, he felt he had come to far to be stopped by the formal purity of stateless programming and he eventually wrote the code. Little is known of what happened afterwards. Some claim he finally found his way to the lend of the Snake, after having extendedly monkey-wrenched the monkey patcher, other say that he went back to the Land of the Clojure.

I, your humble narrator, do not know the truth. I tried nonetheless to follow the student's step and implement myself the function that makes tail recursive calls without using recur. Ugly it is, and is indeed but an illusion, for recur lies inside the implementation. Still, it does the trick:

(defrec factorial [n acc] 
  (if (= n 0) 
    acc 
    (factorial (- n 1) (* n acc))))

And here the code:

(defn uuid [] (str (java.util.UUID/randomUUID))) 
 
(defn tail-recursive [f] 
  (let [state (ref {:func f :first-call true :continue (uuid)})] 
    (fn [& args] 
      (let [cont (:continue @state)] 
        (if (:first-call @state) 
          (let [fnc (:func @state)] 
            (dosync 
              (alter state assoc :first-call false)) 
            (try 
              (loop [targs args] 
                (let [result (apply fnc targs)] 
                  (if (= result cont) 
                    (recur (:args @state)) 
                    result))) 
              (finally 
                (dosync (alter state assoc :first-call true))))) 
          (dosync 
            (alter state assoc :args args) 
            cont))))))