Monday, June 7, 2010

PDF and other tragicomical histories

Today I had the unlucky task of creating a PDF from a Word document we wrote.
In fact, I did not think the thing would have gotten tricky (ah, how naive!). Word 2007 has a nice "save to PDF" menu item and I thought I could use that to solve my problem. Luckily enough, I double checked the generated PDF on the Mac. And, apparently, all the bold fonts were substituted with another font. I already mentioned this in a post earlier today.

This is completely illogical since the font was Times New Roman and I have Word on the Mac as well, which would install a lot of additional fonts, even if TNR was not that standard. So I searched for a Windows PDF creator (which, by the way both OS X and Linux-based desktop programs have). I found a bunch of them, some commercial, some free, some open source. My choice was PDFCreator.

Some reasons are: i) it is free as in free beer, ii) it is free as in free speech, iii) had no ads, iv) is based on the same softwares I happily used with Linux. Of course, reason i) was paramount: I had no intention to buy a stinking PDF creator for Windows unless it is absolutely necessary (which may be). The other three reasons have to be considered together: I searched google for suggestions, but I had no clue on which one was supposed to be better.

Usually the only problem with open source software is lack of support/user friendliness. However, that was not the case of PDFCreator. Unfortunately, for reasons beyond my comprehension, the generated pdf completely screws[0] a TIFF picture I put in the document. The same picture is perfectly fine in the PDF generated with word built-in generator. I tried to include in word a PDF, but that did not work either (the PDF was linked, but not visible -- nor printable --). Amazingly, Word for Mac is capable of using PDF image files, IIRC.

Perhaps, I should have tried more generators. I went home, opened Word for Mac, double checked that the formatting was fine (which was not, but I could fix it in little time) and simply printed to PDF. The PDF is perfect. This goes under: how to completely waste a Monday morning.

Footnotes

[0] This is a technical term, essentially it looked like it had low resolution

A matter of homoiconicity

Somewhere around the late sixties the term “homoiconicity” has been introduced in the computer science community. In fact, the term was older and dates to the 1960, at least according to Wikipedia. Strangely enough, I did not find traces of the term homoiconic or homoiconicity in McIllroy [1]. I was not unable to recover the full text of the paper on TRAC [2]. However, widespread use of the term is due to McKay (along for the very idea of object oriented programming and a good deal of criticism on Lisp – perhaps I will look for some opinions of famous lispers on Smalltalk to make things even) in his Ph.D. dissertation.
The basic idea is simple: program is data. That is to say, the program is represented in the very same format as data. Thus, a program can manipulate programs (and even itself) in the same ways it uses for data. This is great and is the foundation for great and funny meta-programming.
Indeed, references on TRAC should make clear that a structured representation of programs is not needed. TRAC manipulated strings and programs where represented as strings. On the other hand, Lisp manipulates lists and a program is a list. It is quite rare in a discussion on programming languages with a lisper that he does not use the term “homoiconic” somewhere in the discussion, usually before becoming condescending (a proof of this is being developed actively – no trolls have been harmed during the experimentations).
What I find extremely interesting is that this great property was simply obvious in theoretical models. Universal Turing Machines and primitive functions need an enumeration of programs (functions, respectively) in order to manipulate such objects. If such enumeration was not possible, most computer science would not have been created.

The von Neumann architecture also relies heavily on instructions being represented as numbers and stored along with data. It’s a bit of a pity that this properties have not been naturally ported to programming languages (except from Lisp and some others).
Indeed, we all agree that homoiconicity is a great property in programming languages. In fact, I often find frustrating complete lack of it (e.g., Java is completely not homoiconic, since its main data structure is the Object but Java programs are not even near to being objects – ok, you can always call the compiler get an AST… but this is not what the whole thing is about). Other languages have limited support for it.
In Python functions and classes (and objects) are available when compiling a module. The whole decorator concept, for example, relies on the fact that functions are objects and can be passed as such at compile time. Very powerful meta-programming techniques in Python rely on “program” manipulation, where programs are represented as classes and functions (and not syntactically, like in Lisp). This makes some kind of manipulation very hard (without resorting to byte-code manipulation – and then the same objection with the java compiler/AST thing applies – or to basic string manipulation, which I would not consider as a symptom of homoiconicity, since strings are not “the main data” in Python). However, I would not rule Python out of the bunch of homoiconic languages, either. Perhaps “partially homoiconic” applies? Or perhaps do we need better built in tools to manipulate classes and functions in Python to qualify for the full homoiconic(tm) certification from old lisp hackers[3]?

References

  1. McIlroy, M. D. 1960. Macro instruction extensions of compiler languages. Commun. ACM 3, 4 (Apr. 1960), 214-220. DOI
  2. MOOERS, C.N. TRAC, a text handling language. Proc. ACM 20th Nat. Conf. Cleveland, Aug. 1965, pp. 229-246.
  3. Not meant to be offensive… perhaps old could be replaced with ancient?

Using word to write...

Using word to write articles is a rather perverted art. E.g., why are the "regular expression" different from every other syntax I met and there are two different syntaxes depending whether you are using jolly characters or not?

Why is the bibliographic tools such a joke that almost everybody does it by hand (and, moreover, many conferences explicitly tell you to do it by hand? and the W2007 implementation is not compatible with W2003 – at least that is the opinion of W2007 on that matter)

And using images is a PITA. But that is a PITA with Latex as well. Last but not least, I have yet to understand why a PDF created with word for windows visualize bold characters (I'm following the IEEE template, so changing the font is not an option... besides, it's times new roman) with a *different* non-bold font when opened on the Mac. If it were not such a standard font I would have said it's a matter of not including the font.

Thursday, June 3, 2010

Yet another C++ unit-testing library (nano-unit++)

http://bitbucket.org/rik0/nano-unit/
Goals: be easy to use, be easy to install
Right now, it does less than needed. However, I plan to add features as I need them. Besides, I want to make it as DRY as C++ allows, even at the cost of fighting with template metaprogramming.
However, I hope I won't need boost, as it would be a rather important and expensive dependency.

Friday, May 28, 2010

Dropping Perforce

In the last year, I’ve been using Perforce extensively and my opinion about it matured and radically changed with respect to what I expressed in one of my last posts.
Essentially, there are no faults in Perforce I was not aware of or some other problem inherent with Perforce. I just found out I’m far more productive with other tools.
The first problem is typical of my setup and has nothing to do with Perforce: my home server connection is faulty. Sometimes there is no way to access the server from outside and sometimes the 1666 port is not reachable because of firewalls. Of course, it is not Perforce fault: in this situations it is often the case I could not reach 22 as well (that is the port I use mainly with both git and mercurial through ssh). However, I’ve been able to place the repositories on an external server (where I can’t just install the Perforce server) and I could always use github.
The main advantage with both Mercurial and Git is that if I have connection problems, I can just go on and work locally. Perforce simply sucks in this situation. By the way, when I’m travelling it is quite frequent I’m offline. Both git and mercurial allow me to work with no burden added.
Perforce uses a model different from other modern Source Control Management Systems (SCMSs) which somewhat resembles old locking SCMSs. Essentially, if you have to edit a file you have to mark it for editing. This can have several advantages, however, I never got used to it. Moreover, since I used it to manage .configuration files in my home directory as well, to change a single bloody line of code I had to:
  1. cd to the Perforce workspace directory
  2. mark the file for editing
  3. edit the file
  4. commit
Actually, I found this rather cumbersome. I do prefer git/mercurial that do not set permission of all (most[0]) files as not writable.
Moreover, with my desktop computer, my apple laptop, my netbook (dualboot win+lin) and a workstation I had to test some software I already used all the workspaces the free version of Perforce allows me to have. This is rather unfortunate, as I often use virtual machines as well. Of course paying the bloody perforce license to do this kind of things is just stupid.
In the meantime, I found that distributed SCMs had none of those problems. Backup is regularly performed by Time Machine, so that was not an issue either, as long as I keep a clone of each repository on my MacPro. Moreover, the remote server where I keep some projects is very well managed and thoroughly backed-up.
I like the branching/cloning strategy superior (according to my needs) than the integrate stuff Perforce uses. I suppose we can prove them equivalent, however, I like git/mercurial better. I also changed my view on rebase and other commit editing stuff. Sometimes, I find them useful. After all it is my repository: I’m free to screw it up. This does not mean I will or should.
So I stopped moving projects from Mercurial to Perfoce. On the other hand I’m using git to extract projects from Perforce (git-p4, in fact). Well, it works nice. I will keep my Perforce server only for historical purposes, as imported git-p4 project may lose some branching information.

[0] Actually you can manually set an option in order to override the thing

Sunday, May 23, 2010

Back in Black 2

Some things happened in the last two years. Two more Pycon Italia took place in Florence.
I completed a M.Sc. in Computer Science (in 2 years and 2 months, research dissertation included, which is something rare in my former). Two days after I graduated, I won a scholarship to attend a Ph.D. Course in Information Technology here in Parma, in the Dipartment of Computer Engineering.

This blog "re-edition" is kind of an experiment. Don't know if I have the time to write regularly here, nor if I like this platform. Essentially, I would have preferred a least web oriented platform (that is to say, writing posts offline). Blogspot should support this (I did import some of my old posts this way). Howerver, it seems that all of them can be imported. I don't really know why.

I also tried to write my own blog platform. Lack of motivation doomed this projects to failure. Moreover, I would need a rather costly high quality hosting which, by now, is an overkill as I can place massive amounts of data both on mobile me (public.me.com/enricofranchi) and on Dropbox.

I considered mobileme as a host, however, I would have limited choices on the platform (static HTML only). Rapidweaver/iWeb are not my cup of tea, either.

Thus, I chose an online blogging platform. As I already have lots of stuff with Google and I severly dislike Wordpress, I moved the whole thing here.

Thursday, April 9, 2009

Perforce Conversion?

I’ve always been critical on user interfaces. In two different senses.

Bad user interfaces drive me crazy: I think designing crappy user interfaces should be a criminal offence.

On the other hand, I’ve always been on the “command line side” when speaking about development tools.

Of course I use graphical editor (gvim is a graphical editor, and Emacs as well). But the center of my development is the shell. From the shell I call the editor, from the shell I run the build tools, etc etc etc. That’s to say I’m pretty “against” bloated IDE’s, even though recently I started some serious experimentation with Emacs, but that’s wholly another story.

Back to SCM I recently took some SCM classes (a whole course on source configuration management, indeed). And during the course, we had some lab exercises with Perforce. I never used Perforce before that: I knew there was this Perforce thing which was better than CVS but costed lots of $$$ and was closed source and if you used that you were almost as evil as if you used Windows.

In the meantime I got bore with the free software movement, started using the Mac (which is almost as evil as Microsoft) and lots of proprietary software which makes my life far easier. However, many teams switched to subversion (which is far better than CVS as well) and some times later I experienced the distributed SCM revolution. I started with git which I rapidly dropped favouring Mercurial.

Using Perforce seemed to me quite anti-climatic, in this sense. But Perforce had something I always desired in SCM tools. That is to say a very good graphical interface. Especially for version graphs. And a graphical merge tool. An integrated graphical merge tool, so that you don’t have to find one when on Windows, for example.

Moreover, branching is a breeze. The integrate command is something wonderful. Indeed I never used that much branching in my home projects. Now I just can’t thing how could I do without it. In this sense, the GUI is rather didactical: I just can explore menus and commands and learn new functions of my tool.

Moreover, installing the server is really easy (differently from subversion). And my dyndns powered machine can host all my work. Very interesting indeed.

I forgot to mention: as long as I use less than 2 users, I can use the Perforce server for free. Since they are my own projects...

Tuesday, February 24, 2009

Emacs, Carbon Emacs and color-theme

I don’t like very much standard emacs theme, especially when in “GUI” mode (that is to say Carbon Emacs/Aquamacs if you are on the mac).
Luckily color-theme module does an excellent job. I have a long list of themes ready to be selected. In fact, for nostalgic reasons I like the “Classic” theme.
I wanted this theme to be automatically loaded when I start Emacs. Unfortunately default OS X Emacs does not have color-theme installed (and I don’t mind the way it looks by default in the terminal). So it was only a matter of conditionally branching between GUI emacs and terminal Emacs.
Moreover, I could not use (require ‘color-theme), for this would give error when run in the terminal version. This is my solution:
(defun my-color ()
 (load-library "color-theme")
 (color-theme-initialize)
 (color-theme-classic))
    (if window-system
     (my-color))

Sunday, December 28, 2008

Missionaries, Cannibals and Prolog

Apparently, many generations of computer scientists have been concerned with the problem of letting cannibals and missionaries crossing a river with the very same boat.

Before them, logicians were looking for a way to have married couples doing the very same thing with slightly different inherent problems.

Centuries ago, the problem was that no woman should be with another man without her husband being present (this is now called a threesome). However, since women now are emancipated and nobody cares if a woman is with another man (with the possible exception of her own husband, especially when there are no football matches on TV), the cannibal problem seems more interesting.

It is widely accepted that if missionaries outnumber cannibals, they convert the cannibals. Since we believe in freedom of cult, no cannibal shall be converted. Someone may be familiar with another version of this problem, since once having people converted was considering a good thing, while having missionaries eaten was bad.

However, recently cannibals became vegans and until Churches will develop soy-missionaries the problem is how to let cannibals retain their own religion.

Basically, this is the job for a Prolog interpreter. Though humans solved this problem long ago, a human can only solve the problem for small numbers of cannibals and missionary (and though we hope both populations remain small in number, cannibals can always mate and missionaries can convert new people).

This is a short Prolog program that given an initial state (all cannibals and all missionaries on the same side of the river) and the desired final state (all cannibals and missionaries on the other side of the river), computes the strategy our poor bootman has to pursue in order to carry over the whole lot of people.

initial_state(mc, mc(left, bank(3,3), bank(0,0))).
final_state(mc(right, bank(0,0), bank(3,3))).

Above we say that the initial state has the boat on the left, three cannibals and three missionaries are on the left bank and no one is on the other one. The final state encodes what we have already said.

Now we have to describe possible moves:

move(mc(left, L, _R), Boat) :-
    choose_passengers(L, Boat).

move(mc(right, _L, R), Boat) :-
    choose_passengers(R, Boat).

choose_passengers(bank(C, _M), boat(2, 0)) :- 
    C > 1.
choose_passengers(bank(_C, M), boat(0, 2)) :- 
    M > 1.
choose_passengers(bank(C, M), boat(1, 1))  :- 
    C > 0, M  > 0.
choose_passengers(bank(C, _M), boat(1, 0)) :- 
    C > 0.
choose_passengers(bank(_C, M), boat(0, 1)) :- 
    M > 0.

Basically, when the boat is on the left bank, we choose passengers from the left bank. When the boat is on the right bank, we choose passengers from the right bank. We can only take 2 people on the boat. That means that, provided there is enough people on the bank, we can only carry 2 cannibals, 2 missionaries, 1 cannibal, 1 missionary or 1 cannibal and 1 missionary. Since the boatman is usually agnostic, the cannibal aboard with one missionary is not going to be converted.

The Prolog system choses one of the possible moves and proceeds. If the move is wrong, it back-tracks and tries another solution. Until the number of choices is small, it is rather efficient.

The predicates that describe how the system changes when one move is performed are:

update(mc(B, L, R), Boat, mc(B1, L1, R1)) :-
    update_boat(B, B1),
    update_banks(Boat, B, L, R, L1, R1).

update_boat(left, right).
update_boat(right, left).

update_banks(alone, _B, L, R, L, R).
update_banks(boat(C, M), left, 
             bank(LC, LM), bank(RC, RM), 
             bank(LC1, LM1), bank(RC1, RM1)) :-
    LM1 is LM - M,
    RM1 is RM + M,
    LC1 is LC - C,
    RC1 is RC + C.

update_banks(boat(C, M), right,
             bank(LC, LM), bank(RC, RM), 
             bank(LC1, LM1), bank(RC1, RM1)) :-
    LM1 is LM + M,
    RM1 is RM - M,
    LC1 is LC + C,
    RC1 is RC - C.

Eventually, we have to tell legal and illegal configurations apart. Basically, if in the bank we leave there are more missionaries than cannibals, we are breaking the law. If not, everything is fine (since the boatman works for free, so he does not have to pay taxes).

legal(mc(left, _L, R)) :-
    \+ illegal(R).

legal(mc(right, L, _R)) :-
    \+ illegal(L).

illegal(bank(C, M)) :-
    M > C.

The solution algorithm is trivial:

solve_dfs(State, _History, []) :-
    final_state(State).

solve_dfs(State, History, [Move|Moves]) :-
    move(State, Move),
    update(State, Move, State1),
    legal(State1),
    \+ member(State1, History),

solve_dfs(State1, [State1|History], Moves).

print_moves([]).
print_moves([M|Ms]) :-
    writeln(M),
    print_moves(Ms).

go :-
    initial_state(mc, State),
    solve_dfs(State, [State], Moves),
    print_moves(Moves).

We generate new moves and check if the updated state would be legal and if we did not get in the same state before (if it happens we are looping, the boatman becomes hungry and eats whoever is on his boat -- that’s the main reason why he does not travel with the empty boat).

The full source is here:

Tuesday, December 23, 2008

Using template programming for efficiency reasons

Suppose we want to write a

unsigned int pow(unsigned int n, unsigned int exp) 

function which computes the power of a given number.

A C implementation is rather trivial:

unsigned int
traditional_pow(unsigned int n, unsigned int exp) {

        unsigned int value = 1;

        for(unsigned int i = 0; i < exp; ++i) {

                value *= n;

        }

        return value;
}

Basically we could write the very same thing in C++. We could also perform some minor optimizations in the code in order to make it look more efficient. However, compilers are quite good in optimizing code. Indeed, they are better than humans.

Nonetheless, that function contains jumps. Notice that even though the exp parameter is known at compile time (which is a pretty common use-case), there is no way for the compiler to kill jumps short of doing a pretty expensive and thorough inter-procedural analysis. Which is not performed by gcc even using -O3.

The only result is having traditional_pow inlined in main. This is rather unsatisfactory: branching is among the more costly operations in modern CPU's.

We could try a recursive implementation:

unsigned int
traditional_rec_pow(
        unsigned int n,
        unsigned int exp,
        unsigned int acc) {
        switch(exp) {
                case 0: return 1;
                case 1: return acc;
                default:
                return traditional_rec_pow(n, exp-1, acc*n);
        }
}

unsigned int
traditional_rec_pow(unsigned int n, unsigned int exp) {
        return traditional_rec_pow(n, exp, n);
}

Here we are using an accumulator (a standard functional programmer's trick) in order to make the function tail recursive. With mild optimizations turned on, code generated for traditional_rec_pow looks like a less optimized version of traditional_pow (tail recursion is eliminated, though). Full optimizations yield a 74 lines long version of the function, where some special cases have been optimized.

We are very far from the intuitive idea of full optimization of pow if exp is known at compile time. We think that the compiler should be able to produce the very same code it would for the expression:
n * n * n * n * n * n

in case exp==5.

I promise you: we can do that.
I wrote the recursive functional version as an introduction to a templatic variant. The idea is that if exp is known at compile time, it can become a template parameter.

The structure remains similar, with template specialization used instead of explicit switch/if. This is even more readable for a functional programmer!



template<unsigned int exp> inline unsigned int
pow(unsigned int n, unsigned int acc) {

        return pow<exp-1>(n, acc*n);

}

template<> inline unsigned int
pow<1>(unsigned int n, unsigned int acc) {
        return acc;
}

template<> inline unsigned int
pow<0>(unsigned int n, unsigned int acc) {
        return 1;
}

template<unsigned int exp> unsigned int
pow(unsigned int n) {
        return pow<exp>(n, n);
}

Note that we give special cases for exp==1 and exp==0. They are needed because otherwise compilation would not terminate (well, it terminates with an error for implementation reasons).

Here some computation is implicitly performed at compile time. Suppose in the body of the main we call

pow<3>(3);

With no optimizations, the compiler generates some functions. There is the pow<1> specialization (which basically returns acc). Acc is the second parameter and according to OS X intel conventions that is 12(%ebp), while the return value is in %eax:

.globl __Z3powILj1EEjjj
                .weak_definition __Z3powILj1EEjjj
        __Z3powILj1EEjjj:
        LFB13:
                pushl        %ebp
        LCFI0:
                movl        %esp, %ebp
        LCFI1:
                subl        $8, %esp
        LCFI2:
                movl        12(%ebp), %eax
                leave
                ret
        LFE13:
                .align 1

For each exp value greater than one a function is generated. Each of this functions calls the one "before":

.globl __Z3powILj2EEjjj
                .weak_definition __Z3powILj2EEjjj
        __Z3powILj2EEjjj:
        LFB19:
                pushl        %ebp
        LCFI3:
                movl        %esp, %ebp
        LCFI4:
                subl        $24, %esp
        LCFI5:
                movl        12(%ebp), %eax
                imull        8(%ebp), %eax
                movl        %eax, 4(%esp)
                movl        8(%ebp), %eax
                movl        %eax, (%esp)
                call        __Z3powILj1EEjjj
                leave
                ret
        LFE19:
                .align 1
        .globl __Z3powILj3EEjjj
                .weak_definition __Z3powILj3EEjjj
        __Z3powILj3EEjjj:
        LFB18:
                pushl        %ebp
        LCFI6:
                movl        %esp, %ebp
        LCFI7:
                subl        $24, %esp
        LCFI8:
                movl        12(%ebp), %eax
                imull        8(%ebp), %eax
                movl        %eax, 4(%esp)
                movl        8(%ebp), %eax
                movl        %eax, (%esp)
                call        __Z3powILj2EEjjj
                leave
                ret
        LFE18:
                .align 1

Notice that __Z3powILj3EEjjj(pow<3>) calls __Z3powILj2EEjjj(pow<2>), which calls __Z3powILj1EEjjj(pow<1>). A part from this, most instructions deal with the stack (in order to create and destroy the current function stack frame) and multiply the first parameter for the second one, passing the result to the subsequent call.

Notice that no function contains conditional operations, tests or jumps a part from the call to the subsequent function in the chain. This is the kind of things optimizers shine on.

Indeed, even -O does the right thing. All the functions are inlined and the generated code is (not considering stack operations):
movl        8(%ebp), %eax
movl        %eax, %edx
imull        %eax, %edx
imull        %edx, %eax

Wow! This is how we would have written by hand. Notice that if we have multiple pow<n> calls, specialized ( and optimized code) is generated for each variant. For example this is for pow<11>


.globl __Z3powILj11EEjj
                .weak_definition __Z3powILj11EEjj
        __Z3powILj11EEjj:
        LFB17:
                pushl        %ebp
        LCFI0:
                movl        %esp, %ebp
        LCFI1:
                movl        8(%ebp), %eax
                movl        %eax, %edx
                imull        %eax, %edx
                imull        %eax, %edx
                imull        %eax, %edx
                imull        %eax, %edx
                imull        %eax, %edx
                imull        %eax, %edx
                imull        %eax, %edx
                imull        %eax, %edx
                imull        %eax, %edx
                imull        %edx, %eax
                leave
                ret