Do you want to see how PyInstaller magically handles usages of ctypes, whilst py2exe/py2app would simply fail? Follow me in this in-depth tutorial!

PyInstaller comprises many advanced features to make your life easier when packaging a Python program. In my opinion, its main advantage over py2exe/py2app (besides the obvious fact that it is a multi-platform tool) is that it includes many workarounds and quirks necessary to make applications work after packaging. What py2exe “fixes” by adding a page to their wiki, PyInstaller implements it properly, for everybody’s pleasure.

A very good example of this philosophy is the support for ctypes. I am sure most of you are familiar with this little gem, which allows a programmer to dynamicall bind to a shared library and invokes its functions without writing a binding. For instance, let’s say you have some existing and very complicated C code like the following:

/* foobar.c */
#include <stdio.h>
void fooize(int a)
{
   printf("Fooized %d!\n", a);
}

Which is compiled to a shared library:

$ gcc -shared -fPIC -o foobar.so foobar.c

Now, you can access this from Python by simply doing something like:

#!/usr/bin/env python
# example.py
from ctypes import *
foobar = CDLL("foobar.so")
foobar.fooize(4)

which obviously produce the following output when executed:

$ LD_LIBRARY_PATH=. python example.py
Fooized 4!

ctypes handled all the magic for me: it called dlopen(3) to access the dynamic library, it called dlsym(3) to access the symbol fooize, and it even handled marshalling of arguments. This is great.

But what happens if I want to package this application? Well, with py2exe/py2app, it wouldn’t work. In fact, they would totally miss the dependency of the example program on the foobar.so file. This obviously happens because they support only the simplest forms of dependency: a plain Python import statement or dependencies between shared librarys (eg: foo.dll that depends on bar.dll).

Now, watch PyInstaller in action:

$ python ~/src/pyinstaller/Makespec.py --onefile example.py
wrote /tmp/ct/example.spec
now run Build.py to build the executable
$ python ~/src/pyinstaller/Build.py example.spec
checking Analysis
building Analysis because outAnalysis0.toc non existent
running Analysis outAnalysis0.toc
Analyzing: /home/rasky/src/pyinstaller/support/_mountzlib.py
Analyzing: /home/rasky/src/pyinstaller/support/useUnicode.py
Analyzing: example.py
Warnings written to /tmp/ct/warnexample.txt
checking PYZ
rebuilding outPYZ1.toc because outPYZ1.pyz is missing
building PYZ outPYZ1.toc
checking PKG
rebuilding outPKG3.toc because outPKG3.pkg is missing
building PKG outPKG3.pkg
checking EXE
rebuilding outEXE2.toc because example missing
building EXE from outEXE2.toc
Appending archive to EXE /tmp/ct/dist/example
$ ./dist/example
Fooized 4!
$ ls -la ./dist
totale 2927
drwxr-xr-x 2 rasky rasky      72 2010-03-19 03:33 .
drwxr-xr-x 4 rasky rasky     344 2010-03-19 03:33 ..
-rwxr-xr-x 1 rasky rasky 2991835 2010-03-19 03:33 example

It simply worked. What PyInstaller behind the hood is:

• Scan the bytecode of example.py, looking for usages of ctypes
• Collect all the shared libraries used by ctypes calls. Notice that it can’t infer all filenames potentially used by the source code (how would you handle the case where the library named is specified interactively by the user?), but it handles all simple cases (see this page for more details).
• Resolve pathnames using the OS’ library search path.

Notice how foobar.so was also included within the single-file package produced by PyInstaller. And how can this possibly work? Let strace show us the trick:

$ strace -ff ./dist/example 2>&1 | grep foobar.so
stat("/tmp/_MEIDG0H9t/foobar.so", 0x7fff92f09480) = -1 ENOENT (No such file or directory)
open("/tmp/_MEIDG0H9t/foobar.so", O_WRONLY|O_CREAT|O_TRUNC, 0666) = 4
[pid 13942] open("/tmp/_MEIDG0H9t/foobar.so", O_RDONLY) = 6
stat("/tmp/_MEIDG0H9t/foobar.so", {st_mode=S_IFREG|0700, st_size=7985, ...}) = 0
unlink("/tmp/_MEIDG0H9t/foobar.so")     = 0

What happens at runtime is that foobar.so is extracted in a temporary directory, so that ctypes can find it.

So: PyInstaller is designed to do everything required to package an application. If you like this philosophy and you are tired of chasing workarounds in wikis, you should give it a try!