Geert JM Vanderkelen

This post explains how to compile Python v3.2 including support for the GNU Readline Library on MacOS X 10.7 (Lion), not using tools like MacPorts. One of the problems we try to solve is making history work in the interactive Python interpreter.

Pre-requirement is to have Apple’s XCode installed (check the Appstore).

First, you need a fresh copy of the GNU Readline Library and compile it. At the time of writing, readline v6.2 was the latest. Fire up a Terminal and do the following:

shell> tar xzf readline-6.2.tar.gz
shell> cd readline-6.2
shell> MACOSX_DEPLOYMENT_TARGET=10.7 ARCHFLAGS="-arch x86_64" \
       ./configure --prefix=/opt/local
shell> make && sudo make install

Note that you can change the location /opt/local to anything you like, but avoid /usr/lib or other system location.

When readline compiled and installed successfully, compile Python v3.2 as follows:

shell> tar xzf Python-3.2.3.tgz
shell> cd Python-3.2.3
shell> LDFLAGS="-L/opt/local/lib" CFLAGS="-I/opt/local/include" \
       ./configure --prefix=/opt/python/3.2.3
shell> make && sudo make install

Note that you might have more arguments to pass to the ./configure if you have special needs. Note also that /opt/python/3.2.3 is where Python will be installed to avoid clashing with the system provided Python installations.

You should now have history support in the Python interpreter:

shell> /opt/python/3.2.3/bin/python3
>>> import this
>>>

Using MacOSX, when your ISP or wireless access point is changing your hostname, make it sticky editing /etc/hostconfig.

Add or change the following line in /etc/hostconfig to reflect your hostname:

HOSTNAME="your hostname"

You could use Terminal.app to do this, but using Finder works too.

1. Open a Finder window.
2. Using the Menu: Go > Go to Folder..
3. In the dialog that’s opening, type: /etc/
4. /etc/ should now be available in your Finder window.
5. Locate the hostconfig-file and open it using your favorite text editor (e.g. TextEdit)
6. Add or change this line to reflect your hostname: HOSTNAME="your hostname"
7. Save it, enter your password, and be happy.

This post explains how to compile oursql and install it on MacOS 10.6. oursql is a Python database interface for MySQL, an alternative to MySQL for Python (i.e. MySQLdb) and MySQL Connector/Python.

First, find out which MySQL you installed. This can be either the 32-bit or the 64-bit version. To make sure, find the mysqld (e.g. in /usr/local/mysql/bin) and do the following in a Terminal window:

shell> file /usr/local/mysql/bin/mysqld
.../mysqld: Mach-O 64-bit executable x86_64

If you see x86_64, you got 64-bit, otherwise 32-bit. If you see both, then you have a universal build. This is important for specifying the ARGSFLAG when building.

Download oursql from Launchpad and unpack it into some directory. Using the information from above, you’ll have to do following for 64-bit platform (or universal build) in a Terminal window:

shell> ARCHFLAGS="-arch x86_64" python setup.py build
shell> sudo python setup.py install

For 32-bit, you’ll have to do:

shell> ARCHFLAGS="-arch i386" python setup.py build
shell> sudo python setup.py install

Following error will be reported when you don’t specify the correct ARCHFLAGS:

ld: warning: in .../lib/libmysqlclient.dylib,
file is not of required architecture

Tips:

• When building failed, it is good to remove oursql, unpack it and try again.
• If you don’t want to compile anything, or run into more troubles, give MySQL Connector/Python a try (alpha releases). It’s a pure Python implementation of the MySQL Client/Server protocol and doesn’t need compiling or a MySQL installation.
• You can download MySQL from either www.mysql.com or dev.mysql.com.

A few days ago MySQL 5.1.42 got released and it is now available with builds for MacOS X 10.6 (Snow Leopard)! The download website doesn’t show it yet, but if you are burning to try, you can get it from the mirror-picking-website.

As usual, don’t forget to checkout the changelog before upgrading!

If you want to compile it yourself, and need a universal binary, you could try my previous blog entry «Building MySQL universal binaries using MacOS X 10.6 (Snow Leopard)».

On the eve of 2010.. and your boss wants to stick to these MacOS X 10.5 machines, too stubborn or chicken to upgrade. Some developers still have their old PowerBook laptops and they need MySQL flying on PowerPC machines. To top it all, one guy said he wanted to have 32 and 64-bit in one bite. *Sigh* .. But there is an easy way out! A universal binary!

This post shows you a way to create MySQL universal binaries using MacOS X 10.6 so you can run them on MacOS X 10.5/10.6 whether it is PowerPC or Intel, or 32bit or 64bit.

However, if you need libmysqld (Embedded MySQL), this post will not work for you.

Requirements:

• You have MacOS X 10.6 with latest Xcode (fully) installed.
• The MySQL source unpacked somewhere. Get it on the MySQL download website under Source Downloads, package named Compressed GNU TAR archive (tar.gz).
• And some nerves for when the build process fails.

Most complete Universal Binary

First, here is away to build MySQL so it runs on MacOS X 10.5 Intel/PowerPC and 10.6 32 or 64-bit.

Here is the source of the build script names build.sh. Executed it while located inside the source directory of MySQL.

#!/bin/bash

SDK="-isysroot /Developer/SDKs/MacOSX10.5.sdk"
SDKLIB="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.5.sdk"
export MACOSX_DEPLOYMENT_TARGET="10.5"
PREFIX=/opt/mysql/mysql-5.1.42-universal-macosx-10.5

ARCH="-arch i386 -arch x86_64 -arch ppc"

export CFLAGS="-O2 -fPIC $ARCH $SDK"
export CXXFLAGS="-O2 -fPIC $ARCH $SDK"
export LDFLAGS="$ARCH $SDKLIB"

CC="/usr/bin/gcc-4.2"
CXX="/usr/bin/g++-4.2"
OBJC="/usr/bin/gcc-4.2"

INSTALL="/usr/bin/install -c"

./configure --prefix=$PREFIX \
--disable-dependency-tracking \
--mandir=$PREFIX/share/man --infodir=$PREFIX/share/info \
--localstatedir=$PREFIX/var/ --libdir=$PREFIX/lib \
--bindir=$PREFIX/bin --libexecdir=$PREFIX/bin \
--includedir=$PREFIX/include \
--datadir=$PREFIX/share/ --sysconfdir=$PREFIX/etc \
--with-extra-charsets=complex \
--with-mysqld-user=mysql \
--without-docs \
--with-plugins=all \
--enable-thread-safe-client --without-embedded-server \
--with-pic --with-libedit

if [ $? -eq 0 ]; then
    make clean
    time make -j 2
fi

Here is what file shows for the mysqld binary:

Black:mysql-5.1.42 geert$ file sql/mysqld
sql/mysqld: Mach-O universal binary with 3 architectures
sql/mysqld (for architecture i386): Mach-O executable i386
sql/mysqld (for architecture x86_64): Mach-O 64-bit executable x86_64
sql/mysqld (for architecture ppc7400): Mach-O executable ppc

These binaries were tested and work on MacOS X 10.6 Intel and MacOS X 10.5 PowerPC.

32/64-bit Universal binaries for MacOS X 10.6

Same as above, but with the following changes:

SDK="-isysroot /Developer/SDKs/MacOSX10.6.sdk"
SDKLIB="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.6.sdk"
export MACOSX_DEPLOYMENT_TARGET="10.6"
PREFIX=/opt/mysql/mysql-5.1.42-universal-macosx-10.6

Setting the above is probably not needed when you are already on a Mac running 10.6, but it doesn’t hurt to be explicit.

Need it for MacOS X 10.4?

I gave this a spin:

SDK="-isysroot /Developer/SDKs/MacOSX10.4u.sdk"
SDKLIB="-Wl,-syslibroot,/Developer/SDKs/MacOSX10.4u.sdk"
export MACOSX_DEPLOYMENT_TARGET="10.4"
ARCH="-arch i386 -arch ppc"

But it failed with

/Developer/SDKs/MacOSX10.4u.sdk/usr/include/stdarg.h:4:25:
  error: stdarg.h: No such file or directory

.. but I lack intrest making stuff for MacOS X 10.4 (Tiger). Consider this your homework!

A week ago we found a workaround for a bug in MySQL Cluster making it impossible to run a management node on MacOS X. Until the bug is fixed, you should use the --nodaemon option for the ndb_mgmd executable. Both MySQL Cluster v6.3 and v7.0 are affected.

Currently, I’m starting the management node like this:

 (
  cd /opt/mysql/mysql ; 
  ./libexec/ndb_mgmd -f /opt/mysql/config.ini \
    --nodaemon 2>/dev/null 1>&2 </dev/null &
 )

Obviously, you’ll want to change the paths.

Eventually, the bug will get fixed, but until then you got no excuse to not try MySQL Cluster on Mac!

What you do when you’re fed up with a script? Right, you write your own.You’ll have to excuse me for the long shell script you’ll find here below, but I’m not going to bother putting it on some download website.

It’s a shell script which starts and stops the MySQL server. Indeed, a replacement for the init.d script found in the MySQL distributions. I’m using it personally on my Macs and it’s not supported in any way.

But why? Well, I’m playing with MySQL Workbench, Server Administration. The MySQL init.d script didn’t work right away (oh, various reasons for that), so I used mine. So I figured it might be useful for others and it’s not complicated or shocking-new-stuff. If you want to use it, you’ll have to edit the 2 variables at the top. It’s only going to work on MacOS X.

#!/bin/bash
# Author: Geert Vanderkelen <geert@kemuri.org>

BASEDIR="/opt/mysql/mysql"
CNF="/opt/mysql/my51.cnf"

MODE="$1"
_RETURN="" # for returning from functions

pidof() {
    local CMD=$1
    local PID=`ps cax -o "pid,command" | awk -v cmd="$CMD" '{ if (\$2 == cmd) printf("%s",\$1) }'`
    if [ "x$PID" == "x" ]; then
        _RETURN=""
        return 1
    else
        _RETURN=$PID
        return 0
    fi
}

checkprocess() {
    local CMD=$1
    _RETURN=""
    pidof $CMD
    if [ $? -eq 0 ];
    then
        local PID=${_RETURN}
        kill -n 0 $PID 2>/dev/null 1>&2
        if [ $? -ne 0 ]; then
            return 2
        fi
        _RETURN=$PID
        return 0
    fi
    
    return 1
}

waituntildown() {
    local CMD=$1
    pidof $CMD
    while [ $? -eq 0 ]; do
        echo -n '.'
        sleep 1
        pidof $CMD
    done
}

exec_mysqld_safe() {
    (
        cd $BASEDIR
        ./bin/mysqld_safe --defaults-file=$CNF 2>/dev/null 1>&2 </dev/null &
    )
}

status() {
    checkprocess mysqld
    RET=$?
    if [ $RET -ne 0 ]; then
        if [ $RET -eq 1 ]; then
            echo "MySQL is not running."
        elif [ $RET -eq 2 ]; then
            echo "You have no permission to stop MySQL."
        fi
        return 1
    fi
    return 0
}

start() {
    checkprocess mysqld
    if [ $? -eq 0 ]; then
        echo "MySQL is running."
        exit 1
    fi
    echo -n "Starting MySQL.. "
    exec_mysqld_safe
    sleep 2
    checkprocess mysqld
    if [ $? -ne 0 ]; then
        echo " Failed!"
        exit 1
    fi
    echo " OK"

}

stop() {
    status
    RET=$?
    if [ $RET -ne 0 ]; then
        exit 1
    fi
    PID=${_RETURN}
    echo -n "Stopping MySQL.. "
    kill $PID 2>/dev/null 1>&2
    if [ $? -eq 0 ]; then
        waituntildown mysqld
        echo " OK"
    else
        echo " Failed!"
    fi
}

case "$MODE" in
    "start")
        start
    ;;
    
    "stop")
        stop
    ;;
    
    "restart")
        stop
        start
    ;;
    
    "status")
        status
        if [ $? -eq 0 ]; then
            echo "MySQL is running."
            exit 0
        else
            exit 1
        fi
    ;;
    
    "getpid")
        pidof mysqld
        if [ $? -eq 0 ]; then
            echo ${_RETURN}
        fi
    ;;
    
    *)
        echo "Usage: $0 {start|stop|restart|status|getpid}"
        exit 1
    ;;
esac

Work says that we need to store sensitive data like email and customer files on some encrypted media. This is a good thing. My laptop has my home directory secured, but I don’t want to encrypt everything on my desktop. The solution to this is to create an encrypted Disk Image (using Disk Utility) and make Mail.app store my email there. This all works great!

The problem surfaced this week when I decided it would be good to shutdown my desktop to save energy (which I usually did, but I got sloppy). What happened? When Mac OS X shuts down, and there is still some application doing something with your opened Disk Image, it will not remove the mount-point, e.g. in my case /Volumes/FileVaultBlack. Later, if you open it again, Mac OS will create the directory /Volumes/FileVaultBlack 1 and use that as mount-point. The issue here is that I configured the AccountPath in Mail.app to a location in /Volumes/FileVaultBlock.. In mortals speech: “Email is broken”.

The solution is to mount the Disk Image when logging in using a predefined mounting point. This involves making a directory and creating an AppleScript application. So lets get busy!

First, eject the Disk Image using the Finder application. It will complain when applications still need it, so you have to make sure that nothing is accessing it.

Create the mountpoint. The idea is to create a directory in /Volumes/ (it doesn’t really matter where you do it, but I like to keep them in one place). I suggest opening Terminal for this and do the following:

 shell> mkdir /Volumes/FileVaultBlack

You want to name it differently. I’m using the name of the Disk Image, just like MacOS would do when you double click the .dmg-file.

Open AppleScript Editor and save the following script into a file called, for example, FileVaultBlack_mountatlogin.

do shell script "hdiutil attach /Users/geert/FileVaultBlack.dmg -mountpoint /Volumes/FileVaultBlack"

The above script will execute the hdiutil command-line utility telling it to mount the given Disk Image (.dmg-file) on the give mount-point (the one we created earlier).

Test this script by pushing the Run-button in the AppleScript Editor and see whether your Disk Image is visible in Finder. If not, check the locations of both the Disk Image and the mount-point you gave in the script.

When you saved it, save it again as an application. Still within the AppleScript Editor do the following:

1. Choose Save As from the File menu
2. Save As: save it using slighly different name, for example MountFileVaultBlack
3. File Format: Application
4. You don’t have to check Run Only

We save it 2 times, once as a script, once as an application. This way, you can easily edit the script later, and save it again as an application. (I’m using different names because it’s less confusing when writing this post.)

Lets now add it to our Login Items:

1. Open System Preferences and go to Accounts
2. Make sure you choose My Account, i.e. your account, and go to Login Items
3. Add an item to the list, clicking the +-sign
4. Locate the application you created above, in our exampled named MountFileVaultBlack, and press Add.
5. Check also the Hide option next to the new item.

That should be it. Log out and in and check if the Disk Image is mounting. There should be an icon happily jumping in the Dock while the mounting is on-going. Open Finder and see if your files are accessible.

This bugged me for a long time, and just today I googled for a solution. And it’s darn simple! To set a new default column size, size them while holding the Option (Alt) key. Open a new Finder window, et voila: default size changed!

There are lots of these little tricks in Mac OS (and I’m sure in other desktop GUIs). Fun to find one each day!

Yesterday I gave Google’s Picasa again a try. Importing my iPhoto library I ran into the now infamous error message: “corrupted AlbumData.xml”. The Apple support website has some tips on how to fix this, but it’s too vague. Luckily there are smart people out there and I stumbled upon a good way in the Picasa help forums to debug your AlbumData.xml using TextWrangler.

1. Install TextWrangler if you haven’t already.
2. Open your iPhoto Library using Finder: right-click (Ctrl-Click) and select ‘Show Package Contents’. Copy the AlbumData.xml file to your desktop.
3. Open the AlbumData.xml from on your Desktop using TextWrangler.
4. From the Text-menu, choose ‘Zap Gremlins’.
5. Deselect ‘Non-ASCII Characters’ and select ‘Replace with: •’.
6. Wait for TextWrangler to finish the zapping! Can take a while.
7. When done, search (Cmd-f) for occurrence of the • (Option-8)
8. Open iPhoto and fix wherever you find a dot in the copy of AlbumData.xml

If you are lucky, you’ll find quickly the character that is bugging Picasa. I had only one of these Gremlins in the XML file.

So, I gave Picasa a try, but it failed importing captions of the pictures, only tags. I haven’t found a fix/workaround for that, so it’s a no-go for me still.