All weblog entries

2017-02-21 Linux VPS with TeamViewer

Here are my short notes on creating a Linux VPS (virtual private service) which can be remotely accessed via TeamViewer. I prefer TeamViewer over other ways of remotely accessing a desktop, because it works cross-platform and through NAT and firewalls.

The big problem is often that a Linux VPS doesn't have a virtual graphical card. For remote access to the GUI, most documentation advises VNC while I prefer TeamViewer. I've tried a number of Linux distributions on a number of VPS providers, but the combination of Linode and Fedora 25 made this all very easy (provided you can live with a maximum resolution of 1280x768).

First of all, register or log in at Linode, and create a VPS. The smallest one already has 1 GB of memory so that'll do.

After it's started, log in via SSH as root and do the following:

  # yum -y groupinstall "Fedora Workstation"

Then install TeamViewer:

  # wget ""
  # yum -y install ./teamviewer.i686.rpm

Next reboot, make sure the GUI starts:

  # systemctl set-default

Then, add a user for yourself, and add it to the sudoers:

  # useradd -m -U bartvk
  # reboot

Open remote access via glish, which is Linode's way of giving you graphical access to your VPS, through your browser. Log into the Linode management console, click on the node, then in the tab Remote Access, click the link "Launch Graphical Web Console".

You should see the graphical Linux login screen. In the top left corner, set shell to "Gnome Xorg" (this is important!) and then continue to log into Gnome.

Linode login 2017-02-21.png

In Gnome, start TeamViewer. Check the three boxes to enable remote access. You'll need to provide TeamViewer username/password, as well as click the link in the email you'll get, to confirm adding this VPS to your TeamViewer device list.


A current disadvantageis that you can't set the resolution to something bigger than 1280x768.

2017-02-03 Create your own laptop battery test

Recently I wanted to test how long the battery of my 2013 MacBook Air lasts. The quickest solution I found, is as follows:

  • Download Firefox.
  • Get the iMacros for Firefox extension.
  • Make a list of, say, ten websites.
  • After installation, click the iMacros button so the sidebar appears.
  • Open a new tab for the purpose of recording the macro.
  • Click record.
  • Type in these ten sites. Just type the URL in the address bar. Clicking anywhere will often result in failed playback. Searching for a keyword in Google is fine, though.
  • Stop recording and edit the macro. Add the line "WAIT SECONDS=5" a couple of times, to simulate the time spent reading.
  • Play the macro once to make sure you won't have errors occuring during playback.
  • In the settings of iMacros, set playback speed to slow.

Now to test the battery:

  • Click on the battery icon in the menubar and click "show percentage". It should obviously show 100% at this point because we want to start fully charged.
  • In System Preferences, turn off the screensaver. Under Energy Saver, set "turn display off after..." to "Never".
  • Note the current time somewhere, then remove power cord.
  • Set the screen brightness to something reasonable, like 75%.
  • Set the loop field to 10000 or some other high number.
  • Click the Play Loop button.
  • Every half hour, have a look at the battery level and write it down.

So this is my ghetto battery test. It's a bit rough around the edges, but should give you an idea of how many hours the battery lasts. Don't take the results too serious, this is meant to get a ballpark figure.

2017-02-02 Swift app without Interface Builder

Here's an example of an AppDelegate.swift, for those who don't like Interface Builder:

    import UIKit
    class AppDelegate: UIResponder, UIApplicationDelegate {
        var window: UIWindow?
        func application(_ application: UIApplication,
            didFinishLaunchingWithOptions launchOptions: [UIApplicationLaunchOptionsKey: Any]?) -> Bool {
            self.window = UIWindow(frame: UIScreen.main.bounds)
            let vc = ViewController()
            let nc = UINavigationController(rootViewController: vc)
            self.window?.rootViewController = nc
            return true

Remove the Main.storyboard file from the project. And in the Info.plist of your project, remove the entry called "Main storyboard file base name:

main storyboard file base name 2017-02-03.png

2017-01-12 Finder shortcuts

I couldn't find a one-page PDF with all macOS Finder shortcut keys. So I made one: Finder shortcuts.pdf. Here is the original, so you can adjust to your liking: Finder shortcuts.xlsx. Both are licensed under the Creative Commons Attribution 4.0 International License.

2016-12-25 Firefox crashes on late 2016 MacBook Pro

I've been using Firefox on a late 2016 MacBook Pro, running macOS 10.12.2. Every time the laptop wakes from sleep, Firefox has crashed. To remedy this situation, I've created a Hammerspoon script to kill Firefox when the system goes to sleep, and start it upon waking. In Firefox Preferences -> General, you can configure a setting called "When Firefox starts". Set this to "Show my windows and tabs from last time". Then install Hammerspoon, create the folder .hammerspoon in your home directory and create the file init.lua in that new folder. Then paste the following code and in Hammerspoon, reload the configuration.

    local log ='mywatcher','debug')
    function callback(eventType)
        if (eventType == hs.caffeinate.watcher.systemWillSleep) then
            local firefox = hs.application.find('Firefox')
            if not (firefox == nil) then
        elseif (eventType == hs.caffeinate.watcher.systemDidWake) then
            local firefox = hs.application.find('Firefox')
            if (firefox == nil) then
                firefox ="/Applications/")
    local mywatcher =

2016-11-29 Firefox on macOS

To have Firefox look a little more at home on macOS, check out the following addons:

I've also noticed that Firefox 50.0.2 sometimes crashes when disconnecting the external monitor from my 2016 MacBook Pro. After restarting, it brings up the Restore Session screen with the "Well this is embarrassing" message. To turn this off, go to about:config, search for resume_from and double-click on the value browser.sessionstore.resume_from_crash so it's set to false.

2016-11-14 Questions when looking for a coworking space

I've heard the figure that every four hours, a new coworking space is created. Personally, I've got a permanent desk in a coworking space, which allows me to put down a decent monitor, some personal stuff, a USB hub with a decent keyboard and mouse, et cetera. If you're also looking for a nice (semi-)permanent desk somewhere, here's a list of questions to get a feel of the place:

  • Who is the day-to-day floormanager?
  • Who has access to this room?
  • Are desks and chairs supplied, or should I bring my own?
  • Do I get an ethernet cable? This is very important because WiFi can be iffy.
  • Is there a lock on the door? What quality is the lock and the door itself? Can I get additional keys? Can I change the lock if necessary?
  • What's the speed of the internet at work times? If they don't know, ask whether you can place high-quality Skype and Facetime calls.
  • What's the level of sound during the day? Are there people here whose main job it is to call around?
  • What kind of businesses are next to this room? Above and below it?
  • Can I enter the building 24/7 or are there limits?
  • Is cleaning included? How often is this done? What's the cost?
  • What's the cost of parking a car? Are their spots to charge an electric car?
  • Any additional costs for internet, heating, et cetera?

Be sure to write down these answers. I've had the unfortunate situation where an office manager would give me a room next to a studio, and forgot to mention this. Luckily a desk was available in another coworking room.

2016-11-08 AutoLayout in a playground

What got me confused today, was how to play with AutoLayout in a Playground. The trick is to first create a main view, akin to the view in a View Controller, that is sized with a frame (without constraints).

After that's set up, add subviews that use AutoLayout. The following works for Xcode 8.1:

    import UIKit
    let view = UIView(frame: CGRect(x: 0, y: 0, width: 500, height: 500))
    view.backgroundColor = UIColor.white
    view.layer.borderWidth = 2.0
    let licensePlateView = UIView()
    licensePlateView.translatesAutoresizingMaskIntoConstraints = false
    licensePlateView.backgroundColor = UIColor.gray
    licensePlateView.leadingAnchor.constraint(equalTo: view.leadingAnchor).isActive = true
    licensePlateView.topAnchor.constraint(equalTo: view.topAnchor).isActive = true
    licensePlateView.widthAnchor.constraint(equalToConstant: 200).isActive = true
    licensePlateView.heightAnchor.constraint(equalToConstant: 50).isActive = true


autolayout in Xcode 2016-11-08.png

2016-11-04 Veertu now open source

My favorite virtualization software Veertu has been open sourced! Check out their blog entry:

Version 1.2.90 is now available from their website, or via Brew Cask:

 $ brew cask install veertu-desktop

2016-11-03 UIStackView distribution

Nice site that graphically shows the options for UIStackView distribution and alignment:

2016-10-31 Mice and keyboards with USB-C

After seeing Apple coming out with new 13" and 15" machines with only USB-C connections, I wondered whether there are mice with USB-C. This could be a choice when you don't want or can't use Bluetooth due to preference, regulations or interference. I was very surprised that it's very slim pickings. Big brands like Logitech haven't gotten around to those, and the only one I've been able to find is this one:

Currently I'm unable to find keyboards with USB-C, so a good option is to get one of those keyboards with a detachable cable and simply replace it with a USB-C cable. The CoolerMaster keyboards usually (all?) have detachable cables.

2016-10-17 Compare floats in Swift

To compare floats to zero, or each other, use the following functions:

    func fequal(_ a: Double, _ b: Double) -> Bool {
        return fabs(a - b) < DBL_EPSILON
    func fequal(_ a: Float, _ b: Float) -> Bool {
        return fabs(a - b) < FLT_EPSILON
    func fequal(_ a: CGFloat, _ b: CGFloat) -> Bool {
        return fabs(a - b) < CGFloat(FLT_EPSILON)
    func fequalzero(_ a: Double) -> Bool {
        return fabs(a) < DBL_EPSILON
    func fequalzero(_ a: Float) -> Bool {
        return fabs(a) < FLT_EPSILON
    func fequalzero(_ a: CGFloat) -> Bool {
        return fabs(a) < CGFloat(FLT_EPSILON)

To test, paste the above and the below code in a playground:

    var a: Double = 0.1
    var b: Double = 0.2
    var c: Double = 0.3
    let d = a + b
    if c == d {
    } else {
        print("Not equal")
    if fequal(c, d) {
    } else {
        print("Not equal")

2016-10-06 Walking through a String in Swift 3

This is how you walk through a string in Swift 3:

    let str = "Hello, playground"
    let stepsize = 3
    let len = str.characters.count
    for start in stride(from: 0, to: len, by: stepsize) {
        var end = start + stepsize
        if end > len {
            end = len
        let startIndex = str.index(str.startIndex, offsetBy: start)
        let endIndex = str.index(str.startIndex, offsetBy: end)
        print("From \(start) to \(end)")

Note: not optimized in any way.

2016-09-22 Swift 3 naming

Swift 3 is different from version 2 in the naming department. The short version is that the first label of a function is now mandatory, and that the function signature should reflect this.


    func pageViewsfromContent(content: ENFormContent) -> [VIFormPageView]


    func pageViews(fromContent content: ENFormContent) -> [VIFormPageView]

See also:

2016-09-13 weird error when creating constraint in Swift 3

Today, I've transitioned a project to Swift 3 and I bumped into a crash. The crash was as follows:

    -[_SwiftValue nsli_superitem]: unrecognized selector sent to instance 0x170249a50
    *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[_SwiftValue nsli_superitem]: unrecognized selector sent to instance 0x170249a50'
    *** First throw call stack:
    (0x1816741c0 0x1800ac55c 0x18167b278 0x181678278 0x18157259c 0x1820ab104 0x1820a9948 0x1820a879c 0x1820a8340 0x100176cc0 0x10012071c 0x100123474 0x1874b55c8 0x1874cd15c 0x1876556ac 0x18756dd74 0x18756d9dc 0x18756d940 0x1874b2738 0x18497b40c 0x1849700e8 0x18496ffa8 0x1848ecc64 0x1849140d0 0x1874a7e08 0x1816217dc 0x18161f40c 0x18161f89c 0x18154e048 0x182fd1198 0x187520818 0x18751b550 0x10002387c 0x1805305b8)
    libc++abi.dylib: terminating with uncaught exception of type NSException

Turns out this happens at the end of the following bit of code:

 scrollView = UIScrollView()
    scrollView.translatesAutoresizingMaskIntoConstraints = false
    contentView = UIView()
    contentView.translatesAutoresizingMaskIntoConstraints = false
    let viewDict = [
        "contentView": contentView,
        "scrollView": scrollView,
    let vFormat = "V:|[contentView]|"
    let constraints = NSLayoutConstraint.constraints(withVisualFormat: vFormat, options: [], metrics: nil, views: viewDict)

Yeah, I've been creating UIView objects by hand instead of using Interface Builder. Like a savage! After playing with the debugger, it turns out the compiler infers the type of the viewDict as a [String: UIView?] and do notice that UIView optional.

The problem is fixed by declaring the viewDict as follows:

  let viewDict: [String: UIView] = .....

Just leaving this here for DuckDuckGo, plus I've answered a similar question on StackOverflow.

2016-08-16 Filter array in Swift

Here's a bit of code that you can throw in a playground. Given an array with a bunch of objects, it'll filter the array based on a property. Then it'll print the result.

    class MyObj: CustomStringConvertible {
        var enabled = true
        var text = ""
        init(enabled: Bool, text: String) {
            self.enabled = enabled
            self.text = text
        var description: String {
            return "MyObj=\(text)"
    let collection = [
        MyObj(enabled: true,  text: "ja"),
        MyObj(enabled: true,  text: "jawohl"),
        MyObj(enabled: false, text: "no"),
        MyObj(enabled: false, text: "nope"),
        MyObj(enabled: false, text: "non"),
        MyObj(enabled: true,  text: "yep"),
        collection.filter { $0.enabled }
                  .map { $0.text }

2016-08-11 UIImage nsli superitem

Got this weird error today while futzing around with AutoLayout, or more specifically VFL, Visual Format Language:

    2016-08-11 11:05:00.915 TestTruckButton[74067:6491323] -[UIImage nsli_superitem]: unrecognized selector sent to instance 0x7fb8fbc4ae80
    2016-08-11 11:05:00.924 TestTruckButton[74067:6491323] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '-[UIImage nsli_superitem]: unrecognized selector sent to instance 0x7fb8fbc4ae80'
    *** First throw call stack:
    0   CoreFoundation                      0x000000010301cd85 __exceptionPreprocess + 165
    1   libobjc.A.dylib                     0x0000000104dc3deb objc_exception_throw + 48
    2   CoreFoundation                      0x0000000103025d3d -[NSObject(NSObject) doesNotRecognizeSelector:] + 205
    3   CoreFoundation                      0x0000000102f6bcfa ___forwarding___ + 970
    4   CoreFoundation                      0x0000000102f6b8a8 _CF_forwarding_prep_0 + 120
    5   Foundation                          0x00000001034037af -[NSLayoutConstraintParser findContainerView] + 49
    6   Foundation                          0x00000001034033ae -[NSLayoutConstraintParser finishConstraint] + 691
    7   Foundation                          0x0000000103402986 -[NSLayoutConstraintParser parse] + 375
    8   Foundation                          0x00000001034025c6 +[NSLayoutConstraintParser constraintsWithVisualFormat:options:metrics:views:] + 91
    libc++abi.dylib: terminating with uncaught exception of type NSException

This happened because you tried to use an UIImage instead of an UIImageView in VFL. Just leaving this here so AltaVista & Co. can find it.

2016-06-12 Veertu the hidden gem

For my last project, I've been using Veertu. I consider it the hidden gem of desktop virtualization. The big boys in this space are basically Parallels, VMWare and VirtualBox. They all require loads of kernel drivers, and personally I like my Mac as bog-standard as possible.

But since last year, there is Veertu. It uses Apple's own Hypervisor framework and it's distributed through the Mac App Store. I consider this a major accomplishment. Recently some Mac apps have been leaving the store citing limitations of one or the other, but these guys have managed to get their virtualization app running in the sandbox that the App Store requires!

Amazing, really. And it's a good product as well. I've been using it to test Ansible scripts. The backend of my client's project was deployed via Ansible but the script hadn't been maintained for some time. It was very nice to kick up a fresh Ubuntu server instance, test the Ansible script and log any issues that occurred so our backend guy could have a look.

And this is a small thing, but shows attention to detail: the IP address is shown in the list of running VMs: veertu5 shows IP-fs8 2016-06-12.png

Veertu is first and foremost an OS X app so looks and behaves very much like you'd expect, unlike for instance VirtualBox. I've read about some limitations related to graphics performance and USB support. For my purposes, I haven't run into these. I did notice it doesn't yet support sparse files, in the sense that when you reserve 20 GB for your VM, it really does reserve 20 gigs on disk. In my case that wasn't a problem; my VMs don't have a long life.

They're updating it all the time; last update was roughly three weeks ago. So for instance the sparse files thing seems to have been fixed.

In any case, since Veertu is free for basic Linux usage, why not try and see if it supports your use case? To give it a shot with the least amount of hassle, install and start Veertu, add a VM and pick Ubuntu Gnome 16.04 desktop. After downloading, the VM will be started. Then instead of installing, select "Try Ubuntu":

veertu ubuntu 2016-06-12.png

Some links:

2016-06-02 Good error messages in Xcode Run Scripts phase

Xcode has an option to run a script when building. This is necessary for example when using Carthage. Reading the instructions, Carthage says: "Create a Run Script in which you specify your shell (ex: bin/sh), add the following contents to the script area below the shell: /usr/local/bin/carthage copy-frameworks"

However, to make things easy for your (new) colleagues, I urge you to create an extremely clear error message. You can do this as follows. First, when adding a Run Script phase, change the shell to /bin/bash. Then add an error message by checking for prerequisites, and printing to stderr combined with an "exit 1". For Carthage you could do the following:

    if [[ -e "$CARTHAGE" ]]; then
        /usr/local/bin/carthage copy-frameworks
        echo "--------------------------------------------------------"  1>&2
        echo 1>&2
        echo "Please install carthage, see also:"  1>&2
        echo ""  1>&2
        echo 1>&2
        echo "--------------------------------------------------------"  1>&2
        exit 1

This results in the following very obvious message when Carthage is not installed:

carthage build error 2016-06-06.png

I don't think anybody would object to such a nice pointer in the right direction.

2016-05-26 Printing on a Kyocera FS2020D from OS X

Here's how you can install a Kyocera FS2020D printer on OS X.

First, download and install the driver: This will cause an additional icon to appear in the System Properties app. It's labeled "Select PDL". You can ignore it.

If you have the hostname or IP address of the printer, great. If not, find out the IP address of the printer as follows. On the printer, press the menu button, then arrow right and arrow down. The display should say "status page". Press OK and a single status page is printed. It'll mention a couple of things, but most importantly its IP address.

On OS X, in System Preferences, go to Printers & Scanners, add a printer with the plus button, go to the second tab and fill in the IP address.


That should be all you have to do. If you wish, adjust the name, then click the Add button.

Happy printing.

2016-05-25 Quote from Philip Morgan

Here's a nice quote from Philip Morgan:

"If you look at your marketing message and say, “wow, there are probably only 10 companies out there that fit that profile”, then you are in the right ballpark in terms of specificity. The reality is that there will be far more than 10 candidates, and those 50 or 100 truly ideal clients will find your marketing extremely compelling. Trust me, they won’t find it compelling if you don’t have the courage to be extremely specific.”

2016-05-24 home shortcut key for Firefox on OS X

If for some reason you are moving from Safari to Firefox on OS X, you might miss the Cmd-Shift-H shortcut key to go to the homepage. On Firefox, this is by default Option-Home but since there's no Home key on Apple keyboards, this becomes the Fn-Option-LeftArrow shortcut which I can never remember.

An alternative is to press Option-K, Enter. It'll focus on the search field in the toolbar, and pressing enter will take you to the search engine home page.

If you have removed the search field from the toolbar, then simply pressing Option-K will be enough. Very useful when you don't want to close the last remaining tab, but want to clear the current page. Update 2016-11-22: this last option no longer works because it'll open the options menu and focus on the search field there.

2016-05-24 Swift example of CoreData max query

Here's a copy/paste example of doing the equivalent of a SQL MAX() function. I know, I know "CoreData is an object graph". It's in a project where I had to keep track of the sequence in which entities were added. As far as I know, there's no equivalent of AutoIncrement in CoreData.

    func nextSequence(moc: NSManagedObjectContext) -> Int {
        let quantityExpression = NSExpressionDescription() = "max.sequence"
        quantityExpression.expression = NSExpression(format: "@max.sequence")
        quantityExpression.expressionResultType = .Integer32AttributeType
        let predicate = NSPredicate(format: "form == %@", self.form)
        let request = NSFetchRequest()
        request.entity = NSEntityDescription.entityForName("UMAQuestion", inManagedObjectContext: moc)
    //  request.propertiesToGroupBy = ["sequence"]
        request.resultType = NSFetchRequestResultType.DictionaryResultType
        request.propertiesToFetch = [quantityExpression]
        request.predicate = predicate
        var maxSequence = 0
        do {
            let results = try moc.executeFetchRequest(request) as? [[String:AnyObject]]
            if let maxSequenceDict = results!.first {
                if let maxSequenceResult: AnyObject = maxSequenceDict["max.sequence"] {
                    maxSequence = maxSequenceResult as! Int
                } else {
                    fatalError("Couldn't find max.sequence in dict")
            } else {
                fatalError("Didn't get first result")
        } catch let error as NSError {
            fatalError("Error fetching max sequence: \(error)")
        return maxSequence + 1

Paste this in your model file. Adjust the predicate or uncomment and adjust the propertiesToGroupBy.

2016-05-16 Renewing letsencrypt certificates via crontab

Last time, I showed how I installed the LetsEncrypt certificates but not how to automatically renew them. To do so, create a file and call it, say, "" or something, then move it to a directory of your choice. /opt was empty so I put it there. Adapt the webroot and domain parameters of course. This works on Debian 8.0 so your mileage may vary.

  letsencrypt certonly --force-renewal --webroot -w /var/www/ -d
  service apache2 reload

The --force-renewal is there because we renew monthly, while the certs are valid for three months and the client warns about this.

Now create a file in /etc/cron.d so the script gets called. Be sure to not just call it on midnight the 1st of every month -- we want to call it at some particular randomly chosen moment so the letsencrypt people don't get hit with a deluge of calls every 1st of the month. So create a file named "letsencrypt" in /etc/cron.d with the following line:

  33 5 21 * *     root   /opt/

This particular example will call the script on the 33rd minute, on the 5th hour, and on the 21st day of the month. In other words, it'll get called once on 5:33 AM every month. Make up some values for your particular case.

It's prudent to now put a marker after this date in your calendar, to actually check whether the renewal succeeded.

As an aside, the letsencrypt package is called certbot nowadays. It's at version 7. It might be better to follow their instructions :)

2016-04-28 USB speakers

Since ages, I've had these Logitech V20 speakers (serial number S-0155A) which get both their power and their audio over USB:

There are a couple of awesome things about these:

  • If you have a laptop and want something close to a docking station, then just get a USB hub, connect these speakers along with your other peripherals and save yourself the connecting/disconnecting of the 3.5 mm audio jack each morning.
  • If you have a device without a 3.5 mm audio jack, but does have USB, this is a great way to get sound. For example most Synology disk stations support these speakers.
  • If you have a PC with a terribly noisy audio jack, these speakers give you a great way to get decent quality audio without the interference.
  • They're supported on every OS that I know: Linux, OS X, Windows.

Until recently, people would ask me where to get these. But unfortunately they long since went out of stock everywhere and you'd have to rely on the second-hand market. But lately I found out that Logitech makes a new model that strictly uses USB for both audio and power: the Logitech S150.

Edit 2016-11-28: a colleague pointed me to other speakers that are hooked up via USB only:

Edit 2016-12-07: I've tested the Logitech S150 and they're not comparable in sound quality to the old v20 speakers. I'd only advise these as a step up from laptop speakers.

2016-04-24 Swift solution to StoneWall

This is a 100% scoring solution to the Codility StoneWall problem. Note the bunch of print statements which I've left included for now.

  import UIKit
  public func solution1(inout H : [Int]) -> Int {
    var stack: [Int] = []
    var nBlocks = 0
    var i = 0
    var lastHeight = 0
    func append(height: Int) {
      print("append \(height)")
      lastHeight = height
      nBlocks += 1
    while i < H.count {
      let currHeight = H[i]
      print("i=\(i), currHeight=\(currHeight)")
      if i == 0 {
      } else {
        if currHeight == lastHeight {
          print("currHeight == lastHeight")
        } else if currHeight > lastHeight {
        } else { // currHeight < lastHeight
          // Keep popping
          var found = false
          while !stack.isEmpty {
            let lastAppendedHeight = stack.last!
            if lastAppendedHeight == currHeight {
              print("lastAppendedHeight == currHeight == \(currHeight)")
              lastHeight = currHeight
              found = true
            } else if lastAppendedHeight < currHeight {
          if !found {
      i += 1
    return nBlocks
  var H = [8,8,5,7,9,8,7,4,8]
  H = [1,2,3,4,5]
  H = [1,1,1,1,1,1,1,1,1]
  H = [1,2,3,3,3,2,2,2,2,2,1]
  H = [5,4,3,2,1]
  H = [1,2,1,2,1,2]

2016-04-16 Codility supports Swift

If you didn't know Codility: it's a company that devises programming tests that are used to test applicants for a job opening. Besides that, other uses are simply to exercise oneself, gaining skills in solving programming puzzles.

And now it supports Swift! I've been having fun doing some small puzzles here and there. For example, this is my solution to the Number of Intersecting Disks problem.

And for Lesson 7, the Brackets test, here is the solution I came up with:

    //: Playground - noun: a place where people can play
    import UIKit
    enum Bracket: Character {
        case squareOpen = "["
        case squareClose = "]"
        case roundOpen = "("
        case roundClose = ")"
        case curlyOpen = "{"
        case curlyClose = "}"
    public func solution(inout S : String) -> Int {
        var bracketStack: Array<Bracket> = []
        for bracketChar in S.characters {
            guard let bracket = Bracket(rawValue: bracketChar) else {
                return 0
            switch bracket {
            case .squareOpen, .roundOpen, .curlyOpen:
            case .squareClose:
                if let b = bracketStack.popLast() {
                    if b != .squareOpen {
                        return 0
                } else {
                    return 0
            case .roundClose:
                if let b = bracketStack.popLast() {
                    if b != .roundOpen {
                        return 0
                } else {
                    return 0
            case .curlyClose:
                if let b = bracketStack.popLast() {
                    if b != .curlyOpen {
                        return 0
                } else {
                    return 0
        if bracketStack.count > 0 {
            return 0
        return 1
    var validString = "{[()()]}"
    var invalidString = "([)()]"

2016-03-18 Updating Xcode

Erica Sadun about updating Xcode. The Mac App Store prompted her to update Xcode, and that's what she did. Unfortunately, the download is very big and you don't really have control over the download and update process.

I've learned long ago that it's much better to simply download the .dmg file by logging in as a developer, going to the Xcode downloads page, click on the link "Additional tools", and download the .dmg file from there.

I rename the existing installation to, for example /Applications/ so I always can go back. And from the .dmg, copy the new version to /Applications. Then if my current project nicely compiles and deploys, I remove the old Xcode installation.

As an aside, I really hope Apple will give developers a way to download that multi-gigabyte file from the commandline or perhaps in chunks of 500 megs or something. The whole root of the XcodeGhost problem was that developers were downloading Xcode from a fast 3rd party server, because the usual route wasn't feasible.

2016-03-18 Lets Encrypt

Another day, another TLS certificate from the friendly folks at Let's Encrypt! I'm running Debian 8.2 (Jessie) so I took the following steps:

Add the following lines to /etc/apt/sources.list

 # Backports repository
 deb jessie-backports main contrib non-free

Update the lot, then install the client:

 $ sudo apt-get update
 $ sudo apt-get install -t jessie-backports letsencrypt

I'm running Apache and had SSL configured already. So I ran the client:

 $ sudo letsencrypt certonly --webroot -w /var/www/vankuik/www/ -d

Then updated the domain configuration to the following lines:

 SSLCertificateFile    /etc/letsencrypt/live/
 SSLCertificateKeyFile /etc/letsencrypt/live/
 SSLCertificateChainFile /etc/letsencrypt/live/

Restart Apache:

 $ sudo service apache2 restart

And then reload in your browser. Et voilà, the results are in:


See also 2016-05-16 Renewing letsencrypt certificates via crontab.


Just a tip for all you Linux-loving fiends out there: after installing a server, no matter how small, you'd be wise to install sshguard. It's available on Debian, Ubuntu and all other Debian-derivates.

After installation, simply type:

 $ sudo apt-get install sshguard

This package will block bruteforce attacks and the like.

2016-02-19 Swift example of regex

Here's a quick-and-dirty copy-paste example of a regex in Swift 2.0. It pulls the width and height as integers from the string in the first line.

  var str = "codecType: 'avc1' dimensions: 640 x 360} extensions: {<CFBasicHash 0x128511ae0 [0x19f53cb68]>{type = immutable dict, count = 15"
  let regex = try! NSRegularExpression(pattern: "[0-9]{2,4} x [0-9]{2,4}", options: [])
  if let result = regex.firstMatchInString(str, options: [], range:  NSMakeRange(0, str.characters.count)) {
      let dimensionString = (str as NSString).substringWithRange(result.range)
      let dimensionArray = dimensionString.componentsSeparatedByString(" x ")
      let width = Int(dimensionArray[0])
      let height = Int(dimensionArray[1])
      // Now do something with the width and height integers

2016-02-18 Swift error Instance member cannot be used on type

Today, Swift gave me the error:

  Instance member "membername" cannot be used on type "Classname"

The code was as follows:

  public class SomeClass {
      public let defaultGreeting = "Hello, playground"
      public var greeting = defaultGreeting

The solution is as follows:

  public class SomeClass {
      public let defaultGreeting = "Hello, playground"
      public var greeting: String!
      init() {
          self.greeting = defaultGreeting

The reason you're getting the error, is because the class member defaultGreeting is declared but not yet initialized at the class scope. However, when you hit init() then it's available.

2016-02-14 Cleaning the Microsoft Sculpt Ergonomic

I've always been a fan of the Microsoft ergonomic keyboards. They offer the best keyboards in the "ergonomic" category for their price. The Sculpt Ergonomic got a great review from Marco Arment and I totally agree with Mr. Arment on all points.

Recently however, I spilled some tea-with-milk on the keyboard. It was really just a sip, and it hit the right Alt key. But somehow gravity took that liquid and ran it down to the arrow keys and the backspace button. I opened the keyboard but unfortunately it's a scissor-switch keyboard where the rubber keypad cannot be reached. It's backed by an metal underside that is melted to the top case of the keyboard. See the picture below; the black dots are studs whose tops have been melted flat.

MS Sculpt Ergonomic 1 small.JPG

Larger version: MS Sculpt Ergonomic 1.JPG

However I did succeed in getting it working again!

Prerequisites: I bought a 1 liter (33 oz) bottle of demineralized water and a 125 ml (4 oz) bottle of alcohol.

Then take the following steps:

  • Turn around the keyboard.
  • Remove the batteries.
  • Remove the rubber feet and unscrew the five phillips screws below. Note the position of the three longer screws.
  • Turn the keyboard back up and remove the wrist pad. Besides three screws, it's also held in place with a couple of clips. Gently apply force.
  • There's about a dozen screws now accessible below the wrist pad. Unscrew these too.
  • The keyboard top and bottom case are now held together with a dozen or so clips. You can take it apart with a broad screwdriver and some gentle force in the correct direction. Be careful however: the bottom case contains the battery holder, and it's connected to the top case.
  • Gently remove the battery connector from the 5 x 5 cm printed circuit board (PCB) below the top case.
  • Now remove the flat cable from the PCB by pulling back the black lock thus loosening the flat cable.
  • Unscrew the PCB and put it aside. Since these screws have a different length, it might be best to screw them back in after removing the PCB.

MS Sculpt Ergonomic 3 small.JPG

Larger version: MS Sculpt Ergonomic 3.JPG

  • Now remove the key caps from the keys that you spilled on. Use a broad screwdriver, and put the head in the space between the keyboard and the top of each key cap. The reason is that the scissor switch is fragile, and the key caps are clicked to the switch at the top, but slide-in at the bottom side, see details. But note: in this particular picture, the key cap is rotated 90 degrees.

MS Sculpt Ergonomic 4 small.JPG

Larger version: MS Sculpt Ergonomic 4.JPG

  • Put the top case with the metal side down in the kitchen sink.
  • Heat the water to, say, 50 degrees Celsius or something, and then pour it into the key caps you removed. Make sure you avoid dousing the flat cable, simply because it's sensitive to heat.
  • Now rinse with the alcohol.
  • Let it dry for a couple of days, and put it back together again.

2016-01-17 The new Mac Pro

As an app developer, I've always been following the new Mac Pro (usually shortened to nMP, as opposed to the cMP, where c stands for classic). For portability reasons, clients usually give me a retina MacBook Pro to work on, but the Mac Pro is a desktop computer. Thus it offers all that desktops usually offer: more power.

If you're a desktop person then for some time, the Mac Pro hasn't been the first choice; usually the 5K iMacs would be a better choice. App developer Marco Arment explains why: The Retina iMac versus the Mac Pro.

And then there's the fact that the nMP couldn't really be called "new" at this point. Both the CPU and the GPU are pretty old tech. And its SSD is easily beaten by the SSD in the 2015 rMBP. It hasn't been updated since its release end of 2013 and the MacRumors Buyer's Guide maintains a disheartening running tally since its last update.

For 2016 however, the stars might align and Apple may give some love to the nMP. Over at the MacRumors forums, enthusiasts have been speculating and there has been long-running thread with the subject Will there be a new Mac Pro in 2016. User Stacc gave an excellent summary of that thread which explains what we can expect of the new Mac Pro.

2016-01-09 Compress PNG files with pngquant

A little trick if you want to save some space and/or bandwidth; compress your PNG files. Although PNG is a lossless format, you can compress them if the image in question doesn't have a lot of colors. This is often the case when creating screenshots.

In yesterday's screenshot of my terminal window, there's obviously not a whole lot of color. OS X by default saves the image with 24-bits colors, but the image looks perfectly fine with just 8 bits. After compression, file sizes looked as follows:

 -rw-r--r--@ 1 bartvk  staff    88K Jan  8 12:11 Image_2016-01-08_dd_progress.png
 -rw-r--r--  1 bartvk  staff    17K Jan  8 12:13 Image_2016-01-08_dd_progress-fs8.png

That's not so bad. The pngquant homepage is here and on OS X, it's easily installed with a quick

 $ brew install pngquant

2016-01-08 BSD dd progress

The commandline utilities that come installed with OS X originate from the BSD world. This means that option-wise, they're fairly spartan as opposed to GNU userland.

Today, I needed to create a bootable USB stick to install Ubuntu. It works fine and basically tells you to use dd to copy a bootable image to a USB stick.

Now if you have a slow USB stick, this can take ages and you're wondering if it's making progress. To see what it's doing, open another terminal and find the process number of dd:

  $ ps -ef | grep "dd if"
  501 76765   412   0 11:29AM ttys000    0:16.19 dd if=/Users/bartvk/Desktop/ubuntu-15.10-desktop-amd64.img of=/dev/disk1 bs=1m
  501 76786 75363   0 11:29AM ttys002    0:00.00 grep "dd if"

The second number of the first line is the process ID, in this case, 76765. Now run the following command:

 $ while true; do sudo kill -SIGINFO 76765; sleep 5; done

This will cause the running dd to print out statistics (roughly) every 5 seconds.

Image 2016-01-08 dd progress.png

2016-01-06 About an old Google Drive document

John Gordon writes about losing images in an old document on Google Drive:

A Google Doc I wrote in 2010 has lost its images

I encountered this issue right when the transition was made from the old editor to the new editor, which might have been around 2010 (not sure about that). Since then, I had gotten weary. Whenever I'd create an important document, I'd create a folder on Google Drive, put the images and other assets in there, then include them in the document. In other words, they would never be just in that document.

With Google Drive, I first thought that all files (Goole Docs, Sheets, etc.) would be synced locally, and that my local backup solution (Time Machine) would save a copy. But to make sure that happened, I tried to figure it out and create a matrix of what was viewable and editable offline: 2013-09-28 Criticism of Google Drive. In the end, I figured it was very hard to get a reliable backup of Google Docs, Sheets etc.

Back then, I hated the Microsoft Office format lock-in. Google seemed somewhat more open, and it was. But only in the sense that they had an export to the Open Office format. Their own format is very much closed, and has remained so for all this time. With Microsoft Office at least your local backup solution works fine, and the format is understood to a certain degree and imported by a number of software packages, some of them open source.

2015-12-11 Measuring startup time

The startup time of your app is determined by a bunch of things, but one that you can easily and directly influence, is the time spent in your AppDelegate's function didFinishLaunchingWithOptions. You may have put a number of things in there, and if you want to find out which step is taking up all that time, it's not always useful to fire up Instruments. A simple way is to just dump a bunch of logging statements.

Paste the following code at the top of the AppDelegate.swift file.

	var logTimer: CFTimeInterval = -1.0
	var logStep = 1
	func logAppProgressInit() {
		logTimer = CACurrentMediaTime()
	func logAppProgress() {
		let stepTime = CACurrentMediaTime() - logTimer
		NSLog("Step %d took %.3f seconds", logStep, stepTime)
		logTimer = CACurrentMediaTime()

Then paste the following line at the top of your didFinishLaunchingWithOptions:


And the following line after each (functionally meaningful) line of code in didFinishLaunchingWithOptions:


This code is just for a quick round of debugging, and must be removed afterwards. NSLog itself is slow and wasteful in this phase.

2015-12-08 gem install fastlane taking a long time

If you wanted to install fastlane and you issued the following command:

 $ sudo gem install fastlane

... you might have interrupted the command because it seemed to hang. In fact, it is working but may take an awfully long time due to the dependencies. Install with the verbose flag and see what's going on:

 $ sudo gem install -V fastlane
 302 Moved Temporarily
 200 OK
 Installing ri documentation for scan-0.3.2
 Parsing documentation for fastlane-1.46.1
 Parsing sources...
 100% [161/161]  lib/fastlane/version.rb
 Installing ri documentation for fastlane-1.46.1
 62 gems installed

2015-11-26 Delay initialization of constant variable in Swift

Recent versions of Swift allow making a variable constant (with the let keyword), without directly initializing it.

This makes the following code possible:

	let identifier: String
	if flight == self.nextFlight {
		identifier = RosterListActivityCellReuseIdentifier.NextFlight.rawValue
	} else {
		identifier = RosterListActivityCellReuseIdentifier.Flight.rawValue


2015-11-20 Xcode tip

A nice tip for when you're pair programming and you're both sitting somewhat farther from the monitor than usual: big fonts in Xcode.

Go to Xcode -> Settings, then select the Fonts & Colors tab. There's a preset style available called "Presentation".

xcode presentation style.png

2015-11-06 Creating an OS X virtual machine

Automatically creating an OS X virtual machine is getting quite easy and automated nowadays.

If you haven't installed Homebrew, install it according to the command shown at

Additionally, install Homebrew Cask:

  $ brew install caskroom/cask/brew-cask

Then install packer. If you haven't used packer before, it's a tool for automating the creation of Virtual Machine images.

  $ brew install packer

And via Cask, install vagrant and VirtualBox. Vagrant is a tool for managing development environments using Virtual Machines, i.e. starting, stopping et cetera.

  $ brew cask install vagrant
  $ brew cask install virtualbox

Now continue with rmoriz' instructions.

Check the results:

  $ vagrant box list
  macosx-10.10 (virtualbox, 0)

Initialize vagrant and start the VM:

  $ vagrant init macosx-10.10
  A `Vagrantfile` has been placed in this directory. You are now
  ready to `vagrant up` your first virtual environment! Please read
  the comments in the Vagrantfile as well as documentation on 
  `` for more information on using Vagrant.

Now open the file Vagrantfile with an editor and search for the line: = "macosx-10.10"

Below, add the following lines:

  config.vm.provider "virtualbox" do |vb|
    config.vm.synced_folder ".", "/vagrant", type: "rsync"

Then start the VM:

 $ vagrant up
 Bringing machine 'default' up with 'virtualbox' provider...

See if it's really running:

  $ vagrant ssh
  Last login: Fri Nov  6 04:17:05 2015
  osx-10_11:~ vagrant$ uname -a
  Darwin 15.0.0 Darwin Kernel Version 15.0.0: Sat Sep 19 15:53:46 PDT 2015; root:xnu-3247.10.11~1/RELEASE_X86_64 x86_64

Alternatively, start VirtualBox, select the VM on the left side and click the Show button.


OSX Virtual Machine.png

Congratulations! Have a drink 🍻

2015-10-30 Searching Cocoapods

Today, we were joking around in the team, and we figured it would be cool if you could simply include a Cocoapod in your iOS project to add an easter egg. All kidding aside, there's actually a nice command-line utility which allows you to search Cocoapods:

No results for "easter egg", though :) But you might find one via cocoapods-roulette :)

2015-10-08 start of a day with NSDate

There's a nice new function since iOS 9, startOfDayForDate. Some examples:

To get the NSDate for this morning:

 Calendar.current.startOfDay(for: Date())

And to get the end of the previous day:

 let thisMorning = Calendar.current.startOfDay(for: Date())
 let yesterdayEvening = .second, value: -1, to: thisMorning)

And since iOS 8, there's isDateInToday. For example:

 if Calendar.current.isDateInToday(flight.departureDate) {

2015-10-06 Testing NTP server under OS X

When you configure a different time server (NTP server) under OS X, the System Settings panel doesn't actually tell you that the address is valid or not. To check this, open a terminal window and use the ntpq command as follows:

 $ ntpq -p

The time server should list its sources and the command should display something roughly as follows:

      remote           refid      st t when poll reach   delay   offset  jitter
 ==============================================================================    2 u 1030 1024  377    1.904    4.659   0.150    2 u  924 1024  377    1.805    4.721   0.100
 *ntp1qvi.xxxxxxx .GPS.            1 u  921 1024  377   24.874   -1.048   0.330
 +ntp1tls.xxxxxxx .GPS.            1 u  705 1024  377   28.317   -1.072   0.211

If there isn't actually a time server running, then ntpq will report:

 nodename nor servname provided, or not known

2015-09-15 Xcode eating up diskspace in Library folder

Edit 2015-09-25: Don't remove simulators as suggested below. It may stop Xcode from running Playgrounds.

Xcode can eat up quite an amount of disk space. An easy fix may be to remove projects from the Projects window as well as remove Simulators. This only deletes data that can be recreated but as usual you're expected to keep a backup, like any professional does. Except Linus Torvalds, who is excused.

Before you do this, see the amount of disk space that's currently occupied by the Library folder with this Terminal command:

  $ du -sm Library/
  27966	Library/

Take the following steps:

  • Go to the Xcode menu bar, pick Window -> Projects. Remove old projects there.
  • Go to Window -> Devices. Remove simulators of old versions, or simulators which you're currently hardly using (for example because you're working on an iPhone-only app right now)

Now check the disk space again:

  $ du -sm Library/
  25227	Library/

Nice, more than 2.5 gigs cleaned up :)

2015-09-04 View all non-standard kernel extensions on OS X

If you want to list all the drivers (kernel extensions in OS X jargon) that didn't come pre-installed with OS X, open a terminal window and type the following command:

 $ kextstat| grep -v

You'll always get the following line, which are the column headers:

   Index Refs Address            Size       Wired      Name (Version) <Linked Against>

That means there are only Apple-provided drivers present on your system. As an example, I've got the Logitech OS X driver installed, plus I've got VirtualBox and two keyboard remapper tools installed:

 $ kextstat| grep -v
  Index Refs Address            Size       Wired      Name (Version) <Linked Against>
   70    0 0xffffff7f80df6000 0x46000    0x46000    com.Logitech.Control Center.HID Driver (3.9.1) <69 67 37 31 5 4 3>
  130    0 0xffffff7f82a9d000 0x28000    0x28000    org.pqrs.driver.Karabiner (10.8.0) <31 5 4 3 1>
  131    0 0xffffff7f82ac5000 0x6000     0x6000     org.pqrs.driver.Seil (11.3.0) <31 5 4 3 1>
  132    3 0xffffff7f82acb000 0x58000    0x58000    org.virtualbox.kext.VBoxDrv (4.3.24) <7 5 4 3 1>
  133    0 0xffffff7f82b23000 0x8000     0x8000     org.virtualbox.kext.VBoxUSB (4.3.24) <132 95 37 7 5 4 3 1>
  134    0 0xffffff7f82b2b000 0x5000     0x5000     org.virtualbox.kext.VBoxNetFlt (4.3.24) <132 7 5 4 3 1>
  135    0 0xffffff7f82b30000 0x6000     0x6000     org.virtualbox.kext.VBoxNetAdp (4.3.24) <132 5 4 1>

As an aside, if you want to remap some keys on your keyboard, be sure to check out the donation-supported utilities Karabiner and Seil.

2015-08-31 Centered popover on the iPad

Below is a nice centered popover on the iPad:


You can do this by configuring the segue in Interface Builder by:

  • (obviously) configuring as "Present As Popover"
  • Uncheck all directions
  • Set the anchor to the main view of the view controller

This is in Xcode 6.4:

xcode popover.png

It also nicely shifts up when the keyboard appears:


2015-08-27 Swift Switch must be exhaustive

In Swift, the switch/case statement must contain a default case, otherwise you'll get the following compiler error:

    Switch must be exhaustive, consider adding a default clause

We used to put a non-sensical log statement in there, but today, a colleague found out that you can use the break keyword.


	switch (self.type!.integerValue) {
	case CustomerTypeNormal.integerValue:
		name = "\(self.lastname?.lowercaseString.capitalizedString), \(self.firstname?.lowercaseString.capitalizedString)"
	case CustomerTypeCrew.integerValue:
		name =  "Crew"
	case CustomerTypeAnonymous.integerValue:
		name = "Anonymous"

2015-08-27 Open underlying SQLite database when using Core Data

During development of an app using Core Data, you often want to have a quick peep in the underlying SQLite store. Paste the following bit somewhere in your AppDelegate:

	// Log path (to manually open SQLite database) when in simulator. In the File -> Open dialog of
	// SQLPro for SQLite, type Shift-CMD-G and paste this path
		let path = NSSearchPathForDirectoriesInDomains(.DocumentDirectory, .UserDomainMask, true).last! as! String
		println("Documents folder: \(path)")

You can use the commandline sqlite client, but I like SQLPro for SQLite. Start your app, copy the string from the console in Xcode, then go to SQLPro and click the Open button. In the dialogue, type Shift-Cmd-G and you can paste the path of the Document folder of your currently running app. You can then open the SQLite database and peek inside.

2015-08-21 lastPathComponent is unavailable

You may get something like the following error when you open your Swift code in Xcode 7 beta:

  'lastPathComponent' is unavailable: Use lastPathComponent on NSString instead.

Solution is to do something like this:

  let lastPathComponent = (fileName as NSString).lastPathComponent

2015-08-07 Swift Reflect example

Halfway last month, Erica Sadun wrote a blog entry where she used the reflect() function: Swift: Just because

She just gave the code and didn't comment any further so below, I've liberally sprinkled the code with comments:

	// Define a struct to represent a point
	struct Pt {
		let x: Int
		let y: Int
		init(_ x: Int, _ y: Int) {self.x = x; self.y = y}
		init(){self.init(0, 0)}
	// Declare an instance of Pt
	let p = Pt(2, 3)
	// Any means: any reference (i.e. class) or value type (i.e. struct, number, etc)
	// So this function will take an item that can be basically anything in Swift
	// and look through the names of its members to see if the string matches.
	func valueForKey(item : Any, key: String) -> Any? {
		// The built-in Swift reflect() function is used by Xcode to aid in debugging
		let mirror = reflect(item)
		// It returns an array, walk through it
		for index in 0..<mirror.count {
			// Each item in the array has a member called 0 and 1
			// This assignment is only there for readability's sake
			let child = mirror[index]
			// The .0 member is a string and it holds the name of our struct member
			if key == child.0 {
				// The .1 member holds information about our struct member
				// If it's not an optional, return its value with the .value property
				return child.1.value
		return nil
	// Try it out
	valueForKey(p, "x")

Instead of using the valueForKey() function, you can also open a Playground and try out a few things yourself:

	let mirror = reflect(p)

swift reflect.png

2015-08-07 Core Data sum example

I had some trouble last week finding a Swift example to sum across order line entities in Core Data. In the app I'm working on, there's the usual Order and OrderLine entities, and they're exactly what you think they are: clients can use the app to shop, and when they put stuff in their shopping cart, we slowly build an order with order lines.

Note 1: in the code below, order lines are called order items because "legacy".

Note 2: the predicate in the fifth line passes self as the order. That works in our case, because the function is located in the Order class. Adjust as necessary.

Without further ado:

	func orderItemsQuantity(managedObjectContext: NSManagedObjectContext) -> Int {
		let quantityExpression = NSExpressionDescription() = "totalQuantity"
		quantityExpression.expression = NSExpression(format: "@sum.quantity")
		quantityExpression.expressionResultType = .Integer32AttributeType
		let predicate = NSPredicate(format: "order == %@", self)
		let request = NSFetchRequest()
		request.entity = NSEntityDescription.entityForName("OrderItem", inManagedObjectContext: managedObjectContext)
		request.propertiesToGroupBy = ["order"]
		request.resultType = NSFetchRequestResultType.DictionaryResultType
		request.propertiesToFetch = [quantityExpression]
		request.predicate = predicate
		var results:[[String:AnyObject]]?
		var error: NSError? = nil
		var totalQuantity = 0
		if let results = managedObjectContext.executeFetchRequest(request, error: &error) as? [[String:AnyObject]] {
			if error != nil {
				fatalError("Unresolved error \(error), \(error!.userInfo)")
			if let totalQuantityDict = results.first {
				if let totalQuantityResult: AnyObject = totalQuantityDict["totalQuantity"] {
					totalQuantity = totalQuantityResult as! Int
		return totalQuantity

2015-07-09 Short list of VPS providers

Here's my 2015 short list of VPS providers:

  • TransIP Reasonable price, good CPU performance, but I've had storage reliability problems in 2013 (for which they compensated me)
  • DigitalOcean Good price, don't like their CPU performance -- starting a Grails instance took 90 seconds, as opposed to 60 seconds on a TransIP node in the AMS2 data center
  • Linode No experience with them, but good reviews from people I trust
  • Scaleway No experience with them; ARM cores
  • Vultr No experience with them; not a lot of disk space for my price range

2015-07-05 withUnsafeBufferPointer

Update 2016-10-06; a very nice (and updated for Swift 3) blog about this:

I couldn't find a nice example for Array<T>.withUnsafeBufferPointer, so here's one which you can paste right into a Playground:

	//: Playground - noun: a place where people can play
	import UIKit
	var buf = [UInt8](count: 10, repeatedValue: 0)
	// Fill buffer with A to J
	for (var i = 0; i < buf.count; i++) {
		buf[i] = 65 + UInt8(i) // 65 = A
	// Calculate pointer to print contents
	buf.withUnsafeBufferPointer { (pbuf: UnsafeBufferPointer<UInt8>) -> UnsafePointer<UInt8> in
		for (var j = 0; j < pbuf.count; j++) {
			let p = pbuf.baseAddress
			println(String(format:" = 0x%X", (p+j).memory))
		return nil


	A = 0x41
	B = 0x42
	C = 0x43
	D = 0x44
	E = 0x45
	F = 0x46
	G = 0x47
	H = 0x48
	I = 0x49
	J = 0x4A

2015-06-21 Swift 2.0 error handling

Here is a nice article by Erica Sadun on the error handling in Swift 2.0.

I really like how a try statement can be forced, i.e. try! will tell the compiler you ignore the error and this will cause a crash when the exception trips during runtime. From my Java experience, I remember that there were plenty of exceptions that basically couldn't be recovered from; you'd log them and then exit. I'm glad Apple recognized this.

The keyword defer is also interesting, delaying execution until the end of the scope. Notably, multiple statements after each other have a sequence, and these statements are executed in reverse order. On the Debug podcast, Don Melton (former Safari product manager) commented that he thought it was taken from the D programming langauge. A buddy of mine had been developing in D in the past, so I asked him what he thought about it. He remarked that he hadn't actually ever used that particular keyword in D... I wonder if I'll find much use of it.

2015-06-19 QuickLook for mobileprovision files

Craig Hockenberry wrote a Quick Look plug-in for .mobileprovision files (i.e. Provisioning Profiles). Hugely useful, because you can just select a provisioning profile and press space in Finder to see which UDIDs (devices) are included.

To debug provisioning profiles, don't let Xcode manage them. Instead when you need to run your app on a new device, take the following steps:

  • In the Member Center, go to Certificates, Identifiers & Profiles
  • Add the device for this particular App ID
  • Generate either a development or a distribution profile and download it manually
  • Use the above Quick Look plugin to verify that the downloaded profile does indeed contain the UDID of the new device
  • Double-click the new .mobileprovisioning file
  • Check with the terminal that it is now located in ~/Library/MobileDevice/Provisioning Profiles

Now go to Xcode. Since version 6, you can view provisioning profiles by going to menu Xcode -> Preferences. In the Accounts tab, select your Apple ID on the left, select your team on the right and click View Details. The new profile should be there, or else you can click the refresh button. The expiration column should show the date as exactly one year later.

If you want the .ipa file, check out this answer on StackOverflow.

2015-06-16 AutoLayout in WWDC 2015

Be sure to check out these videos on AutoLayout:

There's some stuff in there that you may already know, but they also explain new stuff. My highlights:

  • First use stackviews, then for the rest, use constraints
  • Don't add and remove constraints, activate and deactivate them
  • For text, you might want to use firstBaseline and lastBaseline
  • Use "alignment rects" if you need to ignore extra stuff in your views (like shadows)

And in part 2:

  • Use the .identifier property on views when debugging, it's a string that'll show up in the log
  • Use method in the debugger called constraintsForAxis or something
  • There's a trace method in the debugger: po [self.view _autolayoutTrace] (can show ambiguous layouts); then on the LLDB prompt: po [object exerciseAmbiguityLayout]
  • While your app is running, check the menu Debug, option View Debugging

2015-06-12 Shortcut to lock OS X

There isn't any standard function to have a shortcut key to lock your screen in OS X. The best way to do this without involving the screensaver, is the second method that's mentioned in this link: How to Quickly Lock Your Screen in Mac OS X with Keyboard Shortcut

Do take special care: the command being called won't work if you just copy/paste it from that page. Use the following line instead:

  /System/Library/CoreServices/”Menu Extras”/ -suspend

Their CMS apparently replaces the standard minus-sign with a special en-dash. The above line is correct.

2015-06-06 Dial down the transmit power of your Airport Express

In my neighborhood, there are about 20 WiFi networks visible when I click the WiFi symbol on my MacBook's top menu. Needless to say, WiFi is wonky and unstable. The usual advice of a great WiFi deployment is: multiple access points (APs) with the same SSID, where you dial down the transmit power so you cover only the exact area for that particular access point.

It's thus a damn shame that the latest version of the AirPort Utility (version 6) does not allow you to influence the transmit power. The older versions did provide an option to do so, but it doesn't run anymore on the latest versions of OS X.

Corey J. Mahler automated a workaround, which is described and can be downloaded on his website. If you do, I encourage you to send a small donation to him. Scroll down on that page and click the green dollar sign. He's been keeping this solution in the air for multiple years now.

2015-05-09 discovering bitrot with MD5

Except for modern filesystems such as ZFS and btrfs, your files aren't protected against so-called "bitrot". If, for some reason, a file is corrupted then you won't discover this and your normal backups might not have the correct file anymore. This is because the corrupted file has been backed up and your archives don't go back in time far enough.

There are a number of ways to protect against this:

  • Use a modern file system, which may not be practical at this moment.
  • Calculate hashes for your archived files and check to see if they haven't been changed somehow, with md5deep.
  • Create parity files that can not only check for damaged files but can also repair them: Parchive.

I went with the second option. md5deep is easily installable via Homebrew:

 $ brew install md5deep

To generate a file with hashes:

 $ md5deep -r -l testdirectory > testdirectory.md5deep

Explanation: -r means recursive, -l will use relative paths. This will create a file called "testdirectory.md5deep", where all files are written with their path and their hash.

To print all changed (damaged) files:

 $ md5deep -r -l -x testdirectory.md5deep testdirectory

Regularly, check the hashes against a directory that shouldn't ever change. For example, your archive of last year's family pictures. If one of the pictures got corrupted, then you know you should restore it from backup.

If you don't mind using a bit of extra space (as well as taking a bit of additional CPU time), then you can use par2. It installs nicely via Homebrew as well:

 $ brew install par2

Example command inside the directory with the files of your choice:

 $ cd testdirectory
 $ par2create par2file *

To verify:

 $ cd testdirectory
 $ par2verify par2file.par2

As an indication, a directory containing 1836 megabytes of photos and videos resulted in a couple of par2 files that took 93 megs, so about 5% of extra storage is necessary.

To go through all subdirectories and create a par2 file, I use the following one-liner on MacOS:

 $ START=$(pwd);IFS=$'\n'; for i in $(find . -type d); do if [ "$i" == "." ]; then continue; fi ;cd "${i##./}"; pwd; par2create par2file *.*; cd "$START"; done

The *.* after par2create is there to prevent subfolders being included in the parameters to par2create. I.e. with *.* we are not globbing subfolders.

2015-05-09 DenyHosts no longer available on Debian 8.0 Jessie

When I was configuring a new Debian 8.0 ("Jessie") server, I noticed the very useful DenyHosts package is no longer available in the package repository. The package "sshguard" however, is available and according to my testing, works fine.

These packages block an IP address after a number of failed login attempts. This is very useful to counter brute-force attacks on your SSH server.

2015-05-05 Stop the new Photos app from opening automatically

Since the last Yosemite update, the new Photos app will open automatically when you hook up your iPhone or iPad.

To stop this from happening, open Image Capture (available in your Applications folder), connect your iOS device, select it in the upper left, and in the lower right corner, click the arrow up. Then change the dropdown to not do anything.

Stop Photos from opening.gif

2015-05-03 HealthKit error when uploading to the App Store

Currently I'm getting an error when uploading a new binary to the App Store:

Apps that use the entitlements [] must have a privacy policy URL for [English, Dutch]. If your app doesn’t use these entitlements, remove them from your app and upload a new binary.

I've been playing around with HealthKit, but to my knowledge I removed all traces from the project in Xcode. When I have a solution, I'll post it here.

Edit: StackOverflow to the rescue

2015-04-30 edit in vi from Xcode

Purely looking at the editing part, I find vi much more powerful than Xcode. If you want to do a quick edit in vi, then simply drag the file from the project navigator to an open terminal, and edit away!

quick vi from Xcode.gif

2015-03-28 SSL security

An excellent article on SSL security, which tells you how to disable the RC4 cipher:

2015-03-13 Connecting to Oodrive via FTPS on OS X

For a client, I have to save files to Oodrive. They offer several methods for uploading, one of them being FTPS, i.e. FTP-over-SSL. It turns out it's quite a hassle when you use OS X.

Taking the following steps allowed me to connect.

First install Homebrew as usual, if you haven't yet installed it. Then edit the lftp recipe:

  $ brew edit lftp

Comment out the line that says "--with-openssl" and add a new line saying


Then install this recipe as follows:

  $ brew install --build-from-source lftp

See if you were successful:

  $ lftp -v
  Libraries used: Readline 6.3, Expat 2.0.1, GnuTLS 3.3.13, libiconv 1.11, zlib 1.2.5

Note the final line, where it should say "GnuTLS".

Now add the appropriate settings; create the .lftprc in your home directory, and paste the following lines (courtesy of the Reliable Penguin blog):

  set ftps:initial-prot ""
  set ftp:ssl-force true
  set ftp:ssl-protect-data true

Then, create the .lftp directory and create a file named "bookmarks" with the
following line:

  oodrive ftps://

To test the result, start lftp on the commandline and type "open oodrive",
then type "ls".

If you see the error "Fatal error: Certificate verification: Not trusted", you may want to add the following line to the .lftp settings file:

  set ssl:verify-certificate no

2015-02-06 Swift enum raw values as an array

Here's a quick example of using Swift's map() function. Here it's used to put the raw values of an enum into an array. Paste this into a playground to see the result.

	import Cocoa
	enum ContentType : Int {
		case Unknown = 0
		case PAS = 1
		case PASGENERIC = 2
		case PASROUTE = 3
		case PASBUNDLE = 4
		case TOPIC = 5
		case NEWS = 6
	let contentTypeArray = [ContentType.PAS, ContentType.PASROUTE, ContentType.TOPIC]
	let intArray ={type in type.rawValue})

Even shorter:

	let intArray ={$0.rawValue})

How it looks in a playground:

swift enum raw values as array playground.png

2015-02-05 Swift example of a batch update in Core Data

Here's a Swift example of doing a batch update in Core Data:

	let updateRequest = NSBatchUpdateRequest("some_entity_name")
	let predicateIsRead = NSPredicate(format: "isRead = %@", NSNumber(bool: true))
	updateRequest.predicate = predicateIsRead
	updateRequest.propertiesToUpdate = [
		ContentItemAttributes.isRead.rawValue: NSNumber(bool: false)
	updateRequest.resultType = NSBatchUpdateRequestResultType.UpdatedObjectsCountResultType
	var error: NSError?
	let updateResult = objectContext.executeRequest(updateRequest, error: &error) as NSBatchUpdateResult?
	if let result = updateResult {
		let count = result.result as Int
		println("Updated \(count) rows")
	} else {
		println("Error during batch update: \(error?.localizedDescription)")

Note that you'll need to update the objects in memory. The above code just prints the row count. If you have stuff displayed in the user interface, use this answer on Stack Overflow to adjust the above code.

2015-02-01 Layout of a tableviewcell depending upon device orientation

Last week, a refresh of the design of my client's app called for a layout that changed depending on the device orientation. We're using AutoLayout and didn't want to use any deprecated methods.

Problem was that it called for layout changes within a UITableViewCell. I thought of a way to do this with AutoLayout but the constraints would become quite complex. Instead, I took the following approach:

  • Create a custom UITableViewCell by subclassing it
  • To this subclass, add outlets for the constraints that you want to influence
  • In the view controller, detect whether we're in portrait or landscape, then update the constraints of the cells using the IBOutlet properties

Thus the UITableViewCell subclass looks as follows:

	class CustomTableCell : UITableViewCell {
		@IBOutlet weak var dateLabel: UILabel!
		@IBOutlet weak var nameLabel: UILabel!
		@IBOutlet weak var trailingConstraint: NSLayoutConstraint!

Problem is: where can you set properties all the currently visible cells? The easiest way I could come up with, is in the cellForRowAtIndexPath() function. The view controller class has the following property:

	var size: CGSize = CGSizeZero

The viewWillTransitionToSize() function stores the new screen size and asks the UITableView to reload:

	override func viewWillTransitionToSize(size: CGSize, withTransitionCoordinator coordinator: UIViewControllerTransitionCoordinator) {
		super.viewWillTransitionToSize(size, withTransitionCoordinator: coordinator)
		self.size = size
		NSLog("viewWillTransitionToSize self.size = \(self.size)")

And when dequeueing new cells, we set the constraint its constant:

	override func tableView(tableView: UITableView, cellForRowAtIndexPath indexPath: NSIndexPath) -> UITableViewCell {
		let cell = tableView.dequeueReusableCellWithIdentifier("TestCell") as CustomTableCell!
		cell.nameLabel.text = self.items[indexPath.row]
		if self.size.width > self.size.height { //Landscape
			cell.trailingConstraint.constant = 100
		} else { //Portrait
			cell.trailingConstraint.constant = 0
		return cell

Because viewWillTransitionToSize() is not called when the View Controller is run for the first time, I also added the following code to viewWillLayoutSubviews():

	override func viewWillLayoutSubviews() {
		// Only run once, upon first display
		if self.size == CGSizeZero {
			self.size = self.tableView!.frame.size
			NSLog("viewWillLayoutSubviews self.size = \(self.size)")

layout uitableviewcell depending upon device orientation.gif

Download the example project:

Note: I created the above truly breathtaking animated gif by taking a screen recording with QuickTime Player, running the iOS Simulator, then converting the resulting .mov file with the following ffmpeg command:

  ffmpeg -i -r 15 example.gif

ffpmeg was installed via Homebrew.

2015-01-29 Update interface when app is opened

If you're developing in Swift and want to update your app's interface when it's reopened, put the following piece of code in the App Delegate:

	var appLastUsedTime = CFAbsoluteTimeGetCurrent()
	func applicationWillEnterForeground(application: UIApplication) {
		let timeSinceLastUsage = CFAbsoluteTimeGetCurrent() - self.appLastUsedTime
		log.verbose("App last used \(timeSinceLastUsage) seconds ago")
		//If it's been at least a day since the user accessed the app, send out a notification
		if timeSinceLastUsage > (24 * 60 * 60) {
			log.verbose("Posting notification \(MYFApplicationDidBecomeActiveAfterTimeoutNotification)")
			NSNotificationCenter.defaultCenter().postNotificationName(MYFApplicationDidBecomeActiveAfterTimeoutNotification, object: nil)
			self.appLastUsedTime = CFAbsoluteTimeGetCurrent()

Above the App Delegate, add the following global:

    let ApplicationDidBecomeActiveAfterTimeoutNotification = "ApplicationDidBecomeActiveAfterTimeoutNotification"

In the view controller, add the following code in viewWillAppear():

	NSNotificationCenter.defaultCenter().addObserver(self, selector: "updateInterface",
			name: ApplicationDidBecomeActiveAfterTimeoutNotification, object: nil)

And the following code in viewDidDisappear()

	NSNotificationCenter.defaultCenter().removeObserver(self, name: ApplicationDidBecomeActiveAfterTimeoutNotification, object: nil)

Note that this notification doesn't trigger when activating the app for the first time. If you need that, add the following code to the view controller in question, in its viewDidAppear() function:

	if !self. didInitializeAtStartup {
		didInitializeAtStartup = true

And the following class member variable:

    	private var didInitializeAtStartup = false

2015-01-20 button title truncated

There's a weird situation that when you add an image to a standard UIButton and add some space between the image and the label, the label text gets truncated:

moreinfo 1.png

The reason is that you used the image edge inset. This edge inset is not used to calculate the intrinsic button size.

For the correct steps, check my answer on StackOverflow.

2015-01-16 RestKit Error

Today I got a weird exception in the Xcode console of my iOS project:

  *** Terminating app due to uncaught exception 'NSInternalInconsistencyException',
 reason: 'Cannot perform object mapping without a source object to map from'

Turns out I had a couple of missing commas in my JSON. In case your editor doesn't validate JSON, have a look at this online JSON editor.

When I fixed that, I got the same error again. This time, I opened the file in vi, which showed that there were a number of control characters (CTRL-P) in the JSON file:

control characters in json.png

When I removed those, the error disappeared.

2014-12-15 Shadows and rounded corners

The app I'm currently working on, requires a textfield with shadow and rounded corners. That doesn't work on a UITextField; you'll have to make it transparent, then add it to a plain UIView and style that.

I got curious and tried to add shadows and rounded corners to a number of elements in UIKit. This is the code:

        import UIKit
	class ViewController: UIViewController {
		@IBOutlet weak var contentView: UIView!
		@IBOutlet weak var button: UIButton!
		@IBOutlet weak var textfield: UITextField!
		@IBOutlet weak var textView: UITextView!
		@IBOutlet weak var datePicker: UIDatePicker!
		@IBOutlet weak var toolbar: UIToolbar!
		@IBOutlet weak var imageView: UIImageView!
		override func viewDidLoad() {
		func addShadowAndCornerRadius(view: UIView) {
			let radius:CGFloat = 20.0
			view.layer.cornerRadius = radius
			view.layer.shadowColor = UIColor.darkGrayColor().CGColor
			view.layer.shadowPath = UIBezierPath(roundedRect: view.bounds,
			    cornerRadius: radius).CGPath
			view.layer.shadowOffset = CGSize(width: 0, height: 0)
			view.layer.shadowOpacity = 1.0
			view.layer.masksToBounds = true
			view.clipsToBounds = false

Below is the result in the simulator. Note that the light-blue empty element is a plain UIView.

shadow and rounded corners uikit.png

As you can see, adding shadow and rounded corners doesn't work on UIImageView, UITextField and UIToolbar. It does work on a plain UIView, UIButton, UITextView and UIDatePicker.

2014-11-18 Swift optionals

Recently I dived into Swift and in order to grok optionals, first read this article on Medium: Swift Optionals and then this comment on Stack Overflow.

2014-11-06 From mov to animated gif

On OS X, the standard QuickTime Player application can record (a part of) your screen, or record the screen of an iOS device over the lightning cable.

You then get a .mov file. You can convert this into an animated gif of slightly smaller size with ffmpeg. You'll need to install homebrew first, and using the brew command, install ffmpeg as follows:

 $ brew install ffmpeg

Then open a terminal and change to the desktop folder or wherever you saved the movie file, and issue the following command:

 $ ffmpeg -i -r 15 demo.gif

Drag the resulting demo.gif to Safari to view it. The quality is pretty bad, but hey, animated gifs are all the rage nowadays.

2014-10-18 Remapping your keyboard in OS X Yosemite

If you want to remap your notebook's or external keyboard, there's an excellent and free choice available: Karabiner from the friendly Takayama Fumihiko.

Upon upgrading OS X yesterday, I found it worked straight away in Yosemite as well.

Besides remapping the obvious keys, if you would like to remap that useless Caps Lock key on OS X, get Seil.


On iOS 8, user permission is required for scheduling local notifications (of class UILocalNotification). Here's an Objective-C example.

To check whether you have permission:

  - (BOOL)appHasPermissionForLocalNotifications
      UIUserNotificationSettings *settings = [[UIApplication sharedApplication] currentUserNotificationSettings];
      if(settings.types & (UIUserNotificationTypeSound | UIUserNotificationTypeAlert | UIUserNotificationTypeBadge)) {
          NSLog(@"Permission present: 0x%ulX", settings.types);
          return YES;
      } else {
          NSLog(@"Permission not present: 0x%ulX", settings.types);
          return NO;

To request permission:

  [[UIApplication sharedApplication] registerUserNotificationSettings:[UIUserNotificationSettings settingsForTypes:(UIUserNotificationTypeSound | UIUserNotificationTypeAlert | UIUserNotificationTypeBadge) categories:nil]];

2014-09-26 Tomcat error on Ubuntu 14.04

If you upgraded Ubuntu to 14.04 and you've got Tomcat running, chances are that you get the following error in catalina.out:

 SEVERE: Failed to initialize end point associated with ProtocolHandler ["http-bio-80"] No such file or directory

Solution is here:
Tomcat7 bind to port 80 fails in ubuntu 14.04lts

2014-09-03 iStat Menus 5

Bjango released iStat Menus 5. I really like this app, but found a minor issue today. I've already sent the folks an e-mail, but for the sake of the internet, I'm documenting it here as well.

iStat Menus can show a graph of your CPU usage along with a nice quick lookup table with the most busy process. It seems it tries to do something helpful with the process name, but in the case of ClamXav, a free and open source virus scanner, it says: ScheduleHelper. The process name in Activity Monitor is more to the point: clamscan.

istat menus process comparison.png


Today, I found out that on Debian Wheezy, when using XFCE on the desktop and accessed via VNC, the Tab key is disabled for some reason.

Solution is here.

2014-07-11 Xauthority doesn't exist

If you're connecting via SSH from OS X to Linux, you might get the following error:

  /opt/X11/bin/xauth:  file /Users/yourusername/.Xauthority does not exist

You can fix this by installing (or just running) XQuartz, available here.

2014-07-10 Unattended upgrades

I'm not normally prone to linking to stuff, but this is just too good:

Explains how you can get your Debian Wheezy box to automatically update. While this is bad practice for some situations like fragile production boxes, there's plenty of spaces where automatic updates work great: single-purpose machines like your (secondary) DNS, lonely virtual private servers with your home page, etc.

2014-05-16 VNC on Debian Wheezy

I recently discovered that VNC on Debian Wheezy is a bit of a mess.

VNC can be used in two modes: to mirror the actively running instance of X (i.e. the display on :0), or to have X running headless, in any resolution ("geometry") you want, on :1, :2, etc.

The version that comes with Wheezy by default, is an old version of TightVNC server. It doesn't support the alpha channel, so modern desktops like KDE 4 will look pretty bad.

Development has stopped on TightVNC, and continues in the TigerVNC project. There are packages available here, but these are pretty old now: version 1.1, while development has passed 1.3.

In and of itself that wouldn't be a problem, except the 1.1 version has a horrible bug where moving a window to the left will result in display artifacts.

You may also have found TurboVNC. Development seems to move slower but they do provide Debian packages. Unfortunately, even for the latest version it seems it suffers from the same bug.

There's a couple of options:

  • Build TightVNC yourself from the source files in the TigerVNC SourceForge downloads section. I tried this, and it's pretty daunting. I gave up after an hour of futzing.
  • Get the generic, statically linked binaries from the TigerVNC SourceForge downloads section, and then just unpack these in your filesystem root. This dumps some binaries here and there, and your sysadmin (or you yourself) will definitely consider this abuse of the sudo command.
  • Use the old TightVNC 1.1 version, which is nicely packaged, and live with the obvious bugs. Note that the above mentioned bug doesn't seem to manifest itself when exporting the currently running display on :0. On our in-house built software however, there's another issue where portions of the screen sometimes don't refresh.

I'm currently looking at other solutions such as X2Go, which seems to provide decent packages.

Edit 2014-08-26: For the last two months or so, I started using the standard VNC again, and I've switched to XFCE as the desktop environment. This solves most problems, except you run into an easily fixable bug: the tab key doesn't work to auto-complete. This link
solves the problem.

2014-05-07 Debian Wheezy interface without an IP address

Today, I needed to configure a machine so it brought up its ethernet interfaces without an IP address.

This is useful if you use such an interface with plain ethernet packets. For example, network sniffing or bridging stuff, but also when you communicate with custom electronics that speak plain ethernet.

To configure Debian Wheezy to bring up these interfaces without assigning an IP address, add the following stanza to /etc/network/interfaces:

 auto eth1
 iface eth1 inet static

You might also want to remove any IPv6 addresses from these interfaces. Stack Overflow told me to add the following line for each interface to /etc/network/interfaces:

 net.ipv6.conf.<interface>.disable_ipv6 = 1

2014-04-16 Python vi keybindings on OS X

If you're used to vi keybindings, and you use Python interactively, you're going to do a lot of cursing on OS X. This is because the readline library (responsible for keybindings and history) is GPL licensed and thus not distributed by Apple.

Here's a quick fix.

Add a file called .editrc with the following content:

 bind -v

Add a file called .pythonrc with the following lines:

 import rlcompleter
 import readline

Add a line to your .bashrc which makes Python run .pythonrc when it starts interactively:

 export PYTHONSTARTUP="$HOME/.pythonrc"

Start python from the commandline and voilà, enjoy history and vi keybindings and all that good stuff. Tested on 10.9.2 (Mountain Lion).

2014-01-29 tmux and X11

Recently I've been playing around with tmux . For all you neckbeards, that's the modern equivalent of GNU Screen.

On the surface, it looks pretty cool. Especially because iTerm2 can talk to tmux. From the iTerm2 wiki: When you run "tmux -CC", a new tmux session is created. An iTerm2 window opens and it acts like a normal iTerm2 window. The difference is that when iTerm2 quits or the ssh session is lost, tmux keeps running. You can return to the host you were ssh'ed into and run "tmux -CC attach" and the iTerm2 windows will reopen in the same state they were in before.

That's pretty useful!

A problem that I'm now bumping into, is that when I'm SSH'ing into the remote machine where tmux runs, I'm forwarding X11. This is useful because vim supports the X11 clipboard. That way, if I copy text from vim and paste it on my local machine, layout is retained.

Without the X11 clipboard, I'd have to simply "hard-copy" whatever is on the terminal screen. Vim's line numbers will be copied along. I also couldn't copy more than one screen.

In order to make this work, when ssh opens a session and starts the shell, it sets the environment variable $DISPLAY. Vim then reads it when it starts.

However when you detach tmux, log out and log in again via ssh, DISPLAY is set again to some other value. All shells in your tmux session now have an old/stale value in DISPLAY. Tmux has some commands for this, but it's not automatic. And if that would work, vim has to be restarted again as well.

It would be nice if you could simply configure the DISPLAY variable but this doesn't work -- the ssh server has an option called X11DisplayOffset but the client doesn't. So there's no way to configure this based on the user.

In summary: X11 forwarding and tmux don't work very well.

2013-12-20 Performance test of USB ethernet adapters on OS X

I've got two USB-based network adapters laying around here: the official Apple USB 2 Ethernet Adapter and this USB 3.0 ethernet adapter based on the Asix AX88179 chipset.

Then I connect over a VPN, and use scp to see how much time it takes to pull over a 100 MB test file. With the Apple adapter:

  $ scp testbox:~/tmp/a.log .
  a.log                    100%  100MB   3.0MB/s   00:33

And now with the USB 3 based adapter:

  $ scp testbox:~/tmp/a.log .
  a.log                    100%  100MB   2.9MB/s   00:34

Huh weird, no difference. I'll go and check where the bottleneck lays. VPN? scp?

Not using VPN gives the following result:

  $ scp .
  a.log                    100%  100MB  16.7MB/s   00:06

That's somewhat better. Testing without scp, but with wget instead shows roughly the same number:

  $ wget
  100%[====================>] 126,805,954 16.0MB/s   in 7.6s

Now I test it on a different machine, within the company network:

 $ wget
 100%[====================>] 126,805,954 64.1M/s   in 1.9s

That's better. So the bottleneck is apparently the physical (ethernet) connection I've got, which is VLAN'ed to provide a guest network. Oh well.

Long story short: better check this guy's test results. You should be able to get up to 110 MB/s.

2013-10-25 What is new in OS X Mavericks

I'm running a little comparison of the file system of a Mavericks install and a Mountain Lion install, to see what's new and what has disappeared.

So far, I found that Mavericks added the following drivers:

Mavericks also added Ruby 2.0 (up from 1.8 in Mountain Lion).

I found a number of apps in /System/Library/CoreServices that are new:

  • Install Command Line Developer Tools -- calling any of the binaries related to development (like /usr/bin/cc) will instead call this Xcode installation utility
  • Keychain Circle Notification -- App that allows you to grant other devices access to this Mac its keychain, used for iCloud syncing of passwords (for example those used in the browser)
  • Pass Viewer -- "partial support for Passbook passes, letting you view emailed passes on your Mac" -- source: MacWorld: Get to know OS X Mavericks: Apple Mail 7.0

There's also a number of new drivers in the /System/Library/Extensions folder:

  • AppleHSSPIHIDDriver, AppleHSSPISupport, AppleHWAccess: would seem to be for communication with keyboards, trackpads, and other buttons (like a hinge-button) that use an SPI bus
  • AppleIntelFramebufferAzul: obviously a display driver, but don't know what exactly
  • Driver for the new Intel HD5000 stuff
  • AppleIntelHSWVA driver, HSW is Haswell, but VA?
  • AppleIntelLpssDmac, AppleIntelLpssGspi, AppleIntelLpssSpiController: LPSS is Intel's Low Power SubSystem stuff, the Linux kernel uses this acronym when talking about Intel's Bay Trail architecture
  • AppleThunderboltIP, that's the driver for tunneling IP over Thunderbolt, duh :D I'm confused if this has anything to do with the Thunderbolt ethernet driver, I don't think so.
  • AppleTopCase, ?
  • GeForce Tesla driver, maybe a driver for the Mac Pro
  • IOReportFamily driver, ?
  • vecLib, see also [2]

In the /System/Library/Frameworks folder, there are a number of new libraries:

  • AVKit.framework
  • GameController.framework
  • MapKit.framework
  • MediaAccessibility.framework
  • MediaLibrary.framework
  • SpriteKit.framework

In /usr/bin, there are a bunch of new executables:

  • afida -- Audio File Image Distortion Analyzer
  • footprint -- gathers memory information about a process or set of processes
  • ipmitool -- utility for controlling IPMI-enabled devices
  • ippfind -- find internet printing protocol printers
  • json_pp -- a Perl-based utility for working with JSON
  • jvisualvm -- ?
  • lsappinfo -- Control and query CoreApplicationServices about the app state on the system
  • ncdestroy -- Destroy kernel NFS credentials
  • pluginkit -- manages the PlugInKit subsystem for the current user. It can query for plugin matches, as well as explicitly change the plugin database.
  • powermetrics -- gathers and display CPU usage statistics
  • timer_analyser.d -- ?
  • timerfires -- analyze timers as they fire
  • wdutil -- Wireless Diagnostics command line utility
  • zipdetails -- display the internal structure of zip files

In /usr/sbin:

  • ioalloccount -- Summarize IOKit memory usage.
  • purge -- force disk cache to be purged (flushed and emptied)
  • sndiskmove -- Xsan File System Disk Mover Utility
  • systemstats -- shows a selection of statistics about system power usage
  • taskpolicy -- execute a program with an altered I/O or scheduling policy. Not sure how this relates to the standard Unix commandline utility "nice"

2013-10-13 Extend drive space in your MacBook Air 2013

Recently I got a MacBook Air, but it wasn't within my budget to get the bigger SSD. The standard SSD is advertised as 128 GB. In practice, this means 112 GiB, so I went looking for additional ways to get some extra storage space.

First and most cheapest, you can get an external harddisk. On one hand, it's very cheap. On the other hand, you have to remember taking it with you, and it's another wire and another device on your desk.

As an extension to that, there's the various wireless external harddisks. This Lifehacker article lists a few: Add Wireless Storage to Phones, Tablets, and Laptops with Wi-Fi Drives.

There's a number of USB-sticks that are so small, you can just leave them in. Here's a nice list of them:

  • Sandisk Cruzer Fit, currently max. 32 G (warning: I've seen a forum post that these are very slow)
  • Kingston DataTraveler Micro, maximum 64 GB
  • Patriot Autobahn currently also 32 GB
  • Verbatim Store 'n' Stay, maximum 32 GB

Of these, I really like the Verbatim Store 'n' Stay because it seems the smallest of the bunch.

There's also a number of solutions that use the SD-card slot. They're basically adapters that take a micro-SD card. They're sized such that they sit flush with the outside of the MacBook.

  • The Nifty Minidrive which looks best and is most expensive, with different options for each MacBook (pro retina, air, pro)
  • The MiniDrive a knock-off with mixed reviews on Amazon
  • BaseQi, another knock-off sold via eBay

Of these three, I am now testing the last one, and it sits perfectly flush with the MacBook Air's chassis:
baseqi detail.gif

I've got it fitted with a 16 GB micro-SD and it seems to work fine. There is no loss of speed noticeable, when comparing with the micro-SD-to-standard-SD-adapter that came with this card.

2013-10-01 Find out Debian release name in one line

What you've always wanted: finding out the Debian release name in one commandline:

 $ lsb_release -a 2>/dev/null | tail -1 | awk  '{print $2}'

Edit: it's way easier than that:

 $ lsb_release  -c -s

2013-09-29 MacBook Air 2013 SD card speed test

Since the SD card slot on the 2013 MacBook Air is connected via the USB 3 bus (instead of USB 2), it shouldn't be limited anymore by USB2 speeds (i.e. theoretical maximum of 480 Mbit/s). The new limit is 5 Gb/s.

So I ran a little test of whatever I've got lying around. I used the Black Magic disk speed test, with a 1 GB test file.

Card Read speed Write speed Remarks
SanDisk 2 GB micro-SD card 17.4 MB/s 3.4 MB/s Came with my Android phone
Kingmax 16 GB micro-SD card21.0 MB/s 9.3 MB/s Bought to replace the small SD card above
Nikon-branded 2GB SD card11.3 MB/s 5.6 MB/s Came with my Nikon point-and-shoot camera
No-name 2GB SD 13.7 MB/s 5.2 MB/s

It's not a very interesting test; none of these is above the theoretical maximum of 60 MB/s for USB 2.

2013-09-28 Criticism of Google Drive

Update 2013-12-12

There have been some improvements to Google Sheets, including offline editing in Chrome.

Update 2013-11-16

The release notes mention that since Google Drive version 1.9, "you’ll be able to access your most recent Google documents on, even while offline. Simply open any Google document from your local Google Drive folder and Chrome will display the file for you to edit".

Note that this goes for documents, not for slides or sheets.

Original text follows

Here's a couple of problems I encountered with Google Drive.

The major problem I have is the offline feature of Google Drive. The obvious problem is that although Google drive seems to sync everything to your computer, that's not the case for the Google Apps documents (docs, drawings, sheets and slides). These are just links to their online version. Thus Google Drive is two things: it syncs normal files to between Google and local storage. And it lists all Google Apps documents. In that last instance, no real two-way syncing there. You can't create a local Google Apps document or something.

You can, however, use Google Chrome (not any other browser), and say that you want to be able to work offline. You can then go to

This feature doesn't come with Google Drive, nor is it enabled by it. It's a completely separate thing, and it will download your complete document storage again on your local drive, in your browser's cache. Being able to work offline means that you've got everything replicated twice on your local harddrive.

The second major problem is related to usage of this offline feature. I have tried to use the offline feature in three scenarios:
- Via a severely limited EDGE connection (i.e. not 3G or HSDPA, just plain old EDGE) with a maximum speed of 80 Kbit/s
- Via a crowded wireless network, which my MacBook kept dropping
- Via a wireless network in the train, which in its turn was routed through a 3G connection

In neither of these instances, Google Docs was able to keep up. Sometimes, the interface doesn't load. Sometimes, it thinks it loses connection because the speed is in the range of 8 to 40 Kbit/s. Often, it fails to save the last typed lines and says I should reload the document (which then loses the last sentences entered). All the time, it displays errors about not being able to connect, a vague error about server error, or about reconnecting.

All in all, it works when you have a steady net connection. But the technology isn't there yet.


Small update. Google introduced QuickOffice back into the App Store. It allows you to edit Microsoft Office documents. This sounds great in principle. There's a couple of things I find weird with this. Firstly, this app is separate from the Google Drive app and I can't seem to find a good reason why. Secondly, the iPad now offers something that Google Docs (in your browser) does not offer. Thirdly, the QuickOffice app has a weird 'file cache', a storage area on the iPad itself, where you can create and edit documents. You can mail and print them, but you can't copy them to Google Drive.

Device & software Docs Sheets Slides Drawings MS Office Editable offline
iPad Google Drive app Edit Edit Edit View View Per document
iPad QuickOffice app View View View View Edit No
Browser Edit Edit Edit Edit View Only in Chrome
PC with MS Office --------Edit Always

In my opinion, this is too complicated.


In my opinion, it would be much better when you could simply say: "In your browser as well as on your iPad, Google Drive can edit all sorts of files in your browser, including MS Office".

We're not there yet. And I wonder if we'll ever see a full-fledged offline editing solution for Google Docs. After all, why would they? Then Google has to open their file format, just like Microsoft was forced by law to do with the Office file formats. Now you can say that Google is a lot more open, because you can easily download the documents in other formats. But you can't really -- they're not natively editable anymore after exporting. You could export, then import them again, but that is a bit strange for a document that's yours, and you'll lose lots of markup in the process.

I think Google sees the complex and weird situation that now exists, and I am very curious about the future.

2013-09-26 Speed comparison of different ciphers on an SSH tunnel

Today I got curious what the speed differences are when you use a different cipher for an SSH tunnel.

First, create a 100 MB testfile on the remote box:

 you@remote $ cd /var/www
 you@remote $ sudo dd if=/dev/urandom of=./testfile100mb.bin bs=1M count=100

Then set up a tunnel from your local desktop to your remote webserver, and use wget to test it:

In one terminal (using the default cipher here):

 you@local $ ssh -L8080:localhost:80 remote

In the other:

 you@local $ wget http://localhost:8080/testfile100mb.bin
 (3.87 MB/s) - ‘testfile100mb.bin’ saved

Now exit the tunnel and retry with another cipher:

 you@local $ ssh -c arcfour256 -L8080:localhost:80 remote
 you@local $ wget http://localhost:8080/testfile100mb.bin
 (3.87 MB/s) - ‘testfile100mb.bin.1’ saved

Hmmm, that's not much. And not encrypting anything doesn't work:

 you@local $ ssh -c none -L8080:localhost:80 remote
 No valid ciphers for protocol version 2 given, using defaults.

Then we'll just have to trust the developers:

2013-08-26 Testing real capacity of an USB stick

I've read about fake oversized thumb drives / USB sticks, which report a much bigger size than they actually are.

To test the real capacity of an USB stick under OS X, we'll create a test file and write it to the USB stick a number of times. Then we'll verify what has been written with a couple of terminal commands.

This is a pretty brute-force test and I'm sure it can be done much smarter. I'll give an idea on speeds, and I assume USB 2.0, and also assume a maximum write speed of 15 MB/s, and maximum read speed of 30 MB/s. Furthermore, we're working with 1 GB equalling 1 thousand million bytes, because that's what they use on the packaging.

First, we format the disk. Start Disk Utility and erase it. This is to remove any files in the Trash.

disk utility format usb stick.png

Notice the capacity in the lower right. In this case, it's 16 GB. We'll need that later.

Now open a terminal window. See under which directory the stick is mounted:

 $ ls -l /Volumes
 total 24
 lrwxr-xr-x  1 root    admin     1 Aug 20 07:12 Macintosh HD -> /
 drwxrwxrwx@ 1 root  staff  8192 Aug 26 12:07 STORAGE

In the case above, the USB stick is called 'storage'. Go there.

 $ cd /Volumes/STORAGE

First create a 100 megabyte test file on the USB stick like so (optimally, this takes 1 minute and 10 seconds):

 $ dd if=/dev/urandom of=test.bin bs=1000000 count=100
 100+0 records in
 100+0 records out
 100000000 bytes transferred in 9.029708 secs (11074555 bytes/sec)

Now copy the test file a number of times, as much as the number of gigs you expect on the USB stick, in this example, 16 gigabytes. To write a gigabyte, we need to write our 100 megabyte testfile ten times. 16 gigabyte times 10 is 160. A gigabyte takes about 1 minute and 10 seconds to write optimally. The OS might be able to completely cache the source test.bin file in memory, which saves on reading time.

 $ for i in $(seq 1 160); do echo "Writing file $i"; cp test.bin test$i.bin; done

Probably, the last file will fail because although the stick might be 16 gigabytes, the filesystem also uses up some space.

Display the checksum of the first file. This can take half a minute.

 $ md5 test.bin
 MD5 (test.bin) = 0d3c4fe338109b09b61fcea3e11b0e4b

Now test the checksums of all files, excluding the correct results. My 16 GB stick took 8 minutes.

 $ md5 *.bin | grep -v 0d3c4fe338109b09b61fcea3e11b0e4b

If you're testing a big SSD drive or something similar, better use xargs (because simply using an asterisk might be too much for the shell):

 $ ls *.bin | xargs md5 | grep -v 0d3c4fe338109b09b61fcea3e11b0e4b

If something went wrong, then you should either see the OS give errors, or some files should give a different MD5 hash. As mentioned above, you can probably ignore the last file, which may not be fully written to disk.

2013-08-12 Page up and page down with a Logitech Marble on Linux

On Debian 7.x (Wheezy), I'm using the Logitech Trackman Marble. Mine has product number P/N 810-000767, a fantastic and long-lasting device. It has two small up/down buttons above the normal left/right buttons

I wanted these working under Linux, and give them the PageUp/PageDown function. I went the imwheel route, which involves the following steps:

1) Install imwheel with:

 $ sudo apt-get install imwheel

2) Edit the file /etc/X11/imwheel/startup.conf to enable imwheel when you log into KDE. Find the line that starts with IMWHEEL_START, and set it to 1.

3) Edit the file /etc/X11/imwheel/imwheelrc and at the bottom, add the following lines:

 , Thumb1, Page_Down
 , Thumb2, Page_Up

4) Logout and log into KDE again
The Logitech Trackman Marble trackball.

2013-08-12 Measure the AC gain part 2

I've worked some more on a Python script that measures the gain (transfer) of the AC bias of the SQUIDs. To summarize my previous posting on this subject, this measurement assumes that we have put the array SQUID in flux-locked-loop. This also implies that we're dealing with a dual-SQUID configuration

Dual SQUID configuration.png

More on this procedure later.

2013-06-15 Automatic updating of Google Drive

On my bog standard MacBook Pro, it seems that there's an update problem with Google Drive. It fails to automatically update itself. Perhaps it has to do with the fact that I'm running as a standard user, not the Administrator that OS X supplies by default.

I found others who share this problem, and decided to write a little shell script that checks the current version against the new version. Run it once a week and see whether you need to update or not.

  DESTMOUNT="Install Google Drive"
  wget --quiet --no-check-certificate -O "$DEST" "$URL"
  hdiutil mount -quiet "$DEST"
  PREV_VERSION=$(/usr/libexec/PlistBuddy -c "Print :CFBundleVersion" "/Applications/Google")
  [ $? -ne 0 ] && exit 1
  VERSION=$(/usr/libexec/PlistBuddy -c "Print :CFBundleVersion" "/Volumes/$DESTMOUNT/Google")
  [ $? -ne 0 ] && exit 1
  #echo "old version: [$PREV_VERSION], new version: [$VERSION]"
  if [ "$PREV_VERSION" != "$VERSION" ]; then
      echo "Please install update!"
      echo "Google Drive is up to date."
      hdiutil unmount -quiet "/Volumes/$DESTMOUNT"
      rm "$DEST"

Copy the script into a file called "" and put it in your Documents folder, for example. Drop to the commandline and make it executable:

  $ cd Documents
  $ chmod ugo+x

Give it a test drive:

  $ ./

If it finds that you need to update, switch to finder and see that there's an opened DMG. Stop Google Drive, install the update and have fun.

2013-05-02 Test of current version of SAFARI scripts

Today, we ran a little test of the SAFARI demux/fee-scripts. It's the second time this week, and I'm quite happy how it all turns out.

We're finding some issues, but it's developing into a base from which we build a complete system. First we tested the menu for the testpulse generator.

This is an important part, because the testpulse generator can simulate a detector (with a pulse, it's like a particle hit the bolometer). It's fed into the feedback. Besides pulses, we can also generate a square wave. This allows the user to inject high frequencies into the system, which is apparently used to assess its robustness. For now, I'm just taking note of that.

When that appeared to work, we continued with the pixel menu. This is a part of the script that allows you to put a carrier wave routed to a certain pixel. You can change characteristics such as amplitude and frequency. We found some bugs there, but nothing serious.

We then turned on pixel 1, and routed the testpulse generator to that pixel, configured to generate pulses.

This resulted in the following picture on the scope:
demux test script scope.jpg

We then used the "demux test" part of the script, and configured it to grab data, triggered on the testpulse generator. That resulted in the following plot:
demux test script plot.jpg

The plot option has a number of options. The above plot is a "normal" plot (in the time domain). If you would zoom in, you'd see a sinus. The plot shows 400,000 samples or so, thus we'd just see a dense area plotted. The first part of the plot is pre-trigger data. Then the test signal generator pulses, and the amplitude drops. The logic behind that, is that our sensor behaves like a resistor. Upon particle impact, the resistance raises, thus the amplitude of our biasing signal is lowered. And that's why you see the "trough" in the plot above.

It's also possible to get a spectrum (i.e. plot in the frequency domain). We played with that too, and found some issues there.

Finally, we used the Demux Test menu choice, and configured it to grab data from ADC 1, which measures the error signal. This is the difference between the feedback and the input of the SQUID. The error signal is the cause of overdriving the SQUID. Because it's so sensitive to this, we want to eliminate the error signal as much as possible -- thus it's important that users can measure and plot this signal.

All in all, a nice result to head into the weekend.

2013-05-02 How does forking feel

May 1st brought us an awesome episode of the In Beta podcast on the 5by5 network.

Basically, Gina Trapani explains how it feels to a developer when her project is forked, and that forks take many different forms. Ultimately, it looks to me like good stuff is coming out.

2013-04-26 Asynchronous data from the SAFARI Demux board

Previously, I described how we measure the transfer of the AC bias.

To recap, we assume the user has characterized the DC biasing of the SQUID(s). When we measure the transfer of a single Array-SQUID or a dual-SQUID (i.e. pre-SQUID plus array-SQUID), we need to take the following steps:

  • Set a DC bias on the SQUID(s)
  • Set an AC bias current on the feedback coil
  • Measure back the AC output voltage
  • Increase the DC bias, and repeat the steps until we have covered a specified range (for example, from -5uA to 5uA in 20 steps)

Why do we put a test AC signal on the feedback coil, instead of sending it straight through a pixel on the sensor? Because each pixel has a hardwired LC filter, we could say. But that feedback coil is there, because the SQUIDs hardly have a dynamic range.

Let me put this in other words, because it's extremely important to understand the whole setup.

The SQUID is a very sensitive magnetometer. Thus the smallest change of flux will cause it to change its output. The output looks like a sine wave, thus it has an extremely small dynamic range (namely, the flank of a sinus). This means that we need a feedback loop that holds the SQUID locked to an output range. Because we have more than one pixel, we would run the SQUID out of its dynamic range. Thus the feedback loop ensures that, within its dynamic range, the SQUID can add all the pixel carrier waves.

To make this measurement, we use a set of registers that allow us to configure the Demux board to start spewing data over the second optical network link. It's very flexible and can be viewed as a number of digital data taps located on several points of the Demux board. You can set the data source, the amount of data, the trigger source, and finally whether you want a specified number of data blocks or just continuous data.

The data source can be configured to the ADC, the DACs, or the demodulated output of pixel 1 to 16. The ADC is of course what we need currently, but you can also read out the DACs. This is useful to check that the FPGA algorithms (more on that later) are correctly working. Finally to read the demodulated output of pixels is hugely useful, because (1) (see below).

The trigger source is worth a different topic, which involves explaining what the sensor actually measures (X-rays or infrared signals). The options currently are: testpulse generator (2), level triggering of selected pixels, both of these options, or just auto (get all the data).

For our purpose, we need to configure it as following:

  • Set the data source to read out the ADC that's on the Demux board (there's two ADCs for historical reasons, but we only use the first one).
  • Set the amount of data. We need to experiment with the amount that we actually need to request.
  • As the trigger source, we currently set it so data is always, and immediately, returned.

(1) Demodulation saves us bandwidth as well as processing on the PC.
(2) The testpulse generator is used to generate fake detector signals, to test the complete signal chain. You can test demodulation and/or test the electronical system. It generates the ideal detector signal. You can send the testpulse generator signal to the feedback coil of the SQUID and measure the PPM drift of the signal chain.

2013-04-15 State machine in Objective-C

Here's my version of a state machine in Objective-C, useful for your iOS or Mac OS X projects. There are many like it, but this one is mine. I've modeled an alarm clock here.

What I like about it: it's pretty small and light on the objects. The state transitions are the only thing that's done with an object, and even these could be replaced with a struct, but that causes problems with ARC (which doesn't like function pointers in structs).

The code below is WTFPL-licensed. Just so you know.

The header file:

    /* Enums that we need throughout this class to maintain state */
    enum WSAlarmState {WSAlarmStateInactive, 
        WSAlarmStateActive, WSAlarmStatePlaying};
    enum WSAlarmAction {WSAlarmActionStart, WSAlarmActionPlay,
        WSAlarmActionSnooze, WSAlarmActionStop};
    /* Describes how one state moves to the other */
    @interface WSAlarmStateTransition : NSObject
    @property enum WSAlarmState srcState;
    @property enum WSAlarmAction result;
    @property enum WSAlarmState dstState;
    /* Singleton that maintains state for an alarm  */
    @interface WSAlarm : WSNotification
    @property enum WSAlarmState currentState;

The header file contains the enums for the states and the actions. Note that these actions are both used as a return value and as an input value.

The implementation file starts with the init method, which sets up the state transition table. Basically, this table says: given a state and a resulting action, what is the next state?

Furthermore, it contains a function that does all transitions, a function that looks up the next state, and the state methods.

    #import "WSAlarm.h"
    @implementation WSAlarmStateTransition
    - (id)init:(enum WSAlarmState)srcState :(enum WSAlarmAction)action :(enum WSAlarmState)dstState
        if (self = [super init]) {
            // Do initialization here
            self.srcState = srcState;
            self.result = action;
            self.dstState = dstState;
        return self;
    #pragma mark -
    #pragma mark WSAlarm class
    @implementation WSAlarm {
        NSArray *stateMethods;
        NSArray *stateTransitions;
    - (id)init
        if (self = [super init]) {
            // Do initialization here
            self.currentState = WSAlarmStateInactive;
            /* This array and enum WSAlarmStates must be in sync! */
            stateMethods = @[
              [NSValue valueWithPointer:@selector(noAlarmActiveState)],
              [NSValue valueWithPointer:@selector(alarmActiveState)],
              [NSValue valueWithPointer:@selector(playingAlarm)]
            stateTransitions = @[
               [[WSAlarmStateTransition alloc] init:WSAlarmStateInactive
               [[WSAlarmStateTransition alloc] init:WSAlarmStateActive
               [[WSAlarmStateTransition alloc] init:WSAlarmStateActive
               [[WSAlarmStateTransition alloc] init:WSAlarmStateActive
               [[WSAlarmStateTransition alloc] init:WSAlarmStatePlaying
               [[WSAlarmStateTransition alloc] init:WSAlarmStatePlaying
               [[WSAlarmStateTransition alloc] init:WSAlarmStatePlayedInBackground
               [[WSAlarmStateTransition alloc] init:WSAlarmStatePlayedInBackground
        return self;
    - (void)registerDefaults
    #pragma mark -
    #pragma mark Convenience methods
    - (void)start:(NSDate*)notifDate
        self.notificationDate = notifDate;
        [self transitionToState:WSAlarmStateActive withAction:WSAlarmActionStart];
    - (void)stop
        [self transitionToState:WSAlarmStateInactive withAction:WSAlarmActionStop];
    - (void)play
        [self transitionToState:WSAlarmStatePlaying withAction:WSAlarmActionPlay];
    - (void)snooze
        [self transitionToState:WSAlarmStateActive withAction:WSAlarmActionSnooze];
    #pragma mark -
    #pragma mark State machine
    // Walk through table of transitions, and return new state
    - (enum WSAlarmState)lookupTransitionForState:(enum WSAlarmState)state
        withResult:(enum WSAlarmAction)action
        enum WSAlarmState newState = -1;
        for(WSAlarmStateTransition *t in stateTransitions) {
            if(t.srcState == state && t.result == action) {
                // We found the new state.
                newState = t.dstState;
        if(newState == -1) {
            NSString *msg = [NSString stringWithFormat:
                @"Can't transition from state %@ with return code %@",
                alarmStateString[state], alarmActionString[action]];
            @throw [NSException exceptionWithName:@"TransitionException"
        return newState;
    - (void)transitionToState:(enum WSAlarmState)newState withAction:(enum WSAlarmAction)result
        NSValue *stateMethodValue = (NSValue*) stateMethods[newState];
        SEL stateMethod = [stateMethodValue pointerValue];
        // We need these because otherwise we get a warning
    #pragma clang diagnostic push
    #pragma clang diagnostic ignored "-Warc-performSelector-leaks"
        NSNumber *param = [NSNumber numberWithInt:result];
        enum WSAlarmAction nextAction = [self performSelector:stateMethod withObject:param];
    #pragma clang diagnostic pop
        self.currentState = [self lookupTransitionForState:self.currentState withResult:nextAction];
    #pragma mark -
    #pragma mark States
    - (enum WSAlarmAction)noAlarmActiveState:(enum WSAlarmAction)action
        // Some code to stop the alarm
        return WSAlarmActionStop;
    - (enum WSAlarmAction)alarmActiveState:(enum WSAlarmAction)action
        if(action == WSAlarmActionSnooze) {
            // User tapped "snooze", stop the sound
        } else if(action == WSAlarmActionStart) {
            // No alarm active, user starts alarm
        } else {
            // We reached state alarm active with a weird action
        return WSAlarmActionStart;
    - (enum WSAlarmAction)playingAlarmState:(enum WSAlarmAction)result
        // Some code to play a sound
        return WSAlarmActionPlay;

2013-04-11 measure the AC gain

A little intro

Another routine in the system is the AC gain curve measurement.

This routine is different from the previous one, 2013-04-09 Characterizing the transfer of a voltage biased SQUID. In that routine, we just looked at the DC part of the amplifier chain.

In this measurement, we assume those measurements were successful, and that the amplifier chain was set according to optimal parameters.

And since our instrument basically uses AC biased TESes (as explained here), we want to measure the transfer of the AC bias, after the SQUID has been configured optimally. This is very interesting, because that means you see the instrument as a whole, instead of characterizing a small part.

The sensor:
safari sensor.jpg

The measurement

So what we're going to do, is sweep the AC bias and then measure back the result.

To do this, we have to send the FPGA the appropriate commands. First, some schematics:

On the Demux board, there are two DACs that can drive the SQUID in the cryostat. There is also an ADC to measure back the result. The FEE board is in between here, but for now we can ignore it. The cryostat is where the above sensor is located. It has an array of TESes, each with its own LC filter.

If we want to feed in an AC bias, we don't want to use DAC2, because we don't want it filtered through the LC filters. Instead, the FPGA exposes a register where we can switch the input line of both DACs, so you can feed the AC bias signal to DAC1. Note that this AC bias signal is still digitized, otherwise we couldn't feed it into a DAC :-)


The user needs to be able to adjust the AC bias signal. The FPGA exposes a number of parameters for this, but we just need the frequency and amplitude, it's a really small signal: 2 MHz at 1 uA. Both are adjustable, and I'll add a setting for 1 and 5 MHz as well.

The data

We retrieve the data asynchronously. This is rather different from our usual way of getting data; i.e. normally we'd just polling a particular register.

In this case, the hardware has a special function called the "science switch". A possible analogy could be that this science switch acts as a number of water faucets, where you'd just let the raw data stream out of the board, from specific points. Since water and electronics don't mix much, it's actually a number of registers. I'll write more on that later.

2013-04-09 Characterizing the transfer of a voltage biased SQUID

Currently, I'm finishing a software routine to characterize the transfer (see below) of a voltage based SQUID. Simplified, our electronics could schematically be shown as follows:


Firstly, about this schema.

On the left, you see a single SQUID. This is the pre-SQUID. In the middle, you see an array of SQUIDs. We functionally treat that last one as a single squid, calling it "the" array-SQUID. These two types of SQUIDs are biased differently; the pre-SQUID is voltage-biased, and the array-SQUID is biased with a current.

You can view this combination as one "dual-stage SQUID".

So what we do, is: we set a current over the shunt on the left. This will put a voltage on the pre-SQUID and a magnetic field will form on the inductor next to the array-SQUID. We'll read out an output voltage on the right.

Now normally, the amount of voltage we put on the pre-SQUID would overdrive the array-SQUID. Thus, we put it in flux-locked loop. This is a mode where the array-SQUID simply passes current that results from the changes in flux.

Because the array-SQUID is in flux-locked-loop (FLL), we can measure the output current of the pre-SQUID without a dynamic range limitation (+/- 500 uA). This flux-locked-loop is nothing more than a feedback circuit which forces the bias of the array-SQUID down to zero.

Note that this is all DC-biased. The SQUIDs are part of the amplifier chain, and the whole amplifier chain is DC-biased. After we've characterized the amplifier chain, our primary purpose is to read out the sensor, consisting of a TES array that's AC-biased, see also 2012-07-02 Minimizing amplifier chain offset.

Secondly, about the routine itself.

I said "characterize the transfer". The transfer means: what is the effect of the input on the output. If you draw this function, you'd ideally see a sinus. We want to know in which voltage range our pre-SQUID behaves the best (i.e. where it has the highest dynamic range. In other words, you want to pick the voltage that's right in the middle on a flank. Because that way, with a little bit of input, you get a nice stretch of usable output.

Compare it to a radio that can receive a lot of stations.

2013-03-27 warning Setting locale failed

I was getting the error from all kinds of Linux programs about "setting locale failed". For example, running Perl gave me the following output:

 user@machine:~$ perl --version
 perl: warning: Setting locale failed.
 perl: warning: Please check that your locale settings:
 	LANGUAGE = (unset),
 	LC_ALL = (unset),
 	LC_CTYPE = "UTF-8",
 	LANG = "en_US.UTF-8"
     are supported and installed on your system.
 perl: warning: Falling back to the standard locale ("C").

The easy solution

If you're running Debian Linux or some derivative, you may also see the following warning when you run some command:

 Warning: locale not supported by C library, locale unchanged

The solution is to install all locales:

 $ sudo apt-get install locales-all


If the above didn't help, or isn't applicable, continue.

What it comes down to, is that I was using SSH from my MacBook (running OS X Mountain Lion 10.8.3) to log into my Debian Linux server (running Squeeze).

Apparently, after logging remotely into Linux, the SSH client sets the LC_CTYPE environment variable to the same value as OS X. This conflicts with the LANG environment variable that Debian sets by default. That's because under Linux, the LANG is an overarcing variable, from which others, like LC_CTYPE, can be derived. That wouldn't be a problem, except the LC_CTYPE variable its contents are differently formatted under OS X and Linux.

The warning says that it's "falling back to the standard locale". That means nothing will be translated, and only ASCII will be used when printing characters. No unicode support, I guess?

The irritating warning can be removed with a small change on the Linux side, by unsetting the LC_CTYPE in your .bashrc. Add the following line:

 unset LC_CTYPE

However, this gave me problems when using SVN from the Macbook; whenever I'd do some remote action, I'd get something like the following:

 user@macbook:trunk$ svn log --limit 1
 svnserve: warning: cannot set LC_CTYPE locale
 svnserve: warning: environment variable LC_CTYPE is UTF-8
 svnserve: warning: please check that your locale name is correct 

Thus alternatively, you can fix this with a similar change on the OS X side, by adding the following two lines to .bash_profile:

 unset LC_CTYPE
 export LANG="en_US.UTF-8"

Instead of unsetting LC_CTYPE, you can also go to the Terminal app its preferences, open the Settings tab, select your profile on the left side, go to the Advanced tab, and uncheck "Set locale variables on startup".

2013-03-13 Minimize cellular usage on iPhone

There's a number of guides on the internet that talk about minimizing cellular usage. Often, they go into generalities, like "don't watch streaming video". Well, duh. I'm not going to offer general tips, but instead list a few settings for the iOS and some popular apps so your iPhone (or iPad) prefers WiFi whenever possible.

In the Settings app:

  • iTunes & App Stores -> Use Cellular Data -> OFF
  • iCloud -> (several settings) -> OFF
  • Mail, Contacts, Calendar -> Fetch New Data -> Manually
  • Music -> iTunes Match -> OFF
  • Music -> Lyrics & Podcast Info -> OFF
  • iBooks -> Sync Bookmarks -> OFF

Separate apps:

  • Podcasts: do not use this app; it has a setting to only use WiFi, but seems to have a bug in this feature
  • Instapaper: tap Settings -> Advanced -> Update -> On Wi-Fi Only
  • Evernote: tap Evernote icon -> tap username -> General Settings -> Sync on WiFi only

There's some things I'm not sure whether they use cellular data, but I'm listing them here:

  • In the Settings app: General -> About -> Diagnostics & Usage -> Don't Send

Edit: there's a much better, much longer list here:
TidBITS: What’s Behind Mysterious Cellular Data Usage in iOS 6?
TidBITS: Mysterious iOS 6 Cellular Data Usage: A Deeper Look

2013-03-05 Forcibly drain your cellphone battery

If you want to forcibly drain your iPhone battery for some reason, first configure the display. Turn off the auto-lock, set the auto-brightness off and manually turn up the brightness to 100%.

Then go and warm up that little CPU by running a JavaScript benchmark, either with SunSpider if it's already pretty empty. Or go ahead and release the Kraken, which is a heavier test.

2013-01-07 Onward flight not checked-in

When flying internationally with multiple legs, I've ran into the situation where I'd use the web check-in but get the error message:

  Onward flight not checked-in. [HR 2606]

The possible reason is, that you're trying to check in too early. So although you might be able to check in for the first leg, you're too early for the second leg.

This situation had me stumped twice, until I realised the above reason is what might go wrong. The error message is not very helpful of course, so I'm hoping this is the actual reason, and documenting for everyone else. Here's hoping Google finds this.


Here's a one-liner for you today:

  $ sudo ngrep tcp and port 80 -W byline

This prints HTTP traffic to the console, including full body.

2012-12-04 Objective-C with a Makefile

Here's a little example on how to compile Objective-C on the commandline, with a makefile. All code can be downloaded here: test or copy/pasted from below.

Create a file called Makefile:

 LIBS=-framework Foundation
 all: test_cars
 test_cars: test_cars.m car.m
 	$(CC) $(LIBS) test_cars.m car.m -o test_cars
 	rm -f test_cars *.o
 Now create a header file:
 #import <Cocoa/Cocoa.h>
 @interface Car : NSObject {
         float fillLevel;
 -(float) gasLevel;
 -(void) addGas;

Plus the implementation:

 #import "Car.h"
 @implementation Car
 -(id) init
     self = [super init];
     if (self) {
         fillLevel = 0.0;
     return self;
 -(void) addGas
     NSLog(@"Added one gallon!");
     fillLevel += 1.0;
 -(float) gasLevel
     return fillLevel;

And the file containing main:

 #import <Foundation/Foundation.h>
 #import "car.h"
 int main(int argc, const char * argv[])
     @autoreleasepool {
         // insert code here...
         Car *c = [[Car alloc] init];
         NSLog(@"Level = %f", [c gasLevel]);
         [c addGas];
         NSLog(@"Level = %f", [c gasLevel]);
     return 0;

Type make in the directory containing the above files, and run the example with ./test_cars. Enjoy the commandline goodness.

The code in car.m and car.h is mostly by your friendly neighborhood Java/Grails developer.

2012-11-11 Starting with storyboard

I've never seriously worked with the storyboard before in Xcode, so here's a summary of what I've gathered so far.

  • The storyboard defines the flow of screens throughout your app, it's basically a collection of view controllers and their NIBs.
  • A storyboard is made up of scenes that are connected with segues (pronounced "SEG-way").
  • A scene here means: a view controller plus view.
  • A segue specifies how you go from one view controller to the next.

Container view controllers (such as a navigation controller, or a tab bar controller) have a special place in a storyboard, since these container view controllers define how the navigation takes place.

Thinking about the implementation details, I'd expect the storyboard to actually show a graphical representation of an XML file. In this file, the list of screens is saved, including the way how the screens are shown (with animations or what have you). Also present would be the type of container view controller.

In other words, in the MVC-pattern, the storyboard is the input for an over-arcing controller. A big data file, a specification, according to which all container view controllers are instantiated with the correct parameters. Instead of coding, you drag and drop this specification using Interface Builder.

You still have to put in some navigation code, though. Basically you code which segue to perform:

 [self performSegueWithIdentifier:@"OfferDetails" sender:self];

If I'd had to put it in one sentence: storyboard replaces the code for navigating between views.

2012-11-09 Full-text searching

In a previous project, I solved the full text search problem as follows.

Management overview: it's very easy to do full text searching but very hard to assign a weight to the results. MySQL supports full-text search but only in a degraded type of database (MyISAM instead of the full-featured InnoDB). The problem can be fixed by creating an "archive" table in this degraded MyISAM format, where you store a business ID, plus a summary of the data you're looking for (i.e. a concatenation of business name, address, reviews, locations etc). The MySQL full-text search facility then becomes available, and you get results as such:
(score, businessID, summary)
(0.99, 12, "Clear Cafe A beautiful restaurant that serves raw, vegan and seafood cuisine Restaurants Jl. Hanoman, Ubud, Bali 80571, Indonesia 03618894437 Ubud Cheap and good food Bart I've been there with a 10+ people group and they were happy to accomodate apparently they also blah blah")

As you can see, the summary is just a concatenation of name, address, category, reviews, phone number, everything. There is lots of duplicate information in there, but that doesn't matter for the results.

Tech details: we were doing scientific measurements and these were stored in a bunch of InnoDB tables. You can mix table types InnoDB and MyISAM in a database schema. I created a separate table called 'fulltexts' in the database, with basically two fields: "measurement ID", and "summary". Any human-readable fields from any of the tables were concatenated into a summary. I created a database row-level trigger for a couple of tables, so that each insert or update would concatenate a buch of fields and do an insert into 'fulltexts' (or a replace, if the summary needed updating). Then, this table can be searched by the MySQL facility "full text search", and give you back a result with a relevancy. This is quite nifty and fast.

2012-08-12 Xcode 4.4 won't quit

I've had a problem with Xcode 4.3 and 4.4 where it will not quit after stopping the simulator. The problem is that Xcode thinks that tasks are running (even with the simulator not running). The only solution is to force-quit Xcode, which is a pretty major interruption in your workflow.

Here are some threads that appear to talk about the same problem:
Failure to stop executing task in Xcode 4.3
Why does Xcode 4.3.1 (4E1019) / 4.3.2 (4E2002) hang regularly with iOS simulator?
My 2 XCode 4.4 problems

I haven't found a workaround so far. The last thread on indicates that you should add some extra logging to find the problem:

 defaults write IDEDebuggerLogToFile ~/IDEDebugger.log

Update: the problem seems to occur less, when I quit the task from Xcode. Normally I'd just punch Alt-Q to quit the simulator altogether; now I alt-Tab to Xcode first, then punch Alt-. (Alt-period) to stop the running task.

Update 2: I found a suggestion to clear the project from the organizer window: Xcode 4.3.2 process hangs.

Update 3: It still happens, even with Xcode 4.6... There's another suggestion that says to click the Simulator icon in the dock, which seems to work for now.
XCode 4.3.2, issue with running on simulator

This is my pet issue, and since this is the internet, every time Apple releases a new version of Xcode, I will loudly point to this bug and ask Why haven't they fixed this??!?!?!!111!! When they do fix it, I'll still keep using it as an example why Xcode is "broken".

2012-07-15 strace on Mac OS X

If you're looking for the Linux strace command on OS X, stop looking. OS X doesn't have this command, but what it does have, is dtruss.

I made the mistake of assuming dtruss was a straight replacement, so I ran it with the process name as its argument; i.e:

 $ sudo dtruss /Applications/SomeRandomApp/Contents/MacOS/SomeRandomApp
 dtrace: failed to execute /Applications/SomeRandomApp: file is set-id or
 unreadable [Note: the '-c' option requires a full pathname to the file]

The error message had me stumped, but after some reading up, dtruss is just a different command altogether. Rather you run it as follows:

 $ sudo dtruss -n SomeRandomApp

And in another terminal, or just via the GUI, run the command.

This came in handy when using the excellent JollysFastVNC which was failing to connect over an SSH tunnel without telling me the error. I found out the error as follows:

 $ sudo dtruss -n JollysFastVNC\ Home -t read
  6110/0x3af1f:  read(0x17, "Bad stdio forwarding specification '\\'\r\n\0", 0x1000)		 = 40 0

That gave me a clue that I made a typo in the SSH Options field in JollysFastVNC.

See also Brendan Gregg's blog.

2012-07-12 Deflux SQUID

Another update on the routines that I'm writing for the SAFARI project. I've already explained that we work with a sensor type called TES, and that to read out this sensor, we use a device called a SQUID. Our SQUID is basically a 3x3mm chip, Basically, the TESes and the SQUID are put together in an electronic circuit, which is then immersed in liquid helium.

Since at this temperature (at or below 4 Kelvin), the circuit is superconducting, the SQUID can trap flux because a superconducted current is flowing around in the SQUID.

Since the SQUID is basically a magnetometer, we need trapped flux just as much as we need a punch in the face. The flux can be removed by heating the SQUID a little but, but only just so it no longer superconducts. This is a very common thing for superconducting sensors, so our readout electronics have firmware which contains a "remove flux" procedure.

However, you can't just put the complete procedure in firmware, because we have only one version of electronics and we have multiple different setups in the labs. Each of these setups has a different cryostat with different temperatures (anything between 4 Kelvin and 50 milliKelvin). So the firmware deflux routine has a bunch of parameters which should be controlled through a software routine, which the user can then configure and kick off.

You might wonder why we don't just do it all in software. The reason is, that for the best result, the heating current must be stopped abruptly and the timing aspect is much more tightly controlled by using firmware.

2012-07-09 Mac OS X command line utilities

Recently I upgraded my macbook its SSD drive for a bigger version. Since I wanted to start with a fresh OS X copy, I decided not to use the Migration Assistant but rather just manually copy over the home folders (like Documents, Pictures etc).

I want to re-use the old, smaller SSD, but there might be some files which I forgot to copy. So what I was looking for, is a way to create a disk image like I'm used to when running Linux.

Below, the procedure is documented. It's probably wise to stick nice -n 20 before the commands dd and cat, so normal desktop usage is not bothered by the I/O, but for clarity, I've removed it.

First, connect your USB drive and type mount to show which drive got connected:

 $ mount
 /dev/disk0s2 on / (hfs, local, journaled)
 devfs on /dev (devfs, local, nobrowse)
 /dev/disk1s2 on /Volumes/storage (hfs, local, journaled)
 map -hosts on /net (autofs, nosuid, automounted, nobrowse)
 map auto_home on /home (autofs, automounted, nobrowse)
 /dev/disk2s2 on /Volumes/MacSSD (hfs, local, nodev, nosuid, journaled, noowners)

In this case, the last line shows that /dev/disk2 is the device we need, and partition 2 contains an HFS-formatted partition (or volume, in OS X speak). You can zoom in on this disk with the command diskutil list and show all partitions of all disks.

Since OS X automatically mounts volumes that a user attaches, we can now change to the directory where it's mounted, and zero the empty space:

 $ cd /Volumes/MacSSD
 $ cat /dev/zero > zero.file;rm zero.file

We'll now unmount the volume so we can make a copy of the whole partition (instead of the contents of the volume):

 $ cd
 $ hdiutil detach /dev/disk2s2

Now, we can make a literal copy of this partition:

 $ dd if=/dev/disk2s2 of=MacSSD.hfs bs=1m 

Over USB, this could take a loooong time, though. After that, compress the image with:

 $ gzip MacSSD.hfs

Now it's safe to move this file somewhere else. If you want, you can encrypt the file before doing so.

2012-07-02 Minimizing amplifier chain offset

For the SAFARI project, I'm refactoring some code, and one part of that is a routine to minimize the offset of the front-end electronics or FEE. For the place of the FEE in the setup, see also 2012-03-29 Extensions to Safari library.

The overall setup

Here is a picture of the lab setup.

lab setup with cryostat.jpg

From right to left: the PC with our lab software, the DEMUX board and some other measurement equipment in the rack, and the cyostat with on top the FEE board.

Zooming in on the cryostat; we're looking at a closed cryostat now. It's filled with helium because the scientists are doing a measurement. Before the test, the cryostat was opened:

opened safari cryostat.jpg

The sensor

The type of sensors used by the SAFARI instrument are TESes: Wikipedia article on transition edge sensors. As you can read with in this article, this type of sensor uses a SQUID in its readout electronics.

The TES basically consists of a tiny surface ("pixel") which is biased by a current source from the DEMUX board. When a photon hits the pixel, its resistance increases and the voltage drops. You detect this via a SQUID, whose output is amplified outside of the cryostat.

The TES its setpoint is 20 mOhm and if it's hit, the resistance increases to 100 mOhm. It's superconducting, so it's hard to get it out of its setpoint with heat, so flux is used.

The SQUID is controlled with a DC voltage, meaning we set and characterize it via a DC voltage. It's supplied by our DEMUX electronics board. Reading out happens with an AC voltage. It's supplied by our FEE electronics board, which also amplifies the output so it can be digitized by the DEMUX board. See also this schema:


The signal

We currently use a single SQUID for all the TESes. The SQUID combines these in one signal, which the DEMUX board then splits up again so we can read out each individual pixel. To see what I mean, here is the sensor:

safari sensor.jpg

Description: at the top left, we have the TESes. On the bottom left, we have the LC filters (see below). On the right, we prominently see the black connectors that bring the signal out of the sensor. On the bottom right, we see the single SQUID. Note the connections in the top and bottom middle. If we increase the number of pixels, these connections will stay the same because the TES signals are multiplexed (put together). To get an idea of the physical scale, this thing has a width of 12-13 cm (I haven't measured it exactly, just an estimate).

There is one LC filter for each pixel. Basically, with the LC filter you pick out a certain frequency. Thus the LC filter controls which information the pixel gives us; it makes sure the spectrum we're measuring is nicely divided over the pixels. Wikipedia has an article on LC circuits which is hard to read for me, but do notice the applications header which I've linked to. First entry says that the most common application is tuning, which is exactly what's being done here.

As an aside: two or more SQUIDs

SRON bought the SQUID from elsewhere, namely Magnicon, and then bonded to the silicon substrate. As for the SQUID, there are other possibilities. Per pixel row, you could use one SQUID. Or you could still use the above setup, but with an extra SQUID, which we call the dual-SQUID configuration. This configuration has been designed, but for this to work, we need to characterize both SQUIDs and since they influence each other, we'd need to use a flux-locked loop to lock the first SQUID in its set point, then characterize the other. I might have to expand on this sometime later.

Looking ahead

What I've described above, is just the lab setup. With this setup, we can only detect power intensity, but the SAFARI instrument is a spectrometer. Thus, the instrument consists of an FPU (focal plane unit), which is a box containing mirrors, shutter, filter wheels, a cooler, an FTS and three FPAs (focal plane assemblies). These FPAs are basically little cubes with the above described sensor in there. The focal plane unit is cooled to 1.7 Kelvin, and the focal plane assemblies are cooled to 50 milli-Kelvin via an intermediate 300 milli-Kelvin stage. Outside of the focal plane unit, we also have two "warm" units. Basically these have the function of the current DEMUX and FEE boards, but our current boards are meant for a lab situation, so new electronics will be made.

So the sensor measures power intensity, and the FTS will give us intensity as a function of wavelength. In other words: the instrument will send us power plus mirror position (of the FTS), and these can be transformed mathematically to create a spectrum. Since SAFARI is an infrared instrument, we are looking in the 34 to 210 micrometer wavelength.

Current task

So far I've described the setup. Now on to my current task.

To accurately set the DC bias on the pre-amplifier chain that consists of a single SQUID, an amplifier, and an integrator (??), we need to measure the offset that's introduced by the FEE.

The basic schematic of the SQUID, the amplifier and the integrator look as follows:
schema single squid.png

On the left, we have the single SQUID. It's biased in a range between 22 and 44 uA. On the right, we have two functions. The top right shows a function of the FEE board, and the botton right (with the DC voltage part) shows a function on the DEMUX board; the DAC which sets a voltage, and the integrator that keeps the voltage over the setpoint of the SQUID. Below the SQUID is

This offset can be adjusted via the integrator-offset bias generator (noted as DAC in the above schema (??)). It's also adjusted via:

  • the LF_Gain, which has a range of 100..10k
  • the LF_Input Impedance
  • the LF Amplifier configuration

Note: the ?? indicates I have to check this statement


SPICA - SAFARI Link to the SRON homepage concerning the project

2012-06-01 OS X Security

After the recent FlashBack debacle, it seems prudent to look at the way you work with OS X. A good way to start:

2012-05-19 Telfort ADSL modem Zyxel P-2812HNU-F1 and VoIP Planet

(English readers: this is an explanation on how to configure the modem of a Dutch ISP for VOIP usage).

Onlangs heb ik mijn Telfort ADSL abonnement geupgrade en kreeg een Zyxel modem, type P-2812HNU-F1. Vorig jaar heb ik reeds voor type Zyxel 2601HN-F1 instructies gegeven, hierbij een update.

Om via een SIP account bij VoIP Planet te bellen met een telefoontoestel dat je aansluit op het modem, volg de volgende instructies op.

Om het modem te configureren, log in als admin (standaard wachtwoord: 1234) op het modem via adres en kies in het onderste menu genaamd VoIP voor de optie SIP.

zyxel sip menu.png

Het scherm dat tevoorschijn komt, heeft drie tabbladen. Het eerste tabblad is genaamd "SIP Service Provider". Laat het drop-down menu "Service Provider Selection" staan op "Provider-1" en vul deze als hieronder in. Klik daarna op Apply, en het drop-down menu toont nu netjes "VoipPlanet".

zyxel service provider.png

Klik daarna op het tweede tabblad, en klik op het pen-en-papier icoon in de Modify kolom, eerste rij. Er komt een popup-scherm waarin je slechts een viertal dingen hoeft te veranderen. Eerst moet je de "Active SIP Account" aanvinken, dan vul je de SIP Account Number in, en daarna Username en Password.

Waarbij je natuurlijk de vermeldde SIP Account Number en Username verandert in diegene die je van VoIP Planet hebt gekregen:

zyxel sip account.png

Als het goed is, hoef je niet naar beneden te scrollen. Klik op apply om op te slaan. Ga onderaan het scherm naar het menu System Monitor, en kies voor Log.

zyxel system monitor.png

Activeer de tweede tab, "Phone Log". Hier moet nu staan:

  SIP Registration: SIP:12345: Register Success


zyxel sip success.png

Je kunt nu bellen met het aangesloten (analoge) telefoontoestel via VoIP Planet!

2012-03-29 Extensions to Safari library

I've received the specs for a modification to our software library for the SAFARI project, and after a two-month hiatus, I'm looking again at what's currently present.

What's existing, is basically a script that displays a menu with a bunch of testing options. It's called demuxtest, because it tests the electronics board which we call "the demux board".


The Demux board plays the intermediary role here; all commands are routed through this board to the FEE (front-end electronics) board. If the FEE board isn't present, then it can generate a dummy response for testing purposes.

Zooming in on the electronics, it would look as follows:


You could roughly see the demux board as the digital part, and the rest of this picture as the analog part.

Some visuals

This is our test setup:

demux fee test setup.jpg

The FEE board

This is how the FEE (front end electronics) board looks like:

fee board.jpg

The demux board

This is the demux board:

demux board.jpg

2012-03-20 graphics glitches on OS X Lion

Graphics glitches

For the past days, I've seen a number of glitches on OS X Lion (update 10.7.3) with my mid-2010 Macbook Pro, and apparently, I'm not the only one.

It seems to happen especially on external monitors:

However, iMacs have a lot more trouble:

A possible solution for the graphics glitches is mentioned at the end of the thread:

White screen

UPDATE: this issue seems fixed with the 10.7.4 update

I've encountered a bug where the screen would blank, reproducible as follows:

  • With a closed laptop, hook up an external monitor (mini-DisplayPort) and keyboard
  • Press a key to wake screen, then I start working
  • At the end of day, I unplug keyboard and laptop
  • When coming home, open lid and click "switch user" (I have security settings such that it locks the display when closing the lid)
  • Look at white screen, where the mouse cursor is visible. Sometimes, after up to a minute, the screen will redraw correctly and show the login screen. Sometimes it'll just stay this way, and a hard reboot is required.

Black screen

I've also encountered a bug where the laptop screen would just stay black, not reproducible. I took the following steps:

  • With a closed laptop, hook up an external monitor and keyboard
  • Press a key to wake screen, then disconnect screen with the mini-DisplayPort
  • Open laptop lid, stare at black screen

Remotely I could log into the laptop, though.

Another set of circumstances happens as follows:

  • With a closed laptop, hook up an external monitor and keyboard
  • Go away until display sleeps
  • Come back and discover laptop cannot be woken up, but is still accessible via SSH

The following lines are then visible in the kernel.log:

 --- last message repeated 1 time ---
 kernel[0]: NVDA(OpenGL): Channel timeout!                                                
 --- last message repeated 1 time ---
 kernel[0]: NVDA(OpenGL): Channel timeout!
 --- last message repeated 1 time ---
 kernel[0]: NVDA(OpenGL): Channel exception!  exception type = 0xd = GR: SW Notify Error
 kernel[0]: NVDA(OpenGL): Channel timeout!
 kernel[0]: NVDA(OpenGL): Channel exception! exception type = 0xd = GR: SW Notify Error
 kernel[0]: NVDA(DisplayBase): Channel timeout!

UPDATE: this is still occurring with 10.7.4 update. Workaround is to keep the lid open

2012-03-08 Launchctl on OS X

Comparison of SysV initialization and launchd

If you're an old-school Unix or Linux person, you're probably also used to the SysV system initialization routine. Nowadays, it's slowly being replaced by upstart and the likes, but I'm going to skip that for now. This is meant as a guide for ye olde Unix neckbeards.

Basically, SysV init means:

  • Daemons are controlled with scripts in /etc/init.d
  • These scripts take a couple of standard arguments, such as start/stop/restart/reload
  • There are three or four so-called runlevels, and the higher the runlevel, the more capable the system is (in other words, more daemons are started up)
  • The sequence of starting the daemons (i.e. calling the init.d scripts with "start") is determined by specially named symlinks in directories named for the runlevel, for instance for runlevel 3, the directory /etc/rc3.d contains a bunch of symlinks to start the daemons appropriate for runlevel 3
  • If you want to prevent a daemon from running, then you remove symlinks in the /etc/rcX.d directory. This is usually done through a command which manages this for you. Redhat uses chkconfig, Debian uses update-rc.d.
  • To control specific daemons, you call the appropriate script in /etc/init.d with the start or stop parameter

OS X does this differently, and superficially, much simpler.

When the system boots, the launchd process is responsible for starting all daemons. To sum up the characteristics of this method:

  • Daemons are controlled with the launchd command, there aren't any start/stop scripts
  • Description of how to start a daemon is specified in a configuration file in the folder /System/Library/LaunchDaemons, and these files have the binary plist format, editable via the defaults command
  • Launchd does not support dependencies, thus it's not possible to configure (for instance) starting the database daemon before the webserver.
  • You can prevent daemons from running by using the launchctl command
  • To control specific daemons, you use the launchctl command

There is a further distinction which I haven't talked about. OS X has the concept of user-specific daemons. These are called "launch agents" instead of daemons and they have their configuration files in /System/Library/LaunchAgents, and other locations specified in the launchd man page. Examples of these launch agents: the dock and software for your Wacom tablet.

launchctl examples

Listing present daemons

The stuff below is equivalent to listing the contents of the /etc/init.d directory on Linux.

To see what system daemons are present on the system;

 $ cd /System/Library/LaunchDaemons
 $ ls

Note that this does not tell you whether they've actually been configured to start, or not!

Overview of running daemons

The stuff below is equivalent to listing the contents of the /etc/rcX.d directories on Linux, or using the "chkconfig --list" command on RedHat.

Since launchd is more than just managing daemons, but also a replacement for init, it can show us some bookkeeping on previously run daemons. Use launchctl its 'list' parameter to list the currently running daemons, including the daemons that have already exited (i.e. configured for a one-time run).

To list all daemons:

 $ launchctl list

To show daemons, but filter out those that aren't currently running:

 $ launchctl list | grep -v "^-"

Controlling daemons

The stuff below is equivalent to using the scripts in /etc/init.d on Linux.

To temporarily stop a daemon, you use the unload parameter plus the config filename. For instance to stop the daemon that controls swapping memory to disk:

 $ sudo launchctl unload /System/Library/LaunchDaemons/

If it says "nothing to unload", then the daemon wasn't configured to be started in the first place.

To temporarily start a daemon, use the load parameter:

 $ sudo launchctl unload /System/Library/LaunchDaemons/

Adjusting the boot process

The stuff below is equivalent to using the chkconfig or update-rc.d commands in Linux.

To start or stop a daemon, and make sure it'll be started the next boot, use the -w flag. For example: to start the dynamic pager now, and after rebooting:

 $ sudo launchctl load -w /System/Library/LaunchDaemons/

If the daemon wasn't configured to start before, you might want to pass the -F flag, which forces starting.

Configuring a daemon

The stuff below is equivalent to editing a script in /etc/init.d, or a configuration file in /etc under linux.

To configure the start/stop flags of a daemon, you edit the plist file with the defaults command. To read the current configuration:

 $ defaults read /System/Library/LaunchDaemons/

To change a config file, simply use the defaults 'write' parameter. As an example, to disable Bonjour advertising on the network:

 $ sudo defaults write /System/Library/LaunchDaemons/ \
     ProgramArguments -array-add "-NoMulticastAdvertisements"

Note that whenever software is updated, these changes get written over! OS X does not have a package system to my knowledge, which will save any adjusted configuration files.


Some notes:

  • Launchd doesn't support dependencies, but it's possible to create special configuration files in /System/Library/LaunchDaemons which will start a daemon whenever a client tries accessing a particular network port.
  • Both launchd and launchctl have a man page, and they're okay (not super, though).
  • For more information on how to write the plist files, see also the man page of "launchd.plist"
  • Should you want to periodically run a script, check out the tips here:

2012-02-09 Writing ethernet packets on OS X and BSD

If you want to write raw ethernet frames, you can use so-called raw sockets. Unfortunately, there's a difference between raw sockets under Linux and raw sockets under BSD and derivatives such as Apple's OS X. Linux allows you to create custom ethernet frames with whatever content you want, but BSD assumes you will create IP (internet protocol) packets.

In my case, I wanted to write raw ethernet frames, as in: have complete control of the contents. The content didn't have anything to do with internet; at work we use ethernet to talk to custom electronics. The electronics doesn't contain enough processing power to have a full TCP/IP stack, when ethernet is perfectly fine.

Note: I've tested the code below on OS X, version 10.7 (Lion). I've checked the output using tcpdump on the other end (a Linux box). Most code is from Bastian Rieck's excellent piece "Using FreeBSD's BPF device with C/C++". The only real addition is the write_frames routine that chops up your data before sending it.

    #include <sys/types.h>
    #include <sys/uio.h>
    #include <unistd.h>
    #include <stdlib.h>
    #include <stdio.h>
    #include <fcntl.h>
    #include <string.h>
    #include <sys/ioctl.h>
    #include <sys/socket.h>
    #include <arpa/inet.h>
    #include <net/if.h>
    #include <net/ethernet.h>
    #include <net/bpf.h>
    // Fill in your source and destination MAC address
    unsigned char dest_mac[ETHER_ADDR_LEN]  = {0x00, 0x1b, 0x21, 0x53, 0x83, 0x4f};
    unsigned char src_mac[ETHER_ADDR_LEN] = {0xc8, 0xbc, 0xc8, 0x91, 0x79, 0x2d};
    // My struct for an ethernet frame. There are many like it, but this one is
    // mine.
    struct frame_t {
        struct ether_header header;
        unsigned char payload[ETHER_MAX_LEN - ETHER_HDR_LEN];
        ssize_t len;
        ssize_t payload_len;
    // Some convenience constants
    // Try to open the bpf device
    int open_dev(void)
        char buf[ 11 ] = { 0 };
        int bpf = 0;
        int i = 0;
        for(i = 0; i < 99; i++ )
            sprintf( buf, "/dev/bpf%i", i );
            bpf = open( buf, O_RDWR );
            if( bpf != -1 ) {
                printf("Opened device /dev/bpf%i\n", i);
        if(bpf == -1) {
            printf("Cannot open any /dev/bpf* device, exiting\n");
        return bpf;
    // Associate bpf device with a physical ethernet interface
    void assoc_dev(int bpf, char* interface)
        struct ifreq bound_if;
        strcpy(bound_if.ifr_name, interface);
        if(ioctl( bpf, BIOCSETIF, &bound_if ) > 0) {
            printf("Cannot bind bpf device to physical device %s, exiting\n", interface);
        printf("Bound bpf device to physical device %s\n", interface);
    // Set some options on the bpf device, then get the length of the kernel buffer
    int get_buf_len(int bpf)
        int buf_len = 1;
        // activate immediate mode (therefore, buf_len is initially set to "1")
        if( ioctl( bpf, BIOCIMMEDIATE, &buf_len ) == -1 ) {
            printf("Cannot set IMMEDIATE mode of bpf device\n");
        // request buffer length
        if( ioctl( bpf, BIOCGBLEN, &buf_len ) == -1 ) {
            printf("Cannot get bufferlength of bpf device\n");
        printf("Buffer length of bpf device: %d\n", buf_len);
        return buf_len;
    // Read one or more frames
    void read_frames(int bpf, int buf_len)
        int read_bytes = 0;
        struct frame_t *frame;
        struct bpf_hdr *bpf_buf = (struct bpf_hdr*) malloc(buf_len);
        struct bpf_hdr *bpf_packet;
        int run_loop = 1;
        int i = 0;
        printf("Start reading frames\n");
        while(run_loop) {
            memset(bpf_buf, 0, buf_len);
            if((read_bytes = read(bpf, bpf_buf, buf_len)) > 0) {
                printf("Read %d\n", i);
                // read all packets that are included in bpf_buf. BPF_WORDALIGN is used
                // to proceed to the next BPF packet that is available in the buffer.
                char* ptr = (char*)bpf_buf;
                while(ptr < ((char*)(bpf_buf) + read_bytes)) {
                    bpf_packet = (struct bpf_hdr*)ptr;
                    frame = (struct frame_t*)((char*) bpf_packet + bpf_packet->bh_hdrlen);
                    frame->len = bpf_packet->bh_caplen;
                    frame->payload_len = frame->len - (2*ETHER_ADDR_LEN) - ETHER_TYPE_LEN;
                    // Do something with the frame
                    printf("Got packet, length of frame: %ld, length of data: %ld\n",
                        frame->len, frame->payload_len);
                    ptr += BPF_WORDALIGN(bpf_packet->bh_hdrlen + bpf_packet->bh_caplen);
            } else {
                perror("Meh, couldn't read from bpf device");
    // Read a single frame
    void read_single_frame(int bpf, int buf_len)
        int read_bytes = 0;
        int i;
        struct bpf_hdr* bpf_buf = malloc(buf_len);
        memset(bpf_buf, 0, buf_len);
        char *ptr;
        printf("Headerlength: %ld\n", sizeof(bpf_buf));
        read_bytes = read(bpf, bpf_buf, buf_len);
        if(read_bytes > 0) {
            printf("Got %d bytes\n", read_bytes);
        } else {
            printf("Got 0 bytes\n");
        ptr = (char*)bpf_buf;
        for(i = 0; i < read_bytes; i++) {
            unsigned char byte = (unsigned char) *(ptr + i);
            printf("0x%02X ", byte);
    // Write a single ethernet frame with test data
    void write_single_frame(int bpf)
        ssize_t data_length = 0x4F;
        struct frame_t frame;
        memcpy(frame.header.ether_dhost, dest_mac, ETHER_HDR_LEN);
        memcpy(frame.header.ether_shost, src_mac, ETHER_HDR_LEN);
        frame.header.ether_type = 0x00;
        frame.len = (2*ETHER_ADDR_LEN) + ETHER_TYPE_LEN + data_length;
        // Fill frame with ramp
        unsigned char j;
        for (j = 0; j < data_length; j++) {
            frame.payload[j] = j;
        ssize_t bytes_sent;
        bytes_sent = write(bpf, &frame, frame.len);
        if(bytes_sent > 0) {
            printf("Bytes sent: %ld\n", bytes_sent);
        } else {
            perror("Whoops! Does the device actually have an IP address?");
    // Divide data across ethernet frames
    void write_frames (int bpf, const unsigned char *databuf, size_t datalen)
        size_t start = 0;
        struct frame_t *frame = malloc(ETHER_MAX_LEN);
        size_t bytes_to_send;
        ssize_t bytes_sent;
        memcpy(frame->header.ether_dhost, dest_mac, ETHER_HDR_LEN);
        memcpy(frame->header.ether_shost, src_mac, ETHER_HDR_LEN);
        frame->header.ether_type = 0x0000;
        do {
            // Clear frame
            bzero((void*)(frame+ETHER_PAYLOAD_START), ETHER_PAYLOAD_LEN);
            // Calculate remainder
            if((datalen - start) < ETHER_PAYLOAD_LEN) {
                bytes_to_send = datalen - start;
            } else {
                bytes_to_send = ETHER_PAYLOAD_LEN;
            // Fill frame payload
            printf("Copying payload from %lu, length %lu\n", start, bytes_to_send);
            memcpy(frame->payload, (void*)(databuf + start), bytes_to_send);
            frame->len = ETHER_HDR_LEN + bytes_to_send;
            // Note we don't add the four-byte CRC, the OS does this for us.
            // Neither do we fill packets with zeroes when the frame length is
            // below the minimum Ethernet frame length, the OS will do the
            // padding.
            printf("Total frame length: %lu of maximum ethernet frame length %d\n",
                frame->len, ETHER_MAX_LEN - ETHER_CRC_LEN);
            bytes_sent = write(bpf, frame, frame->len);
            // Check results
            if(bytes_sent < 0 ) {
                perror("Error, perhaps device doesn't have IP address assigned?");
            } else if(bytes_sent != frame->len) {
                printf("Error, only sent %ld bytes of %lu\n", bytes_sent, bytes_to_send);
            } else {
                printf("Sending frame OK\n");
            start += bytes_to_send;
        } while (start < datalen);
    // Create a simple ramp so we can check the splitting of data across frames on
    // the other side (using tcpdump or somesuch)
    unsigned char* make_testdata(int len)
        unsigned char *testdata = (unsigned char*)malloc(len);
        int i;
        unsigned char j = 0;
        for(i = 0; i < len; i++) {
            testdata[i] = j;
            if(j < sizeof(char)) {
                j = 0;
        return testdata;
    int main(void)
        char* interface = "en0";
        unsigned char* testdata;
        size_t testdata_len = 4530;
        int bpf;
        int buf_len;
        bpf = open_dev();
        assoc_dev(bpf, interface);
        buf_len = get_buf_len(bpf);
        //read_single_frame(bpf, buf_len);
        //read_frames(bpf, buf_len);
        testdata = make_testdata(testdata_len);
        write_frames(bpf, testdata, testdata_len);

2012-01-26 TransIP VPS

Since last year, TransIP offers virtual private servers (VPSes). I have been playing with the M version for the past week. This particular VPS has 512 MB of RAM, 10 GB of harddisk space and one CPU core, and I ran UnixBench on it. Here are the results:

 System Benchmarks Index Values               BASELINE       RESULT    INDEX
 Dhrystone 2 using register variables         116700.0   24917048.3   2135.1 
 Double-Precision Whetstone                       55.0       3413.4    620.6
 Execl Throughput                                 43.0       4520.4   1051.3
 File Copy 1024 bufsize 2000 maxblocks          3960.0     882288.4   2228.0
 File Copy 256 bufsize 500 maxblocks            1655.0     260223.5   1572.3
 File Copy 4096 bufsize 8000 maxblocks          5800.0    1378435.4   2376.6
 Pipe Throughput                               12440.0    2369954.3   1905.1 
 Pipe-based Context Switching                   4000.0     387771.4    969.4
 Process Creation                                126.0      14882.1   1181.1
 Shell Scripts (1 concurrent)                     42.4       6816.5   1607.7
 Shell Scripts (8 concurrent)                      6.0        898.3   1497.2
 System Call Overhead                          15000.0    4303337.7   2868.9
 System Benchmarks Index Score                                        1537.7

Edit 29/03/2012: their offer has gotten better, with more disk space.

2012-01-23 Logrotate error for MySQL on Debian 6 Squeeze

When installing a new machine with Debian 6 (Squeeze), I also imported a database from a CentOS 5 box. The next day, the following error appeared in my inbox:

 error: error running shared postrotate script for '/var/log/mysql.log
      /var/log/mysql/mysql.log /var/log/mysql/mysql-slow.log '
 run-parts: /etc/cron.daily/logrotate exited with return code 1

Turns out that Debian needs a user called 'debian-sys-maint' and that this user needs a number of rights on the database. The username and password are configured in /etc/defaults/debian.cnf so write these down, then create the user with the following commands:

 $ mysql -u root -p
 mysql> GRANT SHUTDOWN ON *.* TO 'debian-sys-maint'@'localhost';
 mysql> GRANT RELOAD ON *.* TO 'debian-sys-maint'@'localhost';
 mysql> GRANT SELECT ON 'mysql'.'user' TO 'debian-sys-maint'@'localhost';

and then set the correct password for the debian user:

 mysql> SET PASSWORD FOR 'debian-sys-maint'@'localhost' = PASSWORD('secret');

Test whether logrotate won't give any problems:

 $ sudo logrotate -f /etc/logrotate.d/mysql-server

2011-12-30 Arduino and beer brewing

This space is reserved!
I should really update this...

2011-12-12 Protocol stuff

We have an interesting situation here at work, concerning our internally developed binary protocol.

There are two libraries to parse the protocol. These have different ways to calculate the length of a block of data. And now I want to know which way is the correct. Of course, that's debatable, with a history of hardware that uses the protocol for different purposes.

In the current firmware/software combination, the firmware generates a large number of packets that each fit into the Ethernet maximum packet length (i.e. 1500 bytes). Considering the packet length is 1500 bytes, and our data consists of 2-byte samples, we get 730 samples in a packet. The rest is overhead, namely: 6 bytes primary header, 9 bytes secondary header and 24 bytes data header, plus an end-of-packet byte containing the value 0xA5.

A number of these packets are received by an intermediate piece of software, called the Ethernet Daemon, and then concatenated into a larger packet, where the Data Header/Data part are repeated.


The discussion piece currently, is how to calculate the length of the block of data. You need to know this in order to find the next Data Header. The following fields in the data header can help:

  • samples64; the number of samples per 64 bit; must be a power of 2
  • sample_count: the number of samples
  • data_bits: number of bits that's used in the data block
  • encoding: the encoding of the data block; two complement, unsigned integer, double or zero-terminated string

Basically samples64 says something about how much samples are contained in 64 bits (eight bytes), and this can be represented graphically in the way below:

CCSDSdata bits.png

The above fields are pretty much explained now, except that the data_bits explanation is rather terse. The thing is, we software engineers round off everything to one byte. But in the real world, that's nonsense. For example, measuring a voltage will result in, say, 12 bits. For a 12-bits sample, we set the data_bits field to 12. Rounding off to bytes, this means that each sample will occupy two bytes. Thus, the samples64 field will be set to "4", meaning: four samples will fit in 64 bits.

The developer of our Python library said that the samples64 field implies, that the length of the Data field is always divisible by 8. Or in other words, that the Data field is created using 8-byte pieces. After discussion, it turns out that's not what the firmware does, in our projects.

His calculation of the length of the Data field was:

  Data field length (in bytes) = 8 * ( (sample_count + samples64 - 1) // samples64 )

(The double slash means: throw away the remainder)

The firmware would send a packet with a number of Data Header/Data fields. The first Data Header says contains 31390 samples, each 16-bits. The second Data Header contains 8760 samples. The above calculation would yield for the first Data Header:

  Data field length (in bytes) = 8 * ( (31390 + 4 - 1) // 4 )
  Data field length (in bytes) = 8 * 7848
  Data field length (in bytes) = 62784

And for the second Data Header:

  Data field length (in bytes) = 8 * ( (8760 + 4 - 1) // 4 )
  Data field length (in bytes) = 8 * 2190
  Data field length (in bytes) = 17520

The Perl library does things differently; its way of calculating the length of the Data field is:

  Data field length (in bytes) = sample_count * (data_bits / 8)

Thus for the first and second Data Headers, this looks as follows:

  Data field length (in bytes) = 31390 * (16 / 8)
  Data field length (in bytes) = 62780
  Data field length (in bytes) = 8760 * (16 / 8)
  Data field length (in bytes) = 17520

For this to work, the data_bits has to be rounded off to the nearest power of two. We don't have firmware that does otherwise, but in theory it's possible. This implies that the following combinations should be expected:

samples64 field data_bits field
1Between 64 and 33
2Between 32 and 17
4Between 16 and 9
8Between 8 and 5
16Either 4 or 3

We assume here, that the firmware packs the data as efficient as possible, of course. But that's not a guarantee. So the way to calculate the byte length of the Data field, should not include data_bits. It should be as simple as sample count times sample size (in bytes).

The sample size basically comes from the samples64 field:

samples/64 field sample size in bytes

Of course, now we're pushing the boundaries of other libraries and hardware. Computer and networking equipment dictates that the minimum transfer unit is the byte. Thus when the samples64 field is higher than 8, software must start ignoring the remainder of the Data field.

  Data field length (in bytes) = sample_count * sample_size_in_bytes
  Data field length (in bytes) = sample_count * ( 8 / samples64 )

Because we want to make sure that we're getting whole bytes, the result should be rounded up.

2011-10-27 Compact Google Calendar header


I have created an official extension, available in the Chrome Web Store:

Compact Calendar extension

Original post

The recent changes to Google Calendar are looking nice, but unfortunately the header and navigation take up far too much space when using my 13" MacBook. For a compact Google Calendar header in your Chrome browser, take the following steps:

  • Install the Live CSS Editor extension for Chrome
  • Go to the options, set them all to "Yes" and click Save
  • Go to your Google Calendar
  • Click the new CSS button that the extension provides
  • Add the following code
 #vr-nav { display:none; }
 #onegoogbar { display:none;}
 #vr-header { display:none; }


2011-10-11 OS X and Vim

Vim comes with OS X by default, but it doesn't come with X11 support. You can always install MacVim of course, but some of us prefer to use vim in the terminal. Unfortunately, because of the lack of X11 support, copying/pasting doesn't work correctly in OS X.

As an example, copy a paragraph of text from Safari, then paste it into your vim instance in the terminal. Chances are, if options such as AutoIndent are on, the paragraph will be rearranged.

On Linux, we have the + and * buffers for that. That way, a simple "+p command will paste the most recently copied text into the terminal version of vim.

To enable this on OS X, I added a hint to my Vim page.

2011-10-05 From iPhone to Android

This is part 8 of my series where I document going from an iPhone to Android 2.3, running on the Samsung Galaxy Gio.

Syncing, it turns out, is a royal pain in the ass with the combination Android and OS X. There's a whole bunch of sub-optimal and incomplete solutions out there. And that's assuming that your mail, calendar, contacts and notes/tasks are synced via your Google account.

Product Pros Cons
doubleTwistFree version available. Syncs music, video's and photos, wireless in the paid version Does not transcode music and resize photos to appropriate size for device. Does not sync (only download) photos.
iTuneMyWalkmanOpen source. Syncs music via iTunes, via a specially-named playlist. iTunes does transcoding if you care for that. Moves pictures to your harddisk, but doesn't sync them.Doesn't sync anything else than music (photos are copied). Thinks thumbnails are also pictures. Doesn't transfer cover art. Transferring takes a long time, a couple of Gb of music causes a twenty-minute "collecting data from iTunes" message.
PicasaSyncs photos wirelessly Forces you to use Picasa web albums

As far as I know, there is no one-stop solution on OS X, unless you use Samsung Kies on OS X, which only supports a limited number of Samsung phones.

2011-09-21 From iPhone to Android

This is part 7 of my series where I document going from an iPhone to Android 2.3, running on the Samsung Galaxy Gio.

From version 2.2, Android offers the HotSpot functionality. There's another way, namely using the phone as a 3G modem via USB. This is called tethering, and OS X supports this out-of-the-box, just follow the instructions on TodayOnTech.

Another thing I bumped into, is the lack of audiobook support. doubleTwist doesn't really sync this properly. So, you copy them manually, which is not really a big deal for me. However the default media player will list these as music, which it most definitely isn't. The Android blog has a solution for this, creating a zero-length .nomedia file in your SD-card its /Audiobooks directory. Clumsy, in my opinion. The mediaplayer could look at the ID3-tags of the MP3 files, and see that they're marked as 'spoken word' or 'audio book'.

Another missing thing of the stock music player is the lack of control from the lock screen. From a locked iPhone, you can doubleclick the home button and the player buttons will pop up if music is playing. There are a number of Android apps that simulate this, for instance bTunes, mixZing and Lithiumplayer.

Perhaps a solution is to use iTuneMyWalkman, if doubleTwist continues to bug me, then I'll try this solution.


On the forums, there are powerusers who manage to fill up their phone with apps. Apparently, the Gio isn't meant for those people. If you're one of those, you want to be rooting your Samsung Gio. That way, you can delete some of Samsung's stock apps. It's a process similar to jailbreaking your iPhone, but since the Android phone manufacturers aren't as tight-assed, it's not really as invasive and/or complicated as the typical iPhone jailbreaking. See also the xda-developers forum for instructions on jailbreaking the Samsung Galaxy Gio.

2011-09-19 From iPhone to Android

This is part 6 of my series where I document going from an iPhone to Android 2.3, running on the Samsung Galaxy Gio.

Android and iOS both have home screens, but they really have a different purpose. In iOS, the home screen contains all apps. In Android, your apps are stored in the app drawer (usually the icon in the lower right corner). The Android home screen is much more flexible. It can contain a number of items, of which the most important are: widgets and shortcuts. Widgets are little apps that run on your home screen, showing the date/time, the weather or what have you. A shortcut is just a way of starting up an app via the home screen.

Thus on Android, you could easily have an empty home screen, if you don't mind starting apps with two taps (first go to the app drawer, then tap the app itself).

At first, the Android home screen is a bit confusing since there are so many possibilities compared to iOS. But there are many cool things here, such as the display of upcoming events in your calendar, right on the home screen:

2011-09-15 From iPhone to Android

This is part 5 of my series where I document going from an iPhone to Android 2.3, running on the Samsung Galaxy Gio.

I have some problems with the stock music player. First, it'll show all my audiobooks right in between my music. This makes shuffle play worthless, and clutters the list of albums.

Another thing is skipping/pausing. Sometimes it happens that you want to skip a currently playing song when you're on the home screen or in some other app. The iPhone offers a convenient popup that offers basic player controls when you double-click the home button. I haven't found the equivalent under Android. It's certainly not built in, at least, if you don't count Android multitasking: hold down the home button and a popup appears with all running programs. You can now tap the music player and control it.

2011-09-13 From iPhone to Android

This is part 4 of my series where I document going from an iPhone to Android 2.3, running on the Samsung Galaxy Gio.

Another thing I found out; making screenshots is not something that's universal on all Android phones. It totally depends on the specific phone maker and model. My Gio supports it; holding the back button while pressing the home button will do the trick. For lots of other Android phones, there's no such thing.

Captive bugs

A particularly interesting thing about Android is, that there's an alternative in the Android Market for lots of standard apps. For texting and the calendar app, there are replacements.

That's necessary sometimes, because my Samsung Gio has the Captive "sauce" over the stock Android. There's something weird with the calendar app, where if you add an event, it's added by default to the local calendar. In other words, if you set up the phone to sync to your Google calendar, and you add an event, it doesn't show up on Google. How weird is that? Turns out this is an age-old issue, so after creating the event, you'll have to go back and change the calendar.


A couple of days ago, I got a silicone case for the Gio, but it's not really to my liking so I ordered a hard case on eBay via the Totomagirl seller. This was surprisingly difficult to find. When iPhone cases are all over the place, some Android phones like the Gio clearly aren't that attractive to makers of accessories.

As opposed to the iPhone, it's really easy to swap out the battery of the Samsung Gio. So I also ordered a battery on DealExtreme (the Galaxy Ace and the Gio use the same batteries). The iPhone with its external batteries (or expensive add-on packs) is less than ideal in this regard.

2011-09-08 From iPhone to Android

This is part 3 of my series where I document going from an iPhone to Android 2.3, running on the Samsung Galaxy Gio.

Previous time, I charged the phone with a cheap charger in my car and at home, which resulted in touchpad unresponsiveness, only to be fixed with a reboot. I suspect the chargers and started using only the Samsung-supplied charger and my PC for juice. We'll see.

I've also started using doubleTwist (yeah, lower "d") for syncing Music from my MacBook to my Samsung Gio. It works fine, but I'm missing some iTunes features. A really useful feature for iPhones and iPods with less than 16 Gb is the option "Convert higher bitrate songs to 128 kbps AAC". doubleTwist currently doesn't support downconverting (also called transcoding) music to a lower bitrate.

As for apps, I haven't really missed anything between my iPhone and my Samsung Gio. All the biggies are here; Facebook, Kindle, Skype, an RSS reader which syncs with Google Reader, a number of SimpleNote clients; all there. There's a couple of extra apps that aren't available on iPhone such as the official Google Reader App.

What's nice as well is that Android apps seem to be able to control the phone, more so than on the iPhone platform. For instance, there are apps that can control the screen brightness with a very wide range. Useful stuff.

The Android Market for apps seems to attract developers who aren't in it for the money. A decent SSH client (tool for IT system administrators) for example is ConnectBot, open source and completely free. The equivalent in the iOS AppStore is TouchTerm, which costs $4. Other app markets for Android seem to be popping up as well; for instance there's the Amazon app store (only available for US residents, unfortunately).

2011-09-06 From iPhone to Android

This is part 2 of my series where I document going from an iPhone to Android 2.3, running on the Samsung Galaxy Gio.

Damnit. When I wake up this morning, something is wrong with the touchscreen. It seems to register touches, but the phone thinks you touched the screen somewhere else. I can't even turn off the phone, because although the power button is a real button at the side of the phone, it asks for confirmation on the screen. After a number of tries, I get it to power off.

That's not good, Samsung. The second day, I have to restart the Gio.

When I cycle to work, I start the media player and realize that there is no album art when you manually sync music to your phone. Which you are forced to do, because the OS X version of Kies doesn't support the Gio. Lifehacker tells me to take a look at DoubleTwist, which I may do later.

One of the weird things about Android is, that you have the home screen as well as the application screen. The home screen contains selected shortcuts to apps, if I can say so, while the application screen contains all apps. Why not just limit it to one screen? There must be some reason, but I haven't found it yet.

The Gio is a real nice, light phone. My iPhone 3G weighs 133 g (4.7 oz), and the Samsung Gio only weighs 102 g (3.6 oz). It's also 1.5mm thinner. Since I remembered dropping my iPhone in the first week after purchase, I ordered a cheap silicone casing on eBay store Cellapod.

Just before dinner, the phone warned me to recharge it, although I hadn't heavily used it. That's okay though, my iPhone 3G is very power hungry as well.

I did a little testdrive to see how the navigation is holding up (quite nicely) and charged the phone using my cheap USB car charger that I used for my iPhone as well. When I came home, the phone wouldn't react to screen touches, and when clicking the home button, a screenshot would be made.

Because the unresponsiveness this morning as well as this evening happened right after charging, I've come to the conclusion that this phone can't handle cheap chargers. I'm going to do some more testing, though.

2011-09-05 From iPhone to Android

Today, my iPhone 3G lost its reception in the middle of the city, and that wasn't the first time. Since this was a good time as any to experiment with Android, I figured I'd see what Google's ecosystem is doing, and I'm going to relate my experiences from my point of view: staunch GMail user, a heavy smartphone user, and one who is used to the Apple way of doing things. That means: a MacBook, iTunes and an iPhone.

Since I was limited on budget but still wanted the equivalent in specifications of an iPhone 3G, I went for the Samsung Galaxy Gio S5660:

Wikipedia has all the specs of this phone, but the gist is: an Android phone with a reasonable CPU, a nice 320×480 pixel display, a 3 megapixel cam and enough onboard memory as not to cause any trouble.

The Gio actually has a pretty bad history, as Koen Delvaux diligently blogs. Crashes related to WiFi usage, battery draining, et cetera. But since Samsung rolled out the new Android 2.3 version (nicknamed Gingerbread), these bugs are supposedly fixed.

The phone was advertised as having Android version 2.3, but when I asked the sales guy, it turns out it's still at 2.2 and you have to update it yourself.

Coming home, I discovered that updating the Gio is done through Samsung Kies, a program which runs on both OS X and Windows. Except the OS X version only supports four or five of Samsung's phones. Luckily, I have Parallels running on my Mac, so I can run Kies on a virtualized Windows. For the sake of Google, I repeat: if you have a Mac, you need Parallels in order to run Samsung Kies, because the version for OS X does not support the Galaxy Gio. Samsung Kies has a number of documented problems, for which MobileCowboys thoroughly provides workarounds. But perhaps I was lucky, because installing Kies and updating the phone to Android 2.3 worked fine.

Of course, this will mean a bit of a hassle in the future. Putting music on my iPhone is basically connecting the iPhone, starting iTunes on my Mac, then hit sync. Now I'll have to copy music manually to the phone.

After updating, I configured the phone with my GMail account and installed a bunch of apps (Kindle, Facebook, Skype, etc) that I already used on my iPhone. The configuration of GMail is flawless, even though I use a Google Apps account (I have had my own domain for years now, and mail is handled by GMail).

The iPod application of the iPhone is nice, and supports a sleep timer. I often fall asleep while listening to audiobooks. You need a separate app for this, and I was advised to take a look at MortPlayer. I haven't tried it yet.

2011-08-30 Creating specific password lists with John the Ripper

For security purposes, you sometimes want a dictionary with possible passwords. Perhaps you want to assess password strength in an in-house built application, or you want to test the security of the WPA wireless network.

There are plenty of dictionaries around to use to warn (or exclude) users from creating an account with such a password, or for the latter purpose, to brute force the WPA key.

After the standard dictionaries, a good next step is to create a password list for specific circumstances. For example; you are building an app for a company and want to make sure that easy passwords are excluded. You then need to build a list of passwords that any cracker would try.

There are good commercial products that can do this kind of stuff automatically, but for the poor man, it's easy enough to just get the stuff from the company website, or the Wikipedia article on them. With standard Unix/Linux tools, you're good to go.

I assume we create a directory in your home directory called "crack". First, use wget to get the source of words that shouldn't be a password:

 $ mkdir ~/crack
 $ cd ~/crack
 $ wget "" -O wordlist.html

Then use sed to strip out the HTML:

 $ cat wordlist.html | sed -e :a -e 's/<[^>]*>//g;/</N;//ba' > wordlist.txt

Then edit the list with vi, and manually remove some JavaScript cruft at the bottom if necessary, then clean it up a bit.

First put each word on a separate line. The ^M indicates the Enter key. Create it by pressing CTRL+V, then pressing enter.

 :%s/ */^M/g

Remove extra whitespace:

 :%s/ *//g

Remove empty lines:


Done. Now we're going to expand the wordlist with all sorts of word play that people might think of, like using years, or numbers, or using leet speak. John the Ripper by default has a nice set of mangling rules, but it's very limited.

Matt Weir has greatly enhanced the default set of mangling rules, and that's the one we'll be using.

Make sure you have a recent version of John the Ripper, because Matt's rules need it. Your Linux distribution will probably have an older version, so download and compile version 1.7.8 or something later. I'm assuming you'll put John in your home directory in opt, then add it to your path:

 $ mkdir ~/opt
 $ cd ~/opt
 $ wget
 $ tar xfz john-1.7.8.tar.gz
 $ cd john-1.7.8/src
 $ make

(make now lists a number of targets, choose the best for your architecture and re-run make)

 $ make linux-x86-64

When finished, the executable and configuration file reside in $HOME/opt/john-1.7.8/run. Download Matt Weir's configuration file to this directory, but back up the original configuration first:

 $ cd ~/opt/john-1.7.8/run
 $ mv john.conf john.conf.bak
 $ wget -O john.conf

Since John the Ripper is used by us in WordList mode, we need to edit the configuration file so it uses the correct rules. Edit john.conf, and search for the line


(around line 335) and disable it by adding a few characters, for example


Then find the following line:


(around line 21), and change it to:


Now add John the Ripper to your path, by adding the following line to your ~/.bashrc:

 export PATH=$HOME/opt/john-1.7.8/run

Logout, and login again. Typing 'john' should print the version. Now mangle your newly tested word list. To make sure the new mangling rules are used, create a file called test.txt with only one line, for example 'rotterdam'. Then run john. It should look something like this:

 $ cd ~/crack
 $ vi test.txt
 $ john -w=test.txt --stdout --rules

(skipping a whole lot of lines)

 words: 4745  time: 0:00:00:00 100%  w/s: 39541  current: Eirrwesan

As you can see, Matt's rules generate all sorts of permutations which you can then use in your software or brute forcing efforts. From one word, 4745 mutations have been created! If you see less (for example between 20 and 40), then the default rules have been used and there is something wrong with the configuration file. Perhaps john couldn't find it?

If this went okay, then re-run john on your custom wordlist:

 $ john -w=wordlist.txt --stdout --rules > wordlist_mangled.txt

Voilà, this resulting word list can now be used in your password strength assessment, or brute forcing efforts.

2011-08-18 Google Chrome on Debian Lenny

The latest packages of Google Chrome won't install anymore on Debian 5.0, nickname Lenny. This is the old stable release of Debian.

For the time being, I'm still using Lenny, so for everybody else, here is a link to the most recent release that still installed on Lenny:

Google Chrome

Note that this is somewhat of a security risk!

2011-06-20 Running yum on low end VPSes

Yum is a dog to run on older systems with small amounts of memory, or when you run a VPS (virtual private server) that does not have much memory. When running a yum update, you get errors such as memory alloc (8 bytes) returned NULL. Recommendations are:

  • Running yum clean all
  • Disabling plugins by using the --noplugins parameter

But ofthen that's not enough. A simple yum update can find a huge number of packages to update and nothing will help you getting the required amount of free memory. You can work around this problem by updating them one-by-one.

First generate a list of packages-to-be-updated, then dump them in a file.

 $ yum -q check-update | cut -f1 -d" " > list

Now use a loop that runs yum for each package listed in the file:

 $ for i in $(cat list); do sudo yum -y update "$i"; done

2011-06-17 Encrypting and decrypting files part 2

Some time ago, I gave a duo of one-liners that encrypts/decrypts files.

This wasn't good enough for normal usage, so here's a better version that takes care of directories as well by running tar on them.

  if [ -d "$source" ]; then
  if [ -e "$newfilename" ]; then
      echo "ERROR: A filename with the name $newfilename already exists"
      exit 1
  if [ -d "$source" ]; then
      # Source is a directory, first tar it
      if [ -e "$source".tar ]; then
          echo "ERROR: A filename with the name $source.tar already exists"
          exit 1
      tar cf "$source".tar "$source"
      openssl des3 -salt -in "$source".tar -out "$newfilename"
      rm -f "$source".tar
      openssl des3 -salt -in "$source" -out "$newfilename"

And its evil twin, decrypt:

  origfilename=$(basename "$1" .encrypted)
  if [ "$origfilename" == "$1" ]; then
      echo "ERROR: Encrypted files must have the .encrypted extension"
      exit 1
  if [ -e "$origfilename" ]; then
      echo "ERROR: A filename with the name $origfilename already exists"
      exit 1
  openssl des3 -d -salt -in "$1" -out "$origfilename"
  # Check if the output is tarred
  origfilename=$(basename "$origfilename" .tar)
  if [ "$origfilename" != "$tarfilename" ]; then
      # It was tarred
      if [ -e "$origfilename" ]; then
          echo "ERROR: A filename with the name $origfilename already exists"
          exit 1
      tar xf "$tarfilename"
      rm "$tarfilename"

Save them in your path somewhere and make them executable.

2011-05-19 App Store on OS X a disappointment so far

So far, the OS X AppStore couldn't be called 'wildly popular' since its inception on January this year. Regularly, I checked my installed apps for availability in the App Store, because it allows for such easy updating. Lo and behold, only fairly trivial apps are there, the following list is not available in the App Store:

  • Google software (Chrome, Sketchup)
  • Mozilla software (Firefox, Thunderbird)
  • Adobe software (Flash, Flash Builder CS5, Photoshop etc)
  • Microsoft software (Office, Messenger, Silverlight etc.)
  • OpenOffice
  • Microsoft Messenger
  • Microsoft Silverlight
  • Seashore (painting program)
  • Parallels
  • VLC
  • Skype
  • Calibre (an eBook converter)
  • XBench (a benchmark for OS X)
  • Vuze
  • KisMAC

Now I agree that stuff like a bittorrent client (Vuze) and a network sniffing tool (KisMAC) would probably be refused in the App Store. But all in all, the OS X App Store could be called a disappointment so far.

Note that the Opera browser (which contains a bittorrent client) is in the App Store.

2011-04-21 Not enough CPU time

The datarate of the Demux board of the Safari project is quite high for a desktop PC to work with, even considering the fact that the raw data is being thoroughly filtered before it reaches the PC running the Generic EGSE (the central part of the software, which allows the user to control electronics and plot output).

We've made some tweaks in the past to allow more breathing space, such as good optical GBit network adapters (we dumped cheap DLinks and bought some expensive Intel ones). A separate PC was installed to run a daemon, which concatenates the large amount of small ethernet packets into a couple of big CCSDS packets. We tweaked the kernel TCP buffers and the driver settings on both PCs.

The running problem with the EGSE was unresponsiveness. When retrieving packets and plotting them, the whole software stopped reacting to user input. The cause was that the plotting (and its computations) took a lot of time because computations run in the same thread as the plotting, and although the GUI gets time to accept user interaction at regular interval but this interval is too long. A solution is to allow the GUI to accept user interactions more often. We are testing this solution.

(What didn't help, was that several bugs popped up in the meantime. For instance, it's possible to create two plots for one packet type. What would happen, is that packets would get divided between plots, or packets would interrupt the plotting right after the first plot was drawn, but before the second plot was started. A colleague also found and fixed these.)

We brainstormed with the users, and one had experience with this type of situation. After talking, we had a proposal for a situation where the daemon would maintain a very large buffer in memory or on disk, and the EGSE would then lazily retrieve the buffer. Since the daemon runs on a separate PC, memory contention wouldn't be a problem.


We thought about this solution, but although it's a solution for other (older) programming models, we use the Qt library. The big idea of Qt is that it generates signals when an event happens, and you just write code (a "slot") that handles the signal. The Signal/Slot system of Qt provides an elegant and robust solution to manage a buffer which is filled in with packet coming from a daemon and is read lazily by the EGSE. We're testing this solution.

2011-04-20 Telfort ADSL and VoIP Planet

(English readers: this is an explanation on how to configure the modem of a Dutch ISP for VOIP usage).

Tegenwoordig wordt Telfort ADSL uitgeleverd met een Zyxel modem, type P-2601HN-F1. Het is vrij eenvoudig om via een SIP account bij VoIP Planet te bellen met een telefoontoestel dat je aansluit op het modem. Waarom niet gewoon via Telfort Vast Bellen? Omdat je misschien reeds klant van VoIP Planet bent, omdat het goedkoper is, en omdat de service vele malen beter is.

Om het modem te configureren, log in als admin op het modem via adres en klik aan de linkerkant het mapje VoIP open, en klik op SIP. Vul het eerste tabblad als volgt in:

sip voipplanet1.png

Vul het tweede tabblad als volgt in, waarbij je natuurlijk de vermeldde SIP Account Number en Username verandert in diegene die je van VoIP Planet hebt gekregen:

sip voipplanet2.png

Klik op de Apply button en klik aan de linkerkant het mapje Maintenance open, en klik op Logs. Op de eerste regel moet nu staan:

  SIP Registration: SIP:12345: Register Success

Je kunt nu bellen met het aangesloten (analoge) telefoontoestel via VoIP Planet!

2011-04-08 Using the decorator pattern in Python

For my current project, we are trying to look into the future, where our Python scripts might be used on different electronics.

In order for this to work, our Python library for communication with our Generic EGSE software must easily be rewritten. These changes will not be big: communication with an electronics board will take place over a cable of some sorts. That cable and protocol aspect is abstracted away by a daemon, as part of the Generic EGSE.

However, what is not abstracted away, is the contents of the protocol. Our electronics board incorporates an FPGA which takes a certain address (say, address 42) to mean something such as a bias voltage. The electronics board(s) of our project partner might have entirely different addresses, or might require multiple addresses to be set.

Thus, we need the Python scripts to work with a different set of addresses in the future, and we need the flexibility to make exceptions for specific addresses. On the other hand, for now the library needs to work as-is.

The way to typically reach this goal, is to use a decorator pattern. I'm looking into the best way to do this in Python. Since I'm not (yet :-) a guru in the matter of Python, this'll be an interesting experiment.

(Photo by Altera Corporation)

2011-03-28 Configuring VSFTP on CentOS

I got a full FTP-over-SSL (no, not SFTP) running on CentOS, with the following instructions:

  1. VSFTP chroot or jail users – limit users to only their home directory howto and be sure to check the comment on iptables
  2. Configuring VSFTPD for secure connections

Then test using lftp.

2011-03-03 SAFARI software and hardware setup

Below is a basic setup of our software and electronics.

SAFARI setup software and electronics

We have two PCs, one with our Generic EGSE software and the other with a daemon (which collects data). To give you an idea, the Generic EGSE looks as follows:

EGSE Server.png

The daemon just receives data on an optical ethernet port in raw ethernet frames, then concatenates the packages and transfers them over TCP to the EGSE server.

The reason we use a separate PC for the daemon, is that the EGSE server cannot natively receive data over raw ethernet. Thus we use a daemon for that and the data rate is too high to run both daemon and server on one PC.

The demux is short for 'demultiplexer board', an electronics board:


This board retrieves the science data from the sensor and then decimates it. There are two filters available for that. Still, the amount of data is considerable. An ethernet frame can contain 1500 bytes of data. Since one sample of the sensor is two bytes, we put 730 samples in one CCSDS packet so it fits into one ethernet frame. A good sampling should take 100,000 samples. This is rather a lot of packets so you'd say this could easily be fitted into jumbo frames. However, the demux board firmware doesn't support jumbo frames. So we concatenate the CCSDS packets on the PC with the ethernet daemon, then transfer them over TCP to the EGSE Server.

2011-03-03 From which host am I logged in

A previous entry documented on how to change your settings in bashrc, according to from which host you logged in over SSH.

The solution used the hostname entry from the "who"> command. Thing is, it didn't work well because the "who" command also outputs any X sessions.

Here is a script snippet that uses the "host" command. Be sure you install this (under Debian and derivatives available with package dns-utils).

  FROM_IP=$(echo $SSH_CLIENT | cut -f1 -d" " | grep -v ":0")
  FROM=$(host -t ptr $FROM_IP | cut -d" " -f5)
  case $FROM in
      # Enter your settings here
      set -o vi
      # Enter your settings here
      set -o vi
      # Enter your settings here
      set -o vi

The stars behind the hostnames are there to ignore the domain name. This is because FROM=... line will give you the hostname including domain name, for instance, ""

If you want to strip off everything but the hostname, use something like

  FROM=$(host -t ptr $FROM_IP | cut -d" " -f5 | cut -d"." -f1)

2010-12-25 Google Chrome extensions I cannot live without

In the vein of the previous post; a list of Google Chrome extensions that are very useful:

And if you have a Kindle, this extension is excellent as well:

  • Send to Kindle, sends a webpage correctly formatted to the Kindle via the special e-mail address you got from Amazon

Or as an alternative:

2010-12-19 Firefox extensions I can not live without

Everybody has them: Firefox extensions they can't live without. At least, that one percent of the world population that has a PC, and then a tiny percentage of that to care about Firefox extensions.

Without further ado, here's my list:

  • Adblock Plus
  • Flashblock
  • Speed Dial
  • Last tab close button
  • Default Full Zoom Level

2010-12-06 Installing OpenOffice in a home directory

To install OpenOffice in a home directory on Debian, take the following steps:

Download the tarball with .deb packages from

Unpack it in your home directory:

  $ tar xfz OOo_3.2.1_Linux_x86_install-deb_en-US.tar.gz

Change into the directory with the .deb packages:

  $ cd OOO320_m18_native_packed-1_en-US.9502/DEBS

Unpack these to your home directory with:

  $ dpkg -x ~

You'll now have a new subdirectory named 'opt' in your home directory. All executables are in the ~/opt/openoffice.org3/program subdirectory. Add it to your path to easily run OpenOffice, or create custom icons in your Gnome panel (or other favorite desktop environment).

2010-11-26 From which host am I logged in

Sometimes you want to adjust your settings in bashrc, depending from which host you are logging in from. The who command reveals the host, and then we use cut (from GNU coreutils) to get the correct field.

  FROM=$(who | grep `whoami` | cut -f2 -d"(")
  case $FROM in
      # Enter your settings here
      set -o vi

Useful for those shared accounts, who no IT department seems to admit using, but which are mightily useful sometimes!

2010-10-14 Some highlights of AHS 2010

A colleague of mine recently went to AHS 2010, a series of annual conferences organized by NASA, ESA and the University of Edinburough. Topics are on-chip learning, on-the-fly reconfigurable FPGAs, et cetera. This year, the conference took place in Anaheim, California, USA (Sough of LA).

Some points from my colleague's presentation:

  • One keynote displayed the heat in watts dissipated per square centimeter of chip. Currently, low-power multicore chips are being developed for usage in for example mobile phones. To lower power usage, chips are now divided in layers where the top layer can use optics to form a network that transports data between the multiple cores.
  • GSFC is doing a concept study for the Joint Dark Energy Mission, where the challenges are similar to what SRON has encountered during development of the control/readout electronics for the KID detectors, but these detectors work at lower temperatures than what their concept study is using.
  • JPL has developed the iBoard, a digital data acquisition platform for quick prototyping. It's however also flight ready. Based on the Xilinx Virtex 5.
  • William Zheng, also from JPL, gave an overview of the benefits and challenges of wireless intra-spacecraft communications. There are a lot of possibilities, but mentioned was that no roadmap is in place, which shows how a flight development and/or qualitication track looks like.

2010-10-12 Supporting Python

At my dayjob, we have created an application for sensor readout and control. We are creating a software design to support Python for scripting, analysis and plotting, besides the already present combo of Perl for scripting and IDL for analysis.

The list of steps comes down to:

  1. User defines a script
  2. User defines a plot
  3. User kicks off a Perl script that retrieves data
  4. Certain data triggers an IDL script in the EGSE server
  5. After the IDL script has ran (if you want to plot analyzed data), it calls a glue function so the EGSE server plots something
  6. Sometimes the Perl script requests the results as well

What we really want is that all of this happens in Python:

  1. User defines a plot
  2. Python script retrieves data
  3. Python script sends data to plot, which is maintained by a running Python process

The problem here is that the old situation allows for configuration of plots in advance. The disadvantage is that this needs a bunch of glue code, and doesn't allow for version control. The advantage is these are defined in a graphical way, and doesn't need any scripting.

2010-10-08 SpaceWire at SRON

The purpose of SpaceWire (short: SpWi), is a standard to connect pieces of a satellite for data retrieval and control. The speed is 2 to max. 400 MBit/s. The missions SWIFT and the Lunar Reconnaissance Orbiter use SpaceWire.

SpaceWire Lab Cables.jpg (Image courtesy of STAR-Dundee)

The signal is sent using LVDS, low voltage differential signalling. It's a full duplex line, with two pairs each way. The standard defines the cabling in about eight pages.

The encoding is done using Data Strobe Encoding. Tx and Rx do not share a common clock. The advantage is that you're resistant against clock skew. The disadvantage is that you now have two frequency domains in your FPGA.

There are four- and ten-bit tokens where the first bit is a parity bit and the second is the indicator for the token length. The four bit tokens are for control purposes, and there are four possible tokens. Notice there is no defined packet content; nor is there a defined length. In OSI terms, SpaceWire describes the physical and datalink layer.

An active SpaceWire link is never silent, between data the transmitter sends NULL codes. These can also be sent between the bytes of a packet. The standard also defines a time code, a special data packet for time sync purposes. The contents are not defined in the standard. This packet gets priority above data so you can send it anytime (yes, even right in a packet). For flow control, the receiver sends flow control tokens (FCT) to the data sender. For each token you can send eight characters. These can be send ahead. The FCT is one of the four control tokens. For link management, a handshake is defined. For parity errors there is no retry mechanism.

Although SpaceWire is point to point, it's possible to create networks; you then stick the packet route (the path) in address bytes in front of packets, and like the old Internet bang addresses, these are removed by each router at each step. Thus routing is simple and defined on a relatively low level.

Since there are basically two types of data you'd want to send (sensor and housekeeping data), there are two protocols. RMAP, remote memory access protocol, is most useful for housekeeping purposes. STP (Streaming Transport Protocol) is better for sensor data. In the past, SRON used CCSDS where now RMAP is used in SpWi. STP is meant for bulk transfers. The packet overhead is lower than with RMAP because the stream is first set up, then explicitly closed when necessary.

SRON has set up a test project for which the purpose was: Two SpWi ports, proper Linux driver and 8 MByte/s sustained data throughput via a PCI card. We've tried boards from Aurelia and STAR-Dundee. There are also boards from 4Links and Dynamic Engineering. The Linux as well as the Windows drivers were unable to get the required speed.

SRON has also looked into a SpaceWire IP core which had to be vendor independent (Actel and Xilinx), implemented as an AMBA interface (for inclusion in a LEON core) and available in VHDL (not just a netlist). And reasonably priced. ESA has this available.

In a test setup with a PCI card and an Actel board, we could get up to 6 MByte/s due to a slow Linux driver. Yes, that's 6 megabyte of data per second. A better solution was to put in an intermediary board with a LEON core that translated to Gigabit Ethernet.

There is also a SpaceWire Light IP core via the OpenCores project.

2010-10-05 Ubuntu on an old machine

If you want to use Ubuntu on an older PC, then memory might be tight. I recently got a HP/Compaq 7100dc which only has 512 MB memory, and will be used for light surfing plus web based e-mail. It does not have an attached printer.

The following command removes a number of services which are superfluous in the above situation:

 $ sudo apt-get remove bluez brltty pcmciautils speech-dispatcher \
    apport cups system-config-printer-gnome evolution

Explanation: this removes support for braille input devices, BlueTooth, laptop extension cards (PCMCIA), text-to-speech, crash reporting (to Ubuntu), printing and the memory hungry e-mail/calendar client Evolution.

If you are knowledgable about security, you can make the decision to remove AppArmor. More information here: AppArmor in Ubuntu.

 $ sudo apt-get remove apparmor

Also, on such a machine it is wise to turn off all visual effects by going to menu System, Preferences, Appearance. Switch to the tab Visual Effects and select None, then click Close. Explanation: this switches your window manager from Compiz to the much lighter Metacity.

The above mentioned procedure saved me 30 MB, from 125 MB used memory to 95. To find more memory-hungry processes, take the following procedure. First, find a process:

 $ ps -e -o rss,vsz,cmd --sort=rss | sort -n -r | head

Then find the path to the process:

 $ whereis <processname>

If you have a path, find the corresponding package:

 $ dpkg-query -S /path/to/programname

Then find out if you really need this package:

 $ dpkg -s <packagename>

If you don't need it, you can remove it:

 $ sudo apt-get remove <packagename>

2010-09-23 From Perl to Python

At work, we are currently using Perl and IDL, alongside our (in C++ written) EGSE server software. For the controlling of electronics, we use Perl. For the visualization of the electronics readouts, we use IDL.

For different reasons, we are looking to replace both parts with Python equivalents. This involves both a port of the software as well as a migration path. It also offers the chance to do a clean, object-oriented rewrite which could mirror the C++ libraries.

Perl basically provides scripted control/readout of sensor equipment. These scripts can be edited and run from within the EGSE, but they can also be run from the commandline.

IDL, however is more tightly integrated with the EGSE. It is compiled along with the EGSE. It listens to data requests and these are analyzed and then plotted as well as transported back to the controlling Perl script.

Besides plots by IDL, it's also possible to create plots with the EGSE software itself. We have to look in what way we want to let these co-exist with the Python plotting facilities.

We will create a design document where we look at the following items:

  • How the Perl control/readout libraries will be ported to Python
  • Which plots are present in the EGSE, and what we will do with them
  • Where the Python scripts will run
  • Where the Python plots will display
  • What current problems must be fixed

2010-08-30 How to recover from unexpected reboots

It's pretty interesting to dive into the situation of recovering from unexpected reboots. Our usual lab setup consists of three parts:

  • The PC running our Generic EGSE software: a user interface for powering, controlling and reading out a sensor
  • The so-called Controller board, an off-the-shelf board from Pender Electronics, running our embedded software
  • A project-specific electronics board with one or more DACs and/or ADCs to bias and control the sensor equipment, hereafter called the biasing board.


Any of these could suffer unexpected power loss and subsequent power restore. The basic question is: what do we handle in the way of recovery?

For lots of things, it's necessary to maintain status. An example is the following: you are a scientist and use the above setup to set up and test your sensor. You leave the lab but then the PC unexpectedly reboots because a system administrator mistakenly remotely rebooted the PC.

When the EGSE software automatically starts again, should it attempt to initialize the biasing board? Probably not -- you may be running a test and the sensor settings should not be changed.

But then again, there is the situation of an expected power-up. You want to differentiate between the two, if you want your electronics to always be initialized upon normal startup.

Now there's complexity: both the EGSE and the Controller board will have to maintain state. Any discrepancies will have to be resolved between the two. In the end, it might be much simpler to just say that we do not support automatic initialization when the Controller board comes online.

Choices, choices...

2010-08-24 Thoughts on a software design part 2

When asking for some criticism from a colleague today, I got some additional pointers and ideas on my design sketch. Some concepts were implicit, but would be more clear when mentioned specifically:

  • The board running the embedded software (ESW) actually functions as a controller for the rack. Thus, call it the Controller.
  • The slot object (more on that later).

The Controller board will carry state for the equipment that's in the rack, but since a (possibly accidental) power-down of the rack would lose state, the previously mentioned discovery mechanism still has to be created.

The software will also get a lot more simpler if we assume there is some intelligence in the rack. Thus, we can assume that a future rack will perform the following functions:

  • Sends a signal to the Controller board telling a board has been inserted in a certain slot;
  • Can cut the power of a specific slot.

He also pointed out it's worth thinking about whether we must model the rack functions itself, perhaps as a class called RackDriver. The slots in the rack were previously left out of the model, because they did not have any meaning for the software. This now changes, since we assume the rack has some intelligence.

2010-08-17 Thoughts on a SW design

So far, the custom software we built at SRON assumed that users would manually configure the software when any hardware was replaced, and that there was only one electronics board present. The following is a preliminary design on the software changes required to support an array of boards that is placed in a standard format rack.


The basis is, that we are developing standard electronics boards for certain tasks. For instance, a standard design for an electronics card that contains a DAC and an ADC. These cards must work in array-like structures, for instance a standard 19 inch subrack which contains a number of these standard electronics boards. This could be used door biasing a sensor, reading out a sensor, housekeeping duties and PC communication duties. Such a rack would consist of a number of slots, each which consist of a rail that guides an inserted board to the backplane. This backplane provides power and connection to the software. The central requirement is: the user should be able to add and remove the boards without complex procedures. Any procedures like board-specific power-up/power-down sequences should be handled automatically.

The setup thus consists of two parts:

  • The hardware consisting of:
    • A rack with electronics boards
    • An off-the-shelf, generic electronics board with an LEON-based FPGA core (made by Pender Electronics)
  • The software consisting of:
    • The PC which runs the EGSE software (Electronic Ground Support Equipment)
    • The LEON core running ESW (our embedded software).

Use cases

To support the above sketched requirements, we can recognize several use cases in this setup:

  • Adding, removing or replacing a board
  • Powering up/down the rack (planned or unexpectedly)
  • Powering up/down the software (planned or unexpectedly)

Use case: adding, removing or replacing a board

The user must be able to add or remove a board into the rack, and the software should detect this. Also, most boards must be initialized in some way. Thus there must be hooks that run a particular script when hardware changes. This also means that the hardware must actively identify itself, let the script take care of this, or give the software some uniform way of checking this. More on this later.

Replacing, or the moving of a board from one slot to another can be covered by a simple remove/add action.

Use case: powering up a rack

Since the hardware and software can be powered on and off independently, both situations must be covered. Thus the software must have some sort of discovery mechanism when starting. The hardware must have some way of rate limiting if it actively advertises the adding or removing of a board. More on this later.

Powering down a rack

There are two possible ways in which a rack is powered down: expectedly and unexpectedly. The software does not need to be adapted either way. In the case of an expected power down, there should be a project-specific power down script. In the case of an unexpected power down, it should be determined whether the project needs a way of detecting this.

Powering up the software

When the EGSE is powered up, it should see whether a rack is connected and if so, a discovery mechanism should see what boards are present. More on the discovery mechanism later. When the ESW is powered up, no particular actions are necessary.

Powering down the software

There are two possible ways in which the EGSE is powered down: expectedly and unexpectedly. The software does not need to be adapted either way. In the case of an expected power down, there should be a project-specific power down script. In the case of an unexpected power down, it should be determined whether the project needs a way of detecting this.

The ESW can also be powered down, either accidental or as per request. There is no difference between the two, since the ESW functions as a pass-through and does not maintain state.

Objects and attributes

For the above use cases, the software obviously requires a constant and up to date register of all available boards plus their addresses. The following objects can be found in the use cases: rack, slot, board. A rack is divided in slots. A slot contains a board. Typically, racks can have shelves but for now, we assume that there's only one shelf. Also, racks are contained in a cabinet but again, there can be only one rack for now.

The current requirements do not necessitate that the software exactly knows which slots are occupied. Thus, this concept is currently not taken into account. That leaves us with the following classes:

  • Rack, with attributes: boards.
  • Board, with attributes: version, address and insertion script.

Addresses and discovery

There are two options for addressing. Currently, all boards have an address pre-programmed into the FPGA. This is fine in a situation where we can manually increment the unique address. The software will then simply use a discovery mechanism where a dummy request is sent to each possible address. When a reply is received, the board is then added to the present board list. Discovery must be quick since it inhibits other usage of the bus, and is done periodically. Thus the most logical place to run the discovery, is probably the ESW.

But when using multiple off-the-shelf boards, it is much easier to let the boards actively know that they were inserted, and let the software hand out addresses. The software still needs a discovery mechanism in case the software is brought down for some reason. This can be the same as previously mentioned.


In the first release:

  • We will let the software poll the hardware chain, and thus detect when hardware has been inserted or removed.
  • Boards will use pre-programmed addresses.
  • The concept of slots is not necessary.

For version two, we see the following points:

  • Inserted boards actively trigger a signal that notifies the software. This is currently not incorporated into the protocol that we use for communication between hardware and software. The hardware must have a way of rate limiting its active signal, perhaps by waiting for a random time.
  • After a board is inserted, the software will issue an address that uniquely identifies the board. The board will then start listening only to that address.

For version three, we see the following points:

  • Introduction of the concept of slots into the software and hardware. When (or better: before) a board is removed from the rack, a signal is sent to the software.

Unresolved points

  1. Above, it is assumed that the embedded software runs on a LEON3-based board that's directly connected to the backplane of the rack. Is this assumption correct?
  2. The current bus protocol is not really fast (2 MHz), is this fast enough to support a number of measurement boards?
  3. Do we need to detect when the rack is powered down immediately? Or is it OK to wait for the next action (user or scripted) to generate a communication timeout?
  4. It should be possible for selected boards to be powered down or reset from the software. However, since in the current backplane design one power line is provided for all boards, this does not seem possible right now.

Miscellaneous notes

  1. Some boards are high enough to occupy multiple slots, for instance when the board has a high speed FPGA which has a heat sink stuck on it. But for now, we ignore this. The software does not need to know that some slots appear empty but are actually physically blocked from usage.
  2. If a board is in the middle of a measurement, can it answer the discovery mechanism in the standard hardware design?

2010-07-20 Coverity demo

We got a demo from the Coverity people. We ran their tool on our code base in advance. Via a WebEx session we got an explanation of the results, but first we got an overview of the company and their projects since some of the team were new to this stuff.

It's a pretty young company, founded less than ten years ago, and their aim is to deliver products that improve the quality of your software. Clients are in the medical and aerospace branche. Wikipedia article on Coverity. They have a 1000+ customers.

From the webbased Integrity Center software, several tools can be controlled. One of them is Static Analysis, called the Prevent tool. The tool identifies critical problems, not the more trivial things like style compliance etcetera.

Since bugs are cheaper to fix in development rather than in the field, this gives the user time and cost savings.

The software checks the compiler calls that are made when you do a build (via make) and then works on the code in the same way. It's not a replacement for unit tests. After running, a database of the results is written and there is a web frontend where you can read out the database.

The screen shows a number of defects, with filter options at the left. When clicking on a defect, you can see the code as well as th classification of the defect. Along with the classification, there is a short explanation of this type of issue. Clicking further will also give simple examples so you better understand the defect.

Each defect can be assigned to a certain team member. We have already invested in using Traq so I'm not so sure that's useful.

We had questions about finding concurrency problems. Coverity can help with this but they support pthread it of the box. Since we use QThreads, we should make a model for that library. However since we have the code available (Qt is open souce) and it's using PThreads, it's not a problem and Coverity will be able to pick it up automatically.

Besides the existing checks, it's possible to add your own checks. Perhaps you want to enforce a certain way in which you use an external library.

The software tries to be smart. For example sometimes you do some smart coding which usually triggers an error. Coverity will use heuristics and not report it if the rest of the code base shows that this is not something worth reporting.

We closed off the demo with a discussion on licensing. The account manager teams up with a technical consultant and together they pretty extensively work on the requirements and resulting cost savings. From that, the price is derived. There are other licensing models however.

2010-07-15 Sending email from a Perl script

If you're on Debian or Ubuntu Linux and you want to send a quick e-mail from a Perl script, use the Email::Send module (this module has been superceded by the Email::Sender module, but that one doesn't seem to be present in the Debian Stable package repository yet).

First, install the appropriate packages:

 $ sudo apt-get install libemail-send-perl

Then use the following snippet:

 use Email::Send;
 my $message = <<'__MESSAGE__';
 Subject: This is a mail from a Perl script
 This is the body of an e-mail from a perlscript
 my $sender = Email::Send->new({mailer => 'Sendmail'});

Now go and use this wisely, my young padawan. Some observations:

  1. You rely on Sendmail being present
  2. You trust all input variables
  3. You are not using HTML, attachments, or what have you

2010-01-17 Calibrate good times

We have a DT-470 temperature sensor in the cryostat of the project I'm currently working on. The problem is that our software is displaying the wrong readout. I'm trying to figure out how to display the correct value in Kelvin.

I've got the following to work with:

  • The cryostat is filled with liquid nitrogen, which we know is around 77 K.
  • The sensor was also read out when the cryostat was not cooled, so we have a combination of raw value (as read from electronics) and known temperature as well (room temperature, 20 degrees Celsius, is about 293 K).
  • We've been provided with a data sheet from which a polynomial can be derived.
  • The spec says we need to drive the sensor with a 10uA current, but in actuality, we drive it with a 12.368 uA current. I'm told this is not a big deal, because the difference in current is unlikely to cause a difference in warmth.
  • The spec also gives us a lookup table to transform raw value to Kelvins
  • Because of the way the sensor is read out, we need to apply an additional offset and factor (our electronics influence/amplify the readout).
  • The sensor readout is linear for the range from 10 to 200 K, then follows a curve to 0 K.

The software has the ability to apply a polynomial to a raw value (i.e. a value that's read out from the electronics), as well as apply a user-configurable function to a value. That is usually used for convenience like electronics giving us a negative value, when we'd rather receive a positive value.

In this case, the polynomial is applied to correct the value for the way our electronics influence the raw value. Then, the user-configurable function is applied, which in this case is the polynomial that follows from the data sheet.

So the steps are:

  • Remove the electronics and drive the sensor manually. Do we get the same value (raw) as we get from the electronics?
  • Get the raw value, and see whether our first polynomial is OK (the first polynomial is a simple one, just an offset plus a factor). We can do this by looking up the raw value in the lookup table.
  • Use that in-between value and check the lookup table to see whether the second polynomial is OK.
  • The heating of the electronics board could also play a role, we need to check that as well and if so, correct the first polynomial.

2009-12-09 Installing Linux Google Chrome as a regular user

Early december 2009, Google launched the beta version of the Chrome browser for Linux. They provide RPM and Deb packages, allowing for easy installation.

Sometimes however, you're working on a Linux PC where you do not have root access. The following procedure allows you to install and run Chrome as a normal user:

  • Download the .deb package and save it into your home directory
  • Unpack it into a subdirectory called Chrome with the following command:
 $ dpkg -x google-chrome-beta_current_i386.deb Chrome
  • To easily start up the app, create a launcher icon. Here is a good tutorial on adding a launcher to a panel. In the name field, type Chrome. In the command field, type /home/yourusername/Chrome/opt/google/chrome/google-chrome. Click the icon, and go to Chrome/opt/google/chrome to find the icons there.

2009-12-02 Who is logging in

Who is logging into my Linux workstation?

At my most regular workplace (I have several), I have a Debian Linux workstation. The username/password information is managed over NIS, and is configured such, that every user can log into every workstation.

I have no problem with this, but do like to know who is logging in when I'm using the desktop. Thus at startup, I run the following script in the background:

  [ ! -e /usr/bin/whoami ] && exit 1
  [ ! -e /usr/bin/gmessage ] && exit 1
  while [ 1 ]
    LOGINNAME=`w  -h | cut -f1 -d' ' | grep -v whoami`
    if [ $LOGINNAME ]; then
      gmessage "User $LOGINNAME logged in" -button OK
      sleep 1

Save this script somewhere in your home directory. I've called it 'loginwatch'. Then make it executable and run it in the background as follows:

 $ chmod +x loginwatch
 $ ./loginwatch &

This script assumes that you use the Gnome desktop, because it uses the gmessage utility.

2009-10-19 Rolling back a change

Suppose you inadvertently made changes in some files some time back. You can examine revisions of files with

 $ svn log ccsds.h

You see that the previous revision was 205 and that it was correct. With an SVN merge, you can make that old revision the current revision:

 $ svn merge -r HEAD:205 ccsds.h

Check in your file and you're done!

2009-08-28 PHP client using Thrift

Today I was thinking about a PHP script that uses Thrift to retrieve a couple of results. We have the following Thrift definition:

 /* This contains the three things identifying a logging program */
 struct Logger {
     1:  string      userName,
     2:  string      hostName,
     3:  string      appName
 /* This is a debug message */
 struct Message {
     1:  Logger      origin,
     2:  string      content
 /* This defines the remote logging services */
 service RemoteLog {
     // send a log message to the logserver
     oneway void         newMessage      (1:Message aMessage)
     // get list of loggers available
     list<Logger>        getLoggers      ()
     // get messages from a specific logger
     list<Message>       getMessages     (1:Logger aLogger, 2:i32 aFromID, 3:i32 aMax)

However, I'd then have to implement the reverse of the above description. In other words, I am asking the remote logging service for whatever he has received over time. To get this up and running, the following steps have to be taken:

  1. Define Thrift definition
  2. Generate PHP stubs (client side)
  3. Rework these into script
  4. Generate C++ stubs (server side)

2009-08-24 Archiving to a branch

Today, I needed to archive and clean up old machines that were used for a project that has reached its end-of-life. These PCs were used in a laboratory setup, controlling custom electronics.

We run our custom lab software on those PCs and the installation is done by checking out a copy of the source, and doing a local compile. Problem is that these machines have been off the network for a while and some local modifications were necessary. While cleaning up, I found that these modifications were not committed to the SVN repository.

In the meantime however, we worked hard on the software and now I cannot just do an update and commit these old modifications. The solution is to create a branch and commit the changes to that branch.

First, find out what the local version is:

 $ svnversion .

We'll make a branch of revision 1150 first. Then we'll check out that branch in another directory:

 $ mkdir ~/tmp
 $ cd ~/tmp
 $ svn copy -r 1150 \
 Committed revision 2011.
 $ svn co
 Checked out revision 2011.

Then we'll go to the local copy that was running all the time on this PC, and create a patch of the local version:

 $ cd ~/sw
 $ svn diff > ~/hackwork.patch

Go back to the directory with the newly created branch. Apply that patch to the branch, then commit.

 $ cd ~/tmp/our_software/branch-project-pc0054
 $ patch -p 0 < ~/hackwork.patch
 $ svn ci -m "Archiving local modifications"
 Committed revision 2012.

2009-07-09 Presentation on D-Bus and its usage

Recently I gave a presentation on D-Bus and what its usage would be for the software we create within SRON.

Here is a link: D-Bus introduction SRON

2009-06-08 Using D-Bus in your application

We've got a Qt-based application (daemon style) which writes log files. In order to nicely integrate the app into our Linux environment, the daemon should be able to receive a signal that it needs to close off its log files and start writing a new one.

Rotating logs is done by placing a small instructive text file in /etc/logrotate.d and by adding the possibility to the daemon to receive a signal of some sort.

The old school way is using plain old Signals but these interrupt system calls. We do not want that to happen; the daemons run in a lab environment and such an interruption could disturb a measurement.

The new style is using D-Bus for this stuff.

2009-06-06 Getting Chrome running on Fedora

Recently, the alpha builds of Chrome for Linux became available. Unfortunately, only Debian and Ubuntu packages were released.

To get Chrome running under Fedora 10, take the following steps:

Download the chrome .deb file

Create a temporary directory in your home dir:

 $ mkdir ~/blah

Unpack the .deb file there:

 $ cd ~/blah
 $ ar x ~/Download/chrome*deb

Unpack the binary code:

 $ tar xfz data.tar.gz

Move the binaries to your /opt

 $ mv opt/* /opt

Now create a couple of symlinks in /lib so Chrome can find all the necessary libraries (apparently these are named differently under Debian and Ubuntu):

 $ cd /usr
 $ sudo ln -s
 $ sudo ln -s
 $ sudo ln -s
 $ sudo ln -s
 $ sudo ln -s
 $ sudo ln -s
 $ sudo ln -s
 $ sudo ln -s

Now chrome can be started:

 $ /opt/google/chrome/google-chrome

Create an application launcher on any panel for easy access.

2009-03-27 LEON3

At work, we're currently experimenting with the LEON3 processor. It's an open chip design that contains a general CPU and has buses for several applications. It's possible to design your own logic for controlling custom electronics, and then use the bus to connect your logic to the CPU.

On that CPU, you can just run Linux and with a driver you can read out your custom logic. The LEON3 CPU core has a SPARC instruction set, making it perfect for running Linux. The SPARC instruction set is a bit of a loner when it comes to embedded OSes though -- the ARM architecture is much better supported. But there you go.

The LEON3 is an alternative for an FPGA that's connected with a serial port or somesuch to a full-blown PC. It also makes it possible to move a lot of logic from the FPGA to C or C++ code. Although arguably you lose on simplicity (there's now a full OS running on the LEON core), you gain on flexibility because of the fluidity of software as opposed to an FPGA.

2009-03-06 Looking at Apache Thrift

Today I prepared a presentation that looks at Thrift, a cross-language network library, or in other words, a library for remote procedure calls.

PDF: Apache Thrift.pdf

2008-12-19 Linux USB device handling

For reading/operating lab instruments like multimeters and power supplies, the GPIB standard is used. This standard defines the type of cabling and connection.

However nowadays the interface cards for this type of card are awkward to get and are replaced by an USB-based GPIB adapter like this one:

In this weblog entry, I'm going to explain the situation where you have a Linux machine and a USB device, but as an application developer you don't know how to continue from here. This is all explained using a USB-based GPIB adapter.

These particular devices are handled on Linux using the default gpib_common.ko kernel module and a device-specific driver which the manufacturer distributes (in this case the ni_usb_gpib.ko module).

However that's not the end of it. When the device is plugged in, it has to be connected to a device file like /dev/gpib0. This is done by loading the modules, loading the firmware (with some particular devices), running the gpib_config utility and setting the correct permissions on the device file.

We want this done automatically on our Debian machines. We already have compiled and repackaged the gpib_common.ko and ni_usb_gpib.ko modules, and have installed these.

First we need to know if the module will be loaded when we connect the device. The kernel handles loading appropriate modules by calling modprobe. Typing the following command can tell you whether modprobe knows what to do when such a device is connected. Replace the 'grep gpib' with 'less' if you don't know what string you're looking for.

 $ modprobe -c | grep gpib

If that's fine, then connecting the device should load the modules. Type the following command to check this, again replacing the 'grep gpib' with something more appropriate:

 $ lsmod | grep gpib

Connect the device and check for changes. If a device driver is loaded, then a udev rule should be written. If not, re-examine the output of the modprobe -c command. An example extract:

 $  modprobe -c | grep gpib
 alias usb:v3923p702Ad*dc*dsc*dp*ic*isc*ip* ni_usb_gpib
 alias usb:v3923p709Bd*dc*dsc*dp*ic*isc*ip* ni_usb_gpib

Observing one of those lines a bit closer:

 alias usb:v3923p702Ad*dc*dsc*dp*ic*isc*ip* ni_usb_gpib
            ^^^^ ^^^^
 The vendor ID    The product ID

In my case, no driver got loaded, because the vendor and product numbers that are outputted by modprobe don't match with the USB device its vendor and product number.

To see what the vendor and product numbers of the USB device are, type:

 $ lsusb
 Bus 007 Device 003: ID 3923:702b National Instruments Corp.

Check the string 1234:5678 just before the name of the device. The first part is the vendor number, the second part is the product number, both hexadecimal. As you can see, in this case the vendor number matches, but not the product number.

Add a new file in /etc/modprobe.d and then add a line in there which looks like the lines in the existing modprobe configuration. Adapt the vendor and product numbers as necessary. Then type lsmod and connect your device. Type lsmod again to see whether the appropriate driver loaded. If not, did you make a typo in the vendor or product numbers?

Now that the driver is loaded, we go on to writing udev rules. Check the Writing udev rules document for this.

Some tips for writing udev rules follow.

First you find the device details using:

 $ find /sys -name dev  | grep usb

Find the device its path in /sys and use it with udevinfo. Note that some devices require firmware to be loaded. udevinfo then will not show a lot of information. The device must then be identified purely by its product and vendor ID.

 $ udevinfo -a -p /sys/class/usb_device/usbdev2.18/dev

Before you plug/unplug the device, run the udev monitor to see what events are fired off:

 $ udevmonitor

To test whether your rule fires off, use logger in your udev rule:

 SUBSYSTEM=="usb", DRIVERS=="usb", ATTRS{idVendor}=="3923",
   ATTRS{idProduct}=="709b", ACTION=="add",
   RUN+="/usr/bin/logger My rule fires off!"

Note that the above line is actually ONE line, without any linefeeds except at the end.

When you want to load firmware upon connecting the USB device, you might need to pass the location of the device to the firmware loader. It's possible to use the value of sysfs attributes in your RUN or PROGRAM part of the udev rule:

 SUBSYSTEM=="usb", DRIVERS=="usb", ATTRS{idVendor}=="3923",
   ATTRS{idProduct}=="702b", ACTION=="add",
   RUN+="/usr/bin/logger Running /usr/sbin/gpib_config for device
   USB-GPIB-B on bus %s{busnum}, device %s{devnum}"

The %s{...} is substituted for a particular sysfs attribute its value.

2008-12-09 Offset dependency

Previous year, I wrote about the Fiske steps routine, a routine which automatically searches for an optimum sensor setting.

For this routine to work, the setting and reading back of voltages and currents has to be very precise. Thus, a 'measure offsets' routine was also developed, but it turns out this routine is going to be a bit more complicated than expected.

Here's an image of the output of the Fiske steps routine:
fiske example.png

We had a hard time today getting the routine to work nicely. The physics student told us that it has something to do with the fact that the FFO voltage depends in the first place on the FFO bias current, but also in a much lesser way on the FFO control line.

Some background about this. The sensor behaves as a resistor, thus when setting a particular FFO bias current, the resulting FFO voltage has a direct relation. This relation becomes somewhat less simple when the FFO control line is set. This control line has two influences: running a current creates a magnetic field, and because the copper line at the back of the sensor substrate acts as a resistor, a little heat is given off.

The heating makes the offset somewhat different, and should be taken into account. Also, there's timing involved; setting the FFO CL to zero after a high setting, the FFO voltage can be seen slowly dropping from near zero to zero.

2008-11-21 Temperature sensor versus PT1000

One of the tests we do on the temperature sensing ASIC is to measure the difference between the device and a PT1000.

A number of samples is taken from both and then on the resulting two arrays, we do a first-order fit. The offset should be 0 and the gain should be 1 otherwise an error has crept into either. For each ASIC sample, we repeat this procedure.

2008-11-04 Bitstream mode

The new project its temperature sensor (tsens) ASIC is basically a Delta Sigma analog-to-digital converter. What it basically comes down to, is that the ASIC measures temperature differences as a 1-bit value. Through calculations, a temperature comes out but that raw 1-bit value can still be read for verification purposes.

For the coming radiation test, we'll read out lots of things and among them we will be reading out the raw stream of bits.

The FPGA needs to be set in a particular mode for this:

  • Set the drdy_select register to the appropriate socket (the board is outfitted with two sockets, to test two ASICs)
  • Make the buffer for the bitstream empty by setting the buffer_flush register to 1
  • Set the sequencer_mode register to 2, which means bitstream mode
  • Set the buffer_pktsize register to 4, which means that we want four 32-bit words when we read out the buffer
  • Set the bs_mode register to the appropriate socket which starts the measurement

When this is done, the register buffer_read needs to be read out regularly to make sure the buffer doesn't overflow. The command to read out the buffer can now contain four 32-bit words.

Now it's a given that the time for measurements after radiation is very short since the samples can't be out of the radiation too long (otherwise you invalidate the test). Thus we have 60 seconds to do a measurement.

Read 64 seconds, get 8222 bits per readout, you'll get 16444 32-bit words.
Each seconds, for 0.1 second a measurement is done of 8222 bits.
The rest of the second, the tsensor stops measuring and we can read out the buffer. With a buffer packet size of four 32-bits words means 4111 times reading out 128 bits.

A nice thing about the buffer is that it's so big: a whole whopping megabyte is reserved just for the bitstream that comes out of the temperature sensor. If you don't want to disturb the process with communications from the PC, then just let the buffer fill up, wait for a minute and retrieve the whole buffer in one fell swoop.

2008-10-17 adding a logfile

My thought was; adding a logfile to our software within half an hour should be doable. So let's give it a shot!

The server has a debug macro. So obviously it'd be easiest to add it there. The macro expands to a call to a singleton, which emits a Qt signal. It's caught by a class which, amongst others, writes this stuff to the screen.

That was the first idea, but it turns out someone already made a start using syslog. Before the signal is emitted, I also added a line to write it to syslog.

Update: turns out that's a little shortsighted. I'd like to coin a new law that states that there's always more work involved than you think:

  • Removing useless stuff like date/timestamp, which is added by syslog
  • Mapping our internal log levels to the syslog priority levels

2008-10-16 analyzing a running process

Today I was notified that our custom-developed lab software suite had crashed.

Having a look, it turns out that it hadn't crashed, but using top showed in fact that it was using 100% CPU time.

First I did an strace on the process, which showed hundreds of lines looking like:

 clock_gettime(CLOCK_MONOTONIC, {836478, 309454188}) = 0
 clock_gettime(CLOCK_MONOTONIC, {836478, 311665927}) = 0
 clock_gettime(CLOCK_MONOTONIC, {836478, 313925541}) = 0

Then I decided to get a stacktrace from the process using gdb:

 labbox1:~$ gdb --pid=17732
 (gdb) bt
 #0  0xb75b3e5d in memset () from /lib/tls/
 #1  0x084dad4a in QTextEngine::LayoutData::reallocate ()
 #2  0x084de989 in QTextEngine::attributes ()
 #3  0x084e8c33 in QTextLine::layout_helper ()
 #4  0x084ea124 in QTextLine::setLineWidth ()
 #5  0x085211e5 in QTextDocumentLayoutPrivate::layoutBlock ()
 #6  0x08527825 in QTextDocumentLayoutPrivate::layoutFlow ()
 #7  0x0852544f in QTextDocumentLayoutPrivate::layoutFrame ()
 #8  0x08525910 in QTextDocumentLayoutPrivate::layoutFrame ()
 #9  0x08525b6c in QTextDocumentLayout::doLayout ()
 #10 0x08525c10 in QTextDocumentLayoutPrivate::ensureLayoutedByPosition ()
 #11 0x08525c98 in QTextDocumentLayout::blockBoundingRect ()
 #12 0x0852ccb6 in QTextCursorPrivate::blockLayout ()
 #13 0x0852ea62 in QTextCursorPrivate::setX ()
 #14 0x0853289d in QTextCursor::deleteChar ()
 #15 0x080b7fe8 in ScriptWindow::showOutput () at src/Debug.cpp:50
 #16 0x080b846f in ScriptWindow::runOutput () at src/Debug.cpp:50
 #17 0x0810afd9 in ScriptWindow::qt_metacall (this=0x8fcc560,
     _c=QMetaObject::InvokeMetaMethod, _id=48, _a=0xbfd419c8)
     at src/moc_Windows.cpp:248
 #18 0x08a14892 in QMetaObject::activate ()
 #19 0x08a14f54 in QMetaObject::activate ()
 #20 0x089a4786 in QProcess::readyReadStandardOutput ()
 #21 0x089a94db in QProcessPrivate::_q_canReadStandardOutput ()
 #22 0x089a997e in QProcess::qt_metacall ()
 #23 0x08a14892 in QMetaObject::activate ()
 #24 0x08a14f54 in QMetaObject::activate ()
 #25 0x08a31b61 in QSocketNotifier::activated ()
 #26 0x08a1a90f in QSocketNotifier::event ()
 #27 0x0832426f in QApplicationPrivate::notify_helper ()
 #28 0x083296b9 in QApplication::notify ()
 #29 0x08a00097 in QCoreApplication::notifyInternal ()
 #30 0x08a258dd in socketNotifierSourceDispatch ()
 #31 0xb78c1731 in g_main_context_dispatch () from /usr/lib/
 #32 0xb78c47a6 in g_main_context_check () from /usr/lib/
 #33 0xb78c4d27 in g_main_context_iteration () from /usr/lib/
 #34 0x08a25a38 in QEventDispatcherGlib::processEvents ()
 #35 0x083a9665 in QGuiEventDispatcherGlib::processEvents ()
 #36 0x089ff0fd in QEventLoop::processEvents ()
 #37 0x089ff37d in QEventLoop::exec ()
 #38 0x08a014e2 in QCoreApplication::exec ()
 #39 0x083249d7 in QApplication::exec ()
 #40 0x0805d809 in main ()

After the weekend, we'll analyze this at our leisure but if anyone has helpful comments, let me know.

Update: today I saw that Debian Package of the Day highlighted memstat which is a tool displaying virtual memory usage. Pity I didn't know this earlier, since it looks really useful.

2008-10-09 Reading out a multimeter part 2

Again busy with the HP 8345A multimeter. The electronics guy suspected that the default settings were actually getting more accurate measurements than our scripts were getting.

The default settings are:

 NPLC 10.000000
 FUNC 1, .1
 APER 0.200000
 RES -1.000000
 LINE 49.985386
 NRDGS 1, 1

The above are all functions that influence the measurement accuracy. Some seem to confirm eachother, for instance the NPLC (Number of Power Line Cycles) says that we take one measurement every 10 power cycles (which is 50 Hz, like the LINE setting says), i.e. 1 sample per 0.2 seconds. That's exactly what the APERture setting displays.

Then we have the RANGE setting of 0.1 volts, which is what the second result of the FUNC setting says. So that's good too. What's funny though, is the result of the RESolution setting which returns a negative number. Will look into that later.

After talking to our electronics guy, he mentioned that -- yes he could pass these settings to the script but something still was wrong. After the script ended, the multimeter would display a faster measurement than before the script.

The problem here is that we end the script with the PRESET NORM command. The PRESET command can set the multimeter in three predefined states. Useful since you have something to turn back to. Unfortunately, the state after power-on isn't predefined. Turns out that's a separate command, the RESET command. OK, so that's fixed.

Next problem: when any script ends, the panel of the multimeter is locked out. Not even the RESET command fixes that. Turns out that's pretty easy: the Unlock button at the front of the meter.

After that another problem crops up: from the ten measurements I get back a variable number of them, anywhere between zero to seven. It probably has something to do with the way the EOI setting is done.

This was not fixable by using the EOI setting although it seemed that way for a while. In this case I found the problem (but not the cause) by sending the ERRSTR? command, where the error was TRIGGER TOO FAST.

The manual says here that the interval from one reading to the next is specified by the SWEEP command. Funny thing is, when an aperture of 0.2 seconds is set, with say 10 points then an interval time of somewhat more than two seconds should be enough. But when such a setting is made, the measurement takes up to 20 seconds and no values are actually returned.

I reverted the EOI setting back to 0 and now all the requested measurements are returned. The error TRIGGER TOO FAST still is returned though, and I don't trust the measurements as a consequence. As a workaround, we're now doing one measurement point, which doesn't trigger the error.

Update: after looking at this with a colleague, the issue turns out to be the following. There are two ways to end reading; through a character such as \n (line feed character), or through setting the EOI line high. The first way was being used incorrectly. This caused measurements to be returned while reading simple commands like 'ERRSTR?'. Now that was fixed, simple commands like ERRSTR? were working.

However, it stops working for cases when you request reading in binary format. That format might include that particular character and that's what was causing the early termination of readings.

That's all fine and dandy, but we don't want ASCII transmission of measurements -- we need the binary format since that makes the measurements faster. Roughly, taking a two-second measurement will take six seconds when taking the ASCII route.

This could be fixed by using the EOI line for binary readings. The command END can take care of this; requesting END? results in setting 2. This means: for multiple readings, the EOI line is set true when all measurements have been sent.

Combined this can work as follows:

  1. Open the GPIB device, set the EOS (End-Of-String) to \n
  2. Do some initialization
  3. Before requesting multiple measurements, disable the termination of reads when receiving the EOS (instead rely on the EOI line set high)
  4. After receiving the measurements, enable the EOS again and continue on merry way

Update 2: in the end, we fixed it another way. Instead of letting the read call stop automatically when an EOS character comes by, we just calculate the number of bytes to read. Then do a read call exactly for those. After eliminating the SWEEP command, we also eliminated the TRIGGER TOO FAST error! Hurray!

Also found out that the RES (resolution) setting returns -1 because it's determined automatically from the NPLC setting.

2008-10-03 Software architecture decisions

Today I bumped into a problem which required some decisions about the software. In short it comes down to the following.

We have control scripts (in Perl) which fire off Command Packets to the server. The server is connected to multiple daemons which handle tasks. When receiving a Command Packet, the server broadcasts it to all daemons.

When the daemon replies with a Response Packet, the server broadcasts these to all connected scripts.

Problem is, that broadcast isn't always useful. There could be multiple scripts, giving off commands, and these are all broadcasted. Thus it could be that scripts receive the wrong answer.

Now a solution for this:

  • Create a special packet
  • Eliminate the first broadcast for the new packets
  • Eliminate the second (response) broadcast for the new packets

After some hacking, what remains is the return packet. It shouldn't be broadcasted but the tree of classes seems to point to itself again. Could this be the Friday evening effect?!

Update: partly, it was the Friday evening effect. What was happening is that the return packet is not broadcasted, but instead is neatly redirected back to the sending Connection object.

Currently the Perl library uses a number within the packet to determine what to do with it, the APID (APplication IDentification). However this APID is separated in two types; a hardware and a software APID. The packets with a hardware APID are passed by the daemons straight to the connected hardware, and it's currently used for a binary data, not ASCII. The software APID is unique per daemon. Thus sending a packet needs to include the daemon its software APID.

New planning:

  • Include the software APID when sending packets from the Perl scripts
  • When the response is sent from the daemon, the software APID needs to be set in the response packet.
  • When the response is received in the Perl library, we need to know that when a software APID is present, the packet contents need to be treated in another way

Some important things to remember about this architecture:

  • There is not one single place which maps a particular request to a particular response -- thus it's possible to send packets with one APID, and have the response contain another
  • It's implicit that a request packet triggers a broadcasted response
  • When something else than a hardware APID is sent, the server assumes the packet is meant for the server itself
  • A packet for the server is in the form of two integers, a command and a value (an example of this is the setting of timeouts)

2008-09-22 Reading out a multimeter

We'd like to test the DAC board by reading it out with an HP 3458A multimeter (now Agilent, but this is a pre-spinoff model). It's attached through a GPIB interface with the PC we're running our software on.

hp 3458a.jpg

Currently, we've got some Perl scripts that were used in a previous project however we want long integration times for accurate measurements. The currently used Perl code doesn't allow this.

The current situation is as follows. The multimeter has a trigger arm event (command TARM),,a trigger event (command TRIG) and a sample event (several commands). Normally, all are set to AUTO and the display of the meter shows continuously updating readings. The current code sets TRIG to SYN, which means 'make a reading when GPIB commands you to do so'. Then it does some other setup and before making a reading, sets TRIG to SGL, which means 'do one measurement then disable readings'.

The resolution is set more or less automatically because a range is passed (1.2V) when selecting DV voltage measurement. The SWEEP command is then used to take any number of samples.

But now we want to have long sampling times.

The existing solution was to set the APER command (aperture), which has a value between 0 and 1 seconds, specified in 100ns steps.

Since we wanted longer measurement times, we'd use the NPLC command. NPLC stands for 'Number of Power Line Cycles'. This is useful since measuring for a whole cycle will reduce noise from the incoming power line.

In The Netherlands, the power from the wall socket has a frequency of 50 Hz. Thus if we want to measure for 100 cycles, we'd measure for two seconds.

Funny thing is, that's not what the script does. There are a number of set-up steps before the measurement is taken and I'd like to eliminate those (or at least, move them to the initialization phase).

There's for instance the RATIO command, which is set to OFF. This command measures both your signal and a reference voltage, then returns the division of those. We don't need this and the default setting is OFF. So I eliminated this command from the script.

To find out whether the ratio setting really defaulted to 0, I tried to read the setting from the multimeter. This worked, but besides the setting I received a load of garbage as well. I'd like to find out why that happens.

2008-09-16 Tweaking TCP

For the Telis project, we use the TCP protocol for the radio uplink. This is rather unusual since most other balloon flight projects use UDP and use their own retransmission and sequencing algorithms.

It has worked well for us previously, but we want to have more insight and diagnostics in a flight situation. Also, there's a backup function present in the software that's running on the PC104 on the balloon. Normally, it's not switched on and we'd like to automatically switch it on when a transmission occurs.

To gain more insight in the quality of the radio uplink, we think iptraf will do fine. A screenshot with the detailed statistics for an interface:

 - Statistics for eth0 ---------------------------
                Total      Total    Incoming   Incoming    Outgoing   Outgoing
              Packets      Bytes     Packets      Bytes     Packets      Bytes
  Total:         3142     621643        1665     131825        1477     489818
  IP:            3142     577645        1665     108505        1477     469140
  TCP:           2903     548408        1434      79900        1469     468508
  UDP:            238      29201         230      28569           8        632
  ICMP:             0          0           0          0           0          0
  Other IP:         1         36           1         36           0          0
  Non-IP:           0          0           0          0           0          0
  Total rates:         51.7 kbits/sec        Broadcast packets:          222
                       30.4 packets/sec      Broadcast bytes:          31189
  Incoming rates:       9.2 kbits/sec
                       15.8 packets/sec
                                             IP checksum errors:           0
  Outgoing rates:      42.5 kbits/sec
                       14.6 packets/sec

Note the IP checksum errors. This one would be pretty interesting for us.

Now what we probably also want, is a way to find out how many resends will occur if the radio uplink fails temporarily. We'd probably want to be gentle and not resend too much since the uplink is pretty limited bandwidth-wise. I have found a way to check this per application (man tcp, search for TCP_INFO) but not per interface.

A nice thing to use for testing purposes is Netem, the Linux in-kernel packet mangling software.

2008-08-29 Fighting an ADC

We use an ADC from Cirrus Logic on the DAC board. It's there to check the requirements for noise and linearity of the DAC (and subDACs).

It's the same ADC that we used in the Telis project. It's a nice piece of work, very accurate. For Telis it was used to measure in the hundreds of nanovolts.

dac board adc.jpg

On the DAC board however, there seem to be problems with linearity. Around zero volts, it seems to jump up and down a little. It wasn't just this sample; the ADC was lifted off the board and replaced with another -- the same problem was seen.

The gain on the ADC was lowered and the effect was still seen in the same way, so it's not the DAC that's causing the trouble. This indicates that the ADC or its placement on the board is the problem.

It's probably that last thing since with Telis it worked wonderfully, but that isn't conclusive because back then, the voltage around zero was looked at, but not very carefully.

What we'll now do is firstly use external measurement equipment and secondly, fire off an e-mail to Cirrus Logic support.

Another problem is that the ADC now gives off some REAL funny numbers after some changes in the FPGA. After a full day of analysis with three people (software engineer, FPGA engineer and analogue designer), we come to the conclusion that... the power must be shut down. You'll ask why wasn't this done right away. We did -- except the command that powers off the digital and analogue parts of the board do NOT affect the power of the ADC... That was one of the requirements.

However it remains to be seen whether this fixes the problem of non-linearity around zero.

2008-08-12 Configuring an ADC

On the DAC board, for testing purposes, a Cirrus Logic ADC is present.

It's the same ADC as used in another project. These babies are pretty well-versed feature-wise, as far as my experience goes anyway. The one we use (CS5534) has four channels, each of them with its own configuration registers for selecting measurement time, range selection, offset enablers, offset registers, et cetera.

What's different here is the FPGA in between the software and the ADC. In the previous project, I'd just write the configuration register for a certain voltage range and kick off a macro contained in the FPGA flash. Read back results and voilà.

Occasionally, I'd do those detailed steps in a script and the steps would be:

  • Set voltage range
  • Set measurement time
  • Wait for results
  • Read back results and convert

Currently, it's a bit simpler: just read out and convert.

2008-08-07 Controlling individual DACs

I've written previously about the DAC board, but mainly looking at it as a black box. This time it's time to look at the inner workings.

We perform two tests on the DAC, noise measurements and linearity. The complete range of the DAC is actually covered by five small DACs inside the package, which have a 7-bit range. Although normally we just control the main DAC and let the FPGA/DAC combo figure out what happens next, when the noise and/or linearity tests don't measure up to expectations, we need to look further.

That's why there's a special override register that can be enabled (set to 1) which allows us to control the five small DACs directly. This is done through two 32-bit registers, in a kind of funny way:

First register, called testd1, holds bits 19 to 0 in the table below. Register testd2 holds bits 34 to 20. The highest bit is the most significant bit (MSB).

bit meaning
34Bit 7 (MSB), DAC 1
33Bit 7 (MSB), DAC 2
32Bit 7 (MSB), DAC 3
31Bit 7 (MSB), DAC 4
30Bit 7 (MSB), DAC 5
29Bit 6, DAC 1
28Bit 6, DAC 2
27Bit 6, DAC 3
26Bit 6, DAC 4
25Bit 6, DAC 5
24Bit 5, DAC 1
23Bit 5, DAC 2
22Bit 5, DAC 3
21Bit 5, DAC 4
20Bit 5, DAC 5
1Bit 1 (LSB), DAC 4
0Bit 1 (LSB), DAC 5

Et cetera. Note the interleaving of bits. Since testers use scripts for this, we need to create a simple function which makes it easy to control the DACs separately.

In the end, I created a function which does the following:

  • Retrieves the current registers
  • Turns the user-requested value into a binary string of ones and zeroes
  • For each bit of the new value the registers are updated
  • When done, the registers are written and activated

2008-07-01 testing temperatures

Today I configured the housekeeping feature of our Generic EGSE software (stands for Electrical Ground Support Equipment) and added a couple of temperature sensors from the DAC board. I then configured the software to plot the requested values over time; see below.

temp dac.png

Then we sprayed the sensor with Cold Spray and watched the strip chart dive:

temp dac2.png

Time to show our setup:

IMAGE 497.jpg

2008-06-26 Layout Shamroc board

Below are details of the Shamroc DAC testing board.

IMAGE 496.jpg

  1. Daughterboard, for DAC 1 (DAC not present yet)
  2. Daughterboard, for DAC 2 (daughterboard not present yet)
  3. Filtering, plus the high-accuracy ADC is here as well (which measures back whatever the DACs set)
  4. Reference power
  5. Controlling of heaters
  6. Two housekeeping ADCs, with lower accuracy than the one that's located at 3
  7. Power for DACs and filtering
  8. Power supplies
  9. Additional input for power
  10. Barrier
  11. Additional clock input for DACs
  12. Connection from FPGA
  13. Connection from FPGA for the DAC clock signal (40 MHz)
  14. Output DACs
  15. Output reference power

Interesting stuff:

  • If the clock coming from the FPGA won't live up to our expectations during testing, we can use another source and hang it on the connectors marked by 11.
  • The clock from the FGPA is slightly set apart from the bigger data connection, because the harmonics from that 40 MHz signal have a higher chance of interfering with the data lines. Note that at a later time, we'll have a 1 Hz synchronization pulse coming in from outside. This is because our electronics are only part of the whole system and there must be a system-wide clock as well. Otherwise our clocks would deviate as time progresses (think days here).
  • There's another clock coming in at 1.25 MHz for the housekeeping ADCs, on the wide ribbon.
  • The input marked with 9 is there to test the stability of the DACs. We can feed in funny signals and this shouldn't influence the output of the DACs, marked with 14.

2008-06-23 Shamroc DAC board

The DAC testboard for the Shamroc (part of Exomars) project is now under test. This board is meant to test the extremely stable DACs that the project needs. The stability is necessary because the DACs will control the seismometer, which in turn needs very long measurement times. The accuracy of the DACs will thus directly influence the measurements.

Here is a good presentation on the seismometer itself: 9_Smit_GEP-SEIS.pdf

The image below is the test configuration of the seismometer.


After reading the documentation, I personally was left with a bunch of questions. An appointment tomorrow will change all that.

Questions from a software engineer:

  • What is meant by 'very broad band' seismometers?
  • What is meant by 'seismometers in opposite sensing directions'? Is there a direct relation between the number of ADCs and sensing directions? If so, don't these ADCs have channels for that?
  • Does the GEP-Sphere (the package with the seismometers) contain everything for all directions? How many sensing directions are there?
  • How is the sensitivity compared to earth models?
  • Why is the instrument noise dominated by electronic noise?
  • How should the performance requirements be read?
  • Explanation of the subsystems in the ASIC
  • Why are commercial 24-bits ADCs not really 24 bits? Are we just measuring noise on those highest-accuracy bits? Why? Couldn't we cool it?
  • Why is 1/f elecronic noise (pink noise) the biggest problem?

Update following soon.

2008-06-19 Project SHAMROC

For project SHAMROC, part of the Exomars mission, the testing will start next week.

shamroc dac bord.jpg

This project develops mixed-signal ASICs which have DAC, ADC and temperature sensing functions and the first prototypes will arrive next week. I say prototypes, plural, since the functions are divided onto several ASICs for now. Later, these functions will be integrated on a single die.

This is exciting stuff. These electronics have to conform to the highest demands. Since we're talking about a Mars mission, the ASICs will have be operating in extreme temperatures. The weight has to be the lowest possible, and the limits on energy consumption forces us to use the minimum amount of components. And because of the conditions of the trip, as well as Mars conditions, the components have to be radiation-hard.

When the ASICs arrive, the software group has a generic workbench for testing them. We'll then work together with the electronics guys to make solid, reproducible tests.

2008-05-31 Test for a sysadmin

As part of an interview for a new system administration, we asked the following questions:


  • Background?
  • With which Unix-like systems have you had experience?
  • How many machines or users?
  • Can you name some control panels? Which one have you supported?


  • Which filesystem does Linux use?
  • What is journalling?
  • What's an inode?
  • What happens when you delete a file while an application has it opened?
  • What is: NFS? NIS?
  • What is Kerberos?
  • Which webserver do you have experience with?
  • What is a wildcard DNS entry? How would you configure Apache for that?


  • Do you know how to tunnel traffic?
  • Do you know what OSI layers are?
  • Where does TCP fit in?
  • What is ICMP? Where does it fit in?

Network above layer 4:

  • What is SMTP?
  • What's the difference between IMAP and POP3?
  • Why are both protocols insecure? How to handle this?
  • What's DNS?
  • Did you administrate DNS servers?
  • What does it mean when a server is authoritative for a domain?
  • How do you look up the name servers for a domain?
  • What are virtual hosts in the HTTP protocol?

Network layer 4 and below:

  • What's the difference between UDP and TCP?
  • What's a gateway?
  • What's a multi-homed machine?
  • If there are two network interfaces on a machine, how many IP addresses can you use?


  • How do you stay up-to-date on security news?
  • How do you check for open ports?
  • What is jailing?
  • What is selinux?
  • When is a firewall called for, in your opinion?
  • How would you go about setting up a firewall?
  • What is intrusion detection?

2008-05-25 Finding open ports

When tightening up security on a Linux server, one of the first things the system administrator does, is find out which ports are open. In other words, which applications are listening on a port that is reachable from the network and/or internet.

We'll use netstat for this purpose. On a prompt, type:

  $ sudo netstat --tcp --listen -p

Overview of the options:

--tcp Show applications that use the TCP protocol (exclude UDP)
--listen Show only applications that listen, and exclude clients
-p Show the process ID and name of the application to which the port belongs

Netstat sometimes pauses during output. This is normal; it tries to resolve the addresses into human readable host names*. If you don't want this, use the -n option.

Example output from my laptop which runs Fedora 8 (I have removed the columns Foreign Address and State for the sake of brevity):

 Active Internet connections (only servers)
 Proto Recv-Q Send-Q Local Address                PID/Program name   
 tcp        0      0        2381/nasd           
 tcp        0      0 *:55428                      1965/rpc.statd      
 tcp        0      0 telislt.sron.:commplex-main  6573/ssh            
 tcp        0      0 *:mysql                      2307/mysqld         
 tcp        0      0 *:sunrpc                     1945/rpcbind        
 tcp        0      0         2533/dnsmasq        
 tcp        0      0      3581/ssh            
 tcp        0      0          2553/cupsd          
 tcp        0      0         2352/sendmail: acce 
 tcp        0      0 *:8730                       6030/skype          
 tcp        0      0 *:http                       2371/httpd          
 tcp        0      0 localhost6.localdom:privoxy  3581/ssh            
 tcp        0      0 *:ssh                        2205/sshd

Whenever there is an asterisk (star) instead of a host name, netstat tells us that the port is listened to on ALL interfaces, not only the local interface but also any present interfaces connected to the outside world. These are the ones we want to hunt down.

Now we know the program names, we can find out more about them. We'll take for instance the rpc.statd program. First we locate the complete path of this process:

 $ whereis rpc.statd
 rpc: /sbin/rpc.statd /usr/sbin/rpc.svcgssd /usr/sbin/rpc.idmapd 
 /usr/sbin/rpc.mountd /usr/sbin/rpc.rquotad /usr/sbin/rpc.gssd 
 /usr/sbin/rpc.nfsd /etc/rpc /usr/include/rpc

Whereis does a search and finds /sbin/rpc.statd. On RPM-based systems, we can request more information about the owning package:

 $ rpm -qif /sbin/rpc.statd
 The nfs-utils package provides a daemon for the kernel NFS server and
 related tools, which provides a much higher level of performance than the
 traditional Linux NFS server used by most users.

Now we know whether we want this package or not. If not, just remove it and the port will be closed. If we need the functionality, does it need to listen to the outside network? If not, we would typically Read The Fine Manual to see whether we can configure this package to listen locally.

Repeating this exercise for each line in the netstat output will tighten a server its security.

  • Just like addresses are resolved into host names, the port numbers are resolved into services using the /etc/services file.

2008-05-22 Configuring AIDE

Today I installed AIDE on a CentOS 5 server. This package is an alternative to the traditional Tripwire. The installation is done in a cinch with yum:

 $ sudo yum -y install aide

The first thing that happened, was that this server had SElinux disabled. So I had to make the following changes to the supplied configuration file /etc/aide.conf

I added the following lines close to the top; these are copies of default rules but without the selinux checks (which would otherwise generate errors):

 R =            p+i+n+u+g+s+m+c+acl+xattrs+md5
 L =            p+i+n+u+g+acl+xattrs
 > =            p+u+g+i+n+S+acl+xattrs
 DIR = p+i+n+u+g+acl+selinux+xattrs
 PERMS = p+i+u+g+acl

Then as root, initialize the database:

 $ sudo aide --init
 AIDE, version 0.13.1
 ### AIDE database at /var/lib/aide/ initialized.

Copy the new database, to make it the baseline to check against:

 $ sudo cp /var/lib/aide/ /var/lib/aide/aide.db.gz 

Then create a file named aidecheck in /etc/cron.daily with the following contents:

 /bin/nice -n 18 /usr/sbin/aide --update | \
    /bin/mail -s "AIDE check on host  `hostname`" root
 cp /var/lib/aide/ /var/lib/aide/aide.db.gz

Be sure to make the file executable:

 $ sudo chmod 755 /etc/cron.daily/aidecheck

2008-05-21 Code review

I'm doing a code review for our partner in the current project. In a recent
integration test, we found a problem.

An overview of the equipment. We're talking about software running on a
PC104 device (basically a small PC) running Linux. This PC104 has a bunch of
serial ports.

They have a custom I/O card, let's call it Raba1. This card has two serial
ports. The software reviews the incoming data from port TTY2, thinks about it and
then sends the appropriate commands on the TTY1 port. Problem is, sometimes
the software just halts for a minute. It does nothing while actually data was
coming in.

One thing that can go wrong here, has its root in the structure of the
software. It basically runs in a big loop, at the start of which is a select()
statement which monitors the two serial ports and the internal command channel
for input.

A theory is that it's missing data because the loop isn't fast enough. It
could happen that we send commands to the software through the internal
command channel and while these are parsed and handled, other data comes in
through port TTY2.

What makes this unlikely, is that such buffer overflows can be detected by the
Linux serial device driver. These were watched and were not found to correlate
with the pauses.

2008-05-06 Recovering from a hacked server

A friend of mine had a problem with a server in which a particular PHP script kept changing, including an iframe when it shouldn't.

I took the following steps to see what was happening. This can be used as a checklist.

  • Checked /etc/passwd for strange accounts
  • Did a 'ps -ef' to see what processes were running
  • Checked /var/log/secure for strange logins through SSH
  • Checked for other services running, for example FTP. Checked the logins on that as well.
  • Checked /tmp to see whether any executables were present
  • Checked Apache's default access logs
  • Checked Apache's access logs for each virtual host, paying attention to POST requests
  • Checked world-writeable and tmp directories in user home directories
  • Checked what's in the crontabs
  • Checked the OS version. In this case, found it by doing
  # cat /etc/redhat-release
  CentOS release 4.6

I found nothing weird in Apache's log files, no funny scripts et cetera.

In a bunch of PHP scripts, the following code was appended at the end:

 echo '<iframe src="" width="1" height="1"
 alt="YTREWQhej2Htyu" style="visibility:hidden;position:absolute"></iframe>';

Googling for this turned out it's a pretty common attack. Articles suggest it might be a compomised FTP account. Checking the changed files, the date of the files suggests it's done in one fell swoop.

To see what FTP servers are running:

  # ps -ef

In case I missed anything, see what ports are listened to:

  # netstat --listen --ip -T -p

The -T option won't cut off long addresses and the -p option will print the process that's doing the listening.

Found out in /var/log/messages that a script logged in around the same time that the files were modified.

The conclusion was that a full OS reinstall was done, with a thorough tightening-up and a code review.

2008-05-06 CSS box model

Lately I've been lurking on the #CSS channel on the Freenode IRC server and I've noticed a lot of questions that basically boil down to a developer not understanding the CSS box model.

The most basic things to remember are, in my opinion, the following.

There are block and inline elements. In simple terms, block elements get a newline after them. Elements like div and p are block elements. Inline elements usually reside in a block element; think of them as stuff you'd put in a paragraph. Anchors and markup like em are all inline elements.

Then there is floating. If you let elements float left or right, they'll "float" to the top of the containing element. Floating elements must have width applied to them. Multiple elements will float nicely besides eachother if there's room.

When you've got a problem, think about the above two things and see if you can figure out what's happening. Don't just ask in the #css channel "what's happening", try to come up with a theory and state it. It'll help you a great deal more.

Also read Mike Hall's primer on CSS Positioning.

2008-04-16 Qt database widgets part 2

Last time I got away with creating a custom QItemDelegate but not this time. I need a radiobutton for a database column of which all rows must contain 0 (zero), except one row which must be set to 1 (one). This database column designates the default or current setting.

Presenting the user with a 1 or 0 isn't terribly friendly. Since a QItemDelegate is about both presenting and editing, this is an obvious candidate to be subclassed.

A nice example is the Spin Box Delegate, however, this example creates an editor but not a custom paint method for the normal viewing of the item.

2008-04-16 Qt database widgets

I've been playing with a couple of Qt classes to get an easy-to-use SQL user interface. The combo QTableView and QSqlTableModel seemed very nice. Using the Designer, I could click an interface together and voilà, a nice grid where the user can edit the database.

There were a couple of hicks. If fields were NULL in the database, these could be filled in but never put back to NULL. After reading and trying out lots of stuff, I found the following classes are involved:

QTableView Provides the general table widget on the screen
QItemDelegate Provides view/edit functions for specific cell types in the table widget on the screen
QSqlTableModel This class provides an easy-to-use API for a database table
QAbstractProxyModel A class that can be stuck over QSqlTableModel to transform, sort or filter data when it comes in or goes out of the QSqlTableModel

You have to think about what functions exactly you want to change. If it's the overall interface, think about the QTableView. In my case, I wanted to enable the user to set fields to NULL. The normal QItemDelegate doesn't allow this.

In the end I avoided the whole issue by using one type field (which can contain 'string', 'integer', 'double' et cetera.