That issue sounded familiar to me as I lost my painstakingly set up keyboard equivalents a few times as well. As this only happens once in a year or two, it’s usually not worth investigating, but I had a little theory at the back of my mind which - together with the details mentioned by Pierre - I can make sense of and spell out now.

History

To understand this properly, we need to look back. Cocoa started out back in the NeXt days and back then it already had a very simple and powerful way to let users customise their keyboard equivalents. Each Cocoa application can have a set of preferences - known as User Defaults. If an application finds a user default named NSUserKeyEquivalents which contains a dictionary it will interpret the contents of that dictionary as follows: The names (keys) of the entries correspond to the names of menu items, and the values for those keys are the keyboard equivalents assigned to them:

There are a few special characters involved in this: @ represents the Command key, ~ represents the Option key, $ represents the Shift key and ^ represents the control key. A combination of these and a single character following them make up the keyboard equivalent. The example seen above uses the Command-V keyboard equivalent to ‘Paste and Match Style’ while assigning the more complicated Command-Option-Shift-V to the simple ‘Paste’ command, thus reverting the keyboard equivalents of the two (which has proven to be good for my nerves over the years as I Mac OS X’s clipboard management is by far too effective at preserving style information I do not want).

Note that this way of assigning custom keyboard equivalents is no way perfect or even good. As it is based on the names of the menu items it is prone to failures in numerous cases: Switch the language you are using? FAIL. Have application authors change the name of a menu item? FAIL. Have Apple decide to change HIG rules for menu items (e.g. by introducing a complimentary space just before ellipsis characters in German for Mac OS X.5)? FAIL. Have menu items with dynamic names (e.g. the submenus with items for Mail accounts in Mail)? FAIL. Have menu items whose names change depending on the menu item’s state (quite stupidly it seems to be Apple’s policy since Mac OS X.5 to use different verbs in menus items such as those for showing/hiding things rather than the checkmark next to it which is much quicker to see)? Not quite FAIL, but you’ll need custom keyboard equivalents for both menu item names. Have two menu items with the same name (e.g. various submenus of Mail’s Mailbox menu)? FAIL. Despite this world of FAIL, even a hacky way of setting up custom keyboard equivalents is better than no way at all or having to manipulate the NIB files to do it.

Centralised Preferences

While the feature for customising keyboard equivalents has been around ‘forever’ in the Mac OS X universe, it used to be a ‘secret’ that was communicated from the NeXt greybeards to the Mac newbies back in the days. One had to manually use the defaults command or edit the preferences file to set the keyboard equivalents up back then.

But at some stage (Mac OS X.3, think), Apple decided to include a graphical user interface for the feature with the system - the last tab in the Keyboard & Mouse preferences seen above. This never really impressed anyone as it still forces you to manually type the name of the command whose keyboard equivalent you want to set, but at least it saves you the hassle of manually manipulating preference files.

As the keyboard equivalents are stored in the preference files of their respective applications, I always wondered how System Preferences could easily display them. There are hundreds, if not thousands, of preference files in your ~/Library/Preferences folder as pretty much any application you ever launched will have left one there. Going through all of them may be a bit of a drag and things will be even worse once you take into account the system’s global or network preferences as well.

The little trick the system uses here seems to be the following: it keeps a list of the applications which have custom keyboard equivalents set up. Taking the cue from Pierre’s observations, this list is stored in the com.apple.custommenu.apps key of the com.apple.universalaccess preferences (which doesn’t make particularly much sense to me, but there it is). With this being a static list that seems to be edited by System Preferences when you add or remove keyboard equivalents, it particularly means that it will not contain any applications for which you added custom keyboard equivalents manually. It also means that deleting those entries in the Universal Access preferences will lead System Preferences to think that you have no custom keyboard equivalents set up at all.

That isn’t a big deal yet as this won’t keep your custom keyboard equivalents from working - they’re stored in each application’s preference file after all. They just won’t be displayed in System Preferences.

A Small Deal becoming Data Corruption

It takes another error in Apple’s logic to lead to the data corruption you see. I’m guessing this as I don’t have Apple’s code, but I can imagine the following pattern happening here: When you add a keyboard equivalent for an application, System Preferences checks whether there are keyboard equivalents for that application already. If there are, it adds the new one to the existing dictionary of keyboard equivalents; if there aren’t it creates a new dictionary with just the new keyboard equivalent and adds it to the preferences.

I assume that System Preferences makes the mistake of using the cached list from Universal Access preferences to determine whether keyboard equivalents already exist for the application in question. It would be much safer to look into the actual preference file to see whether or not there are custom keyboard equivalents already. As we’re only dealing with the preferences of a single application now, this should be the same speed as the original method.

And with the wrong result stating that the application has no custom keyboard equivalents, System Preferences then writes the new dictionary containing the single new keyboard equivalent to the application’s preference. Only now the original preferences have been deleted.

Extras

The Mac’s defaults system gives you the handy defaults tool to poke around the preferences on your machine. You can get the list of applications whose custom keyboard equivalents the system knows about by running

defaults read com.apple.universalaccess com.apple.custommenu.apps

The command’s result gives you the list of bundle identifiers for the affected applications and NSGlobalDomain for the global keyboard equivalents you can set up in System Preferences (that come handy if you want to set up keyboard equivalents for things like Services or common menu commands).

To get a list of all custom keyboard equivalents set up for your applications you can run

defaults find NSUserKeyEquivalents

defaults find NSUserKeyEquivalents | grep Found | sed "s/.*domain .\(.*\).:.*/'\1'/g" | tr "\n" "," | sed "s/\(.*\),/(\1)/g" | sed "s/Apple Global Domain/NSGlobalDomain/g" | xargs -J appnames defaults write com.apple.universalaccess com.apple.custommenu.apps appnames

Its idea is to grab the bundle IDs of the applications which have custom keyboard equivalents and shove them back into the Universal Access preferences. Worked for me and doubled the number of applications on my system with custom keyboard equivalents noted in System Preferences.

If you want to run it on your system, be sure to quit System Preferences beforehand and don’t blame me if it breaks anything.

One feature I particularly like about X.5 are its redesigned network settings. Even though OS X’s network settings have always been relatively straightforward compared to other operating systems, people at Apple still did a complete re-work of the Network System Preference pane to let you gain an overview of the current setup at a single glance and hide even more of those pesky technicalities. While they were at it, they also killed the ‘Internet Connect’ application which always looked a bit out of place. Instead they included the relevant ‘Connect’ buttons in the preference pane itself.

Just look at that screenshot and enjoy how rich it is in information, how it gives you the quotidian options within a click’s reach and still leaves the more obscure options accessible. My careful guess is that it took quite a bit of effort and determination to reduce things this far while keeping them usable. All that for an interface that did already work in the previous iteration.

And - apart from VPN, which still lacks both OpenVPN and Cisco VPN support, i.e. the only VPN connections I actually need -

As Cisco VPN works on the iPhone, let’s hope they’ll port that to the Mac proper as well so we can rid ourselves of Cisco’s own client. Which must be among the ‘shittest’ software there is. Somehow they manage to make it ugly, incompatible to any non-trivial system upgrade as well as of poor performance (If you’re on a good wireless connection you will notice that Cisco VPN on the Mac is limited to data rates around 1,5MB/s because Cisco’s software can use a single core only and apparently needs that much CPU power to compress the data.)

they tried to streamline many other details of the networking experience as well. For example the Airport menu got the (arguably long overdue) update of displaying little locks next to the network names to indicate encryption and will display the details about your current wireless network if you hold the option key when clicking the menu:

Similar observations go for the ‘Sharing’ preference pane. It looks relatively simple, yet is a surprisingly powerful bit of software. Just look how many services you can set up in this little space without things looking too crowded. And imagine what it would take to make the same settings on other systems.

I don’t think this preference pane is quite perfect yet as there do seem to be some bugs which can keep settings from being applied properly (say, for Windows sharing) and it remains somewhat incomplete (Camera, iTunes, iPhoto sharing come to mind), but considering the technical fiddling involved with the jobs it does, it’s a nice and simple interface.

Easy Animations

As long as you have a bunch of readymade media which you want to move around or distort on screen, Core Animation will be great. Essentially you get a bunch of layers in your views within which you can easily animate readymade graphics. While the graphics need to be 2D, you can even do those animations in 3D style and you’ll have all the filtering capabilities of Quartz within a line of code’s reach. In particular you won’t have to worry about low level stuff like Open GL to get the right graphics effects or about threads and timers to keep things moving smoothly.

Standard examples for using Core Animation may be things like smooth resizing of views, views fading in or out or things moving around on screen in CoverFlow style. Core Animation should be a great technology for adding such niceties which make things appear or disappear smoothly - rather than just toggling between discrete modes - simply because it greatly reduces the effort needed to implement them. Frequently a few mouse clicks and a few lines of code will be all that is required.

This advantage could also end up being a disadvantage, though. Simply because it makes animating things too easy; thus encouraging people to make user interfaces overly animated with animation ending up as a distracting extra that gets in the way of using the application rather than something helping the interaction. An ugly example of that can be seen when clicking the ‘Dynamic global host name’ check box in the sheet for computer name settings in X.5’s Sharing preferences. But I’m sure people will come up with much worse stuff.

Core Animation further encourages such unfortunate and overly animated application behaviour by making animated transitions the default. If you have a CALayer and change its opacity to hide it,

    myLayer.opacity = 0.0;

    [CATransaction begin];
    [CATransaction setValue:[NSNumber numberWithFloat:0.0 forKey:kCATransactionAnimationDuration];
    myLayer.opacity = 0.0;
    [CATransaction commit];

In Practice

There’s no better way to learn about things than actually using them. Which is how I met - and clashed with - Core Animation. When implementing Symmetries, it seemed natural to use Core Animation. The application’s on-screen display consists of different layers:

1. The (white) background
2. The stroked (black) and filled (grey) path
3. The (blue) guides indicating where you can drag things
4. The (yellow) highlight for the point that is moving
5. The (red and blue) anchor points and handles
6. The help layer

For that alone having a convenient way of splitting things up into layers some of which (e.g. the guide layer) can be shown and hidden on demand seemed a very cool thing. As the demo and animation modes also animate the graphics on screen, the original plan was to use Core Animation for that as well. Unfortunately most of those plans failed and only the help layer is displayed by Core Animation in the final application. The following sections contain a few lessons I learned while doing this. Feel free to learn from them or to mention a better way of doing the same thing in the comments.

Lesson 1: Core Animation vs. Custom content

While it’s dead simple to move Core Animation layers around on screen, setting the content of those layers seems a bit odd at first. Layers do have a content property which can be used to set the layer’s content to a CGImageRef, i.e. a data type which seems rather foreign in Cocoa. You can convert an NSImage to a CGImageRef with a few lines of code and I think the comment in the NSImage category found in Scott Stevenson’s simple ArtGallery Core Animation demo project expresses this well:

- (CGImageRef) cgImage
{
    // I'm open to better ideas. :) 
    NSData* data = [self TIFFRepresentation];
    return CreateCGImageFromData(data);
}   


// from http://developer.apple.com/technotes/tn2005/tn2143.html
CGImageRef CreateCGImageFromData(NSData* data)
{
    CGImageRef        imageRef = NULL;
    CGImageSourceRef  sourceRef;

    sourceRef = CGImageSourceCreateWithData((CFDataRef)data, NULL);
    if(sourceRef) {
        imageRef = CGImageSourceCreateImageAtIndex(sourceRef, 0, NULL);
        CFRelease(sourceRef);
    }
    return imageRef;
}

Thus, if performance does play a role for you, you probably want to set up a delegate object for the layer in question and implement the -drawLayer:inContext: method there. This lets you draw things straight into the correct graphics context and will not create all those superfluous objects on the way. To make things ‘easier’, it seems you need to pimp the CoreGraphics graphics context to give a Cocoa graphics context before drawing:

- (void)drawLayer:(CALayer *)layer inContext:(CGContextRef)ctx {
    [NSGraphicsContext saveGraphicsState];
    NSGraphicsContext * graphicsContext = [NSGraphicsContext graphicsContextWithGraphicsPort:ctx flipped:NO];
    [NSGraphicsContext setCurrentContext:graphicsContext];

    /* YOUR DRAWING CODE HERE */

    [NSGraphicsContext restoreGraphicsState];
}

In short: This does work in the end, but it may not be the most obvious thing.

Lesson 2: Performance

As mentioned in the previous section, using the wrong implementation can easily degrade your performance significantly. While moving from setting the content property to implementing -drawLayer:inContext: in the delegate doubled my frame rate (from 25 to 50 frames per second), the result still wasn’t good. Somehow the updates felt nowhere as immediate as those I got when drawing without Core Animation (at 60 frames per second). I don’t think that the extra frames per second played a relevant role there, but rather it seemed to be some delay coming from Core Animation itself.

I have no idea how to measure this, but I certainly ‘felt’ the difference and so did some of my testers. Which led me to think that Core Animation may not be the right tool for timely screen updates where you want cursor movements and redraws to be matched as tightly as possible.

Quite likely these delays don’t play a huge role when doing ‘imprecise’ Cover Flow-style interfaces or screensaver, but they were certainly a deal-breaker for me. Hence, Core Animation was out of the game for performance reasons.

Lesson 3: Does not Animate

This wasn’t entirely clear to me from the beginning: Core Animation is severely limited in what it can animate. The ‘animatable properties’ are restricted to properties related to layer positioning, appearance and the effects applied to layers. In particular - despite its name -, Core Animation is not made to animate your own arbitrary properties.

Allegedly Core Animation was called LayerKit before marketing got their dirty fingers on it. And seeing this limitation of Core Animation, makes the original name seem much more precise and less filled with broken promises. The framework is a tool for juggling layers around and not for animating arbitrary things.

Luckily, there is also the NSAnimation class which was introduced in Mac OS X.4. It ended up being the star I needed to save the day when realising that the magic of Core Animation doesn’t suffice to make the Demo and Animation in Symmetries work.

Conclusion

With all these problems, I ended up using traditional Cocoa view drawing in Symmetries, used NSAnimation to do the interesting animations and all that was left over for Core Animation to do was the Demo mode where panels with help text slide in above the main view. As should be clear by now that’s pretty much what Core Animation was made for, so it was reasonably easy to get it to work.

Yet, even this wasn’t 100% as simple as I would have liked it to be. My help pages slide in one by one, and it took a bit of care to make sure that resizing the window doesn’t reveal things it shouldn’t. Likewise, I found the positioning and scaling magic Core Animation uses somewhat confusing and it’s still not really clear to me whether it’s my or the system’s responsibility to trigger the drawing of a layer. Things didn’t work quite as automatically as I expected them to there.

In general the installer and software update utilities on Mac OS X work reasonably well, although a few things could probably be said about things like the transparency they offer (NOT). Those who like poking around may enjoy looking at files like /var/log/install.log or the contents of the /Library/Receipts/ folder. In Mac OS X.5 it contains a huge SQLite database in the db/ subfolder as well as all the BOM files in the boms/ subfolder.

find /Library/Receipts/boms -name "*.bom" -print0 | xargs -t -n 1 -0 lsbom -p F | grep "^lsbom\|foo"

There are two things which always strike me as odd about OS X’s software update feature: The first is it’s speed. Or rather the lack thereof. While there are all these neatly organised files and databases on the system and Apple usually have no problem serving files quickly, for some reason the Software Update application tends to need ages to determine whether or not new updates are available. Which is both odd and a bit inconvenient. It means you may get to see the following window longer than you wanted to:

This also gives you the first opportunity to wonder about the strange user interface Apple designed for the Software Update feature. Why a sheet for the progress indicator? Where does the spacing of the progress indicator come from? And don’t get me started on the fact that Apple reasonably localise ‘update’ as ‘Aktualisierung’ for their German software - except for the places where they use the word ‘Update’ or ‘neue Software’, that is. in Englisch is also what I consider dumb use of German (usually seen when stupid/lazy journalists translate English press releases) which made it to Apple’s German GUI here. But the the real insult is that they offer a link to non-localised legalese, I suppose.

Once the wait is over you get a reasonable list of the updates that are available and can read extra remarks on the selected one. Apple being Apple, said remark is void of information in the best case and void of information with a generous dash of marketing diarrhea in worse cases.

Depending on the requirements of the updates that are listed, the machine may or may not need to restart after the installation has finished. I tend to find the actual effect of this hard to predict. Occasionally the machine asks you to log out immediately and will only run the installer after the user has logged out. In other cases Software Update will install the new software and tell you the machine needs to restart after it did its job.

In that case you can simply ignore that window for days, or you can force quit the Software Update application, or - which happens to me quite regularly even when I intend to restart - you can use the Restart button and the launch a new application while the system is busy quitting existing applications. This will simply cancel your logout process and you got rid of the restart nag.

I tend to find this a bit worrying. My intuition is that the vast majority of software should be installable without a restart being required. However, bad planning on the part of the programmers or the danger of confusing the user may result in situations where restarting the machine is a simple way to ensure that no running applications keep using an old version of a library. Currently the system doesn’t make this distinction clearly enough. Rather, it gives us updates which absolutely force you to restart (by logging you out before starting to run), updates which tell you you need to restart (but don’t force you and, in my experience, don’t cause problems when you don’t) and the good updates which don’t mention the word restart at all. It’d be good if the second kind of updates vanished - preferably by turning them into the third kind.

After telling the machine to run the updates, you usually have to enter an administrator password. That’s a good opportunity to sneak in a screenshot illustrating a few interesting design decisions / bugs in OS X: First the absolutely abysmal support for custom keyboard layouts in password fields (which always switches me back to the German keyboard layout, seen as the flag in the title bar from my custom one and is buggy in about a million other ways as well). Second The password window always using the Aqua blue selection colour for text rather than my preferred shade of orange. This hints that the window is not owned by me and is not using my preferences. Which - apart from the visual distraction - makes me wonder whether that’s a good design decision security-wise.

Once that’s done the updating can begin. As soon as you are installing more than a single update in one go, the progress bar displayed can serve as a really bad example for progress bars. It simply gives no indication about how par the update proceeded so far. It doesn’t even monotonously increase but jumps back and forth during the run. A note that the update ‘will be finished in a few minutes’ would be more useful I think.

After the updating has finished things are really strange as well. The old big software update window appears again, and the sheet pops out telling you it’s checking for new updates. As far as I can tell that is because X.5 Software Update mechanism isn’t sophisticated enough to predict whether or not the system will qualify for further updates after the first round of updates has been installed. Not extra painful, but not particularly elegant either.

In particular I find the window management of the GUI embarrassing. There could be a single window which guides you through all of this and possibly smoothly resize to and appropriate shape. Instead there are different windows which are switched around harshly and there is an additional sheet for progress indication while checks for new updates are taking place. To me that seems unnecessarily complicated and distracting. If no new updates appear after that check, you are dumped back into the main Software Update window. As the window’s main button is inactive now, you’ll have to figure out that there’s nothing more to do and that you have to hit the window’s close button.

As a consequence I think Software Update would be improved consideraby if it came in a single window which handles all the tasks smoothly and without opening extra windows (save the authentication one), sheets or even modal windows. It will also offer clear and correct information about its status in a consistently localised way.

Getting this right doesn’t look like an overly complex task to me and I wonder why Apple don’t do this properly. Updating the machine can be a risky thing to do. With more and more ex-Windows users using the Mac, there may even be plenty of users these days who are weary and scared of running updates at all. Both for their benefit and my snobism (mainly the latter, though) it would seem advantageous to make the experience of Software Update a totally straightforward and pleasing one.

You are invited to add your pet loves and hates at the end.

XCode

XCode has become faster and more powerful in its third incarnation. The slowness of Cocoa’s text engine is pretty much forgotten and XCode can now open and syntax-colour large source files quickly enough that an onlooking Linux person doesn’t have enough time to recite the benefits of emacs along with a full list of key bindings. That is an extremely welcome improvement.

Just like code folding is. Having it invalidates excuses for not having lengthy comments, say, and it can simplify understanding structures and problems on a global level. Unfortunately XCode 3 seems to have shipped so immaturely that code folding is wildly incomplete and buggy. A rather obvious problem is that red or black markers in the editor’s scroll bar, which indicate compilation problems, appear in the wrong position when lines have been folded above the error. Another is that the foldings aren’t reliably persistent which they certainly should be if they are to be useful.

A neat new feature of XCode’s editor is its ability to ‘refactor’. In a way I am a bit disappointed by this because I used to think that ‘refactoring’ is a big computer sciency thing where you re-think your application and re-design its data structures for a new version to make things better, more efficient and fantastically future-proof. In XCode’s terminology (and quite possibly in reality - my respect for all programmers who spent a hard night ‘refactoring’ just dropped!) refactoring means that you re-name some variables. Probably all for the sake of consistency, beauty and future-proofing as well - but somewhat less spectacular than what I had in mind.

That said, it is a wonderful feature. You click a variable in your code, it will become underlined and an arrow button appears to its right (most of the times, anyway). That button will contain a menu which activates ‘refactoring’ and after selecting it, any change you make to the variable’s name will be copied to all other occurrences of that variable name as well. In effect this means that you are much more likely to end up with good variable names simply because replacing the bad ones you initially put there is easy.

Just be warned that XCode’s interpretation of the Edit all in scope command is a bit - ahem - idiosyncratic: When I define a variable inside an if branch, I (and my debugger) consider the scope of that variable to be that branch only, XCode’s refactoring, however, will happily rename all other variables with the same name inside your method but outside that branch as well. I rate this somewhere between ‘not helpful’ and ‘dangerous’.

The new XCode version also cleaned up a few other things to make your work more streamlined. Compilation errors will now show up amidst your code, the Run and Debug modes work more similarly and the debug mode has generally been improved. Most notably you just need to hover over variables in your code while debugging to see their value and inspect them. Thousands of clicks and sighs will be saved this way.

XCode’s breakpoints gained a GUI for things like playing a sound or logging some string or backtrace as well as the ability to have Global, that is cross-project, breakpoints. That’s great for those of us who program occasionally and forget whatever little they know about the gritty gdb commands in between. That said, the GUI implemented here is far from perfect once again:

After just a few uses I noticed that: (a) you cannot use the scroll wheel in the breakpoint list when there are expanded items with text fields in them, (b) switching between the different actions like Log, [Play] Sound &c deletes the settings you made in your previous choice; that’s very inconvenient and bad for experimentation, (c) no Undo (but at least Copy and Paste work, so it’s still better than an iPhone…), (d) no Duplicate command, (e) after expanding a breakpoint by clicking the triangle at the left of the breakpoint you always have to click the ⊕ button at its far right side; that’s annoying; relocating the buttons or simply letting a double click in the area have the same effect would be handy.

XCode’s help now seemingly gained the ability to automatically subscribe to documentation updates, download and install them (in what seems to be universally considered the most resource hogging background process Apple gave us so far) and thus save you the hassle of that login, download and install process.

Another addition -  which may well end up being a star of XCode -  is the Research Assistant, an inspector window that displays information and documentation about the method, variable type or function your cursor is currently on. It’s a wonderful idea which could be tremendously useful if it were well implemented. But it’s still quite far from that.

Even in my shallow experience the Research Assistant is one of the main factors making XCode 3 terribly sluggish - waiting a full second before seeing what you typed appear on screen sluggish, that is. This needs major improvement. Allegedly there are cutting edge technologies like multi-tasking or multi-threading which let applications look up one thing without stalling your typing while doing that. Let’s hope Apple catch up with those technologies at some stage. The Research Assistant also isn’t terribly good at displaying information for many things it should know about. Specifically it doesn’t display information when you click into a child part of a struct or Objective-C 2 property statement or when you click into one of the methods of your own classes. - Nice idea, but not really there yet.

XCode’s autocompletion improved as well. But it remains ridiculously bad. I don’t think I ever used an autocompletion feature which so consistently managed to suggest the most useless autocompletion when several possible completions are available. Making this perfect should be hard but a bit of work might lead to considerable improvements already: XCode has a lot of information at hand, it knows about Apple’s data types and the functions/methods belonging to them, it knows about your classes and it knows the code you use close to where you are typing. Humans and even dull logic will probably manage to complete [bezierPath moveToPoint: NSMak in a useful way - a task which is already too hard for XCode’s autocompletion. And this is just a random example I had on screen at the moment. There are many many others you will come across.

I am lazy, so I really want to use autocompletion, but with XCode’s - as it is today - I frequently think that it would have been quicker to type things myself than to wait for the autocompletion suggestion to appear or to fight XCode to provide me the correct autocompletion. This is even harder and frustrating because even after using it for a while I am unable to predict what hitting the tab key will do. You cannot develop ‘muscle memory’ for the autocompletion for (at least) two reasons: The first is that you cannot simply type a few letters and hit tab. You have to wait for a moment before hitting tab until XCode has caught up with you. That costs time and ‘flow’. And then, XCode seems to try to be ‘clever’ about the default completion it suggests. Which could be a good thing. But somehow I found that in most cases it’s just not clever enough and hands me the wrong completion.

Objective C 2

Objective-C - the best programming language since Basic or Pascal - got a version bump as well. And it surely resolves the single thing Objective-C lacked in comparison with RealBasic: Garbage collection. While, personally, I think that Objective-C’s retain-release concept really isn’t so bad once you got the idea, leaving out all those explicit retain, release or autorelease calls will certainly make programming more fun and less of a hassle.

I’m sure there will also be situations where having proper garbage collection will solve real memory management problems. Just as there will be situations where it will hurt performance. In fact, I am fairly sure that I already observed such a situation in a fun little project where I ended up using 60MB of extra RAM while displaying an animation. Probably just being able to release the images in question as soon as you don’t need them anymore rather than waiting for the garbage collector to feel like doing the job could avoid this generous waste of precious RAM.

Properties look great as well because - as big as my love for brackets may be - it’s quicker to type than accessor or Key-Value-Coding methods. I’m lazy in that way. XCode’s autocompletion still mostly sucks when you are using properties and I am not sure whether I fully grasped the concept and its consequences yet. (What’s the cost of using a property? Does it really make sense to access a class’ instance variable via self.myInteger? I keep thinking that it may be the ‘cleaner’ version of doing things but that in many cases it’s probably just an absurd waste of CPU cycles - i.e. global warming. So what’s the responsible thing to do?) My lazy self is also a bit disappointed that setting up a property requires three lines of code in three different locations. I was thinking more about a single magic statement in a single place. As XCode’s ‘refactoring’ also fails to make renaming properties easy by offering to change both the source and header file and listing all other places where the property is used by files (code and nibs) in the project, this isn’t entirely convenient.

Objective-C 2’s new looping techniques are very welcome as well. I doubt that anybody will miss having to type NSEnumerator all over the place. I certainly won’t.

Interface Builder

One of the more problematic parts of X.5’s developer tools is the completely reworked Interface Builder. While re-engineering and re-writing the ancient Interface Builder application to better match today’s Cocoa environment with things like Bindings and the iPhone runtime environment around is most likely a good idea, the application itself seems terribly under-engineered and prematurely released to me.

Somehow the new Interface Builder seems to require a lot more screen real estate than its predecessors with its huge, overly animated and split viewed Library palette being the most obvious problem. Just try creating a medium sized window on a MacBook screen. It will end up being frustrating because you’ll either have the Library palette covering your window or you’ll have to fight the battle between the Library and Inspector palettes about who is on top. That’s not very convincing.

Interface Builder’s - just like XCode’s - New File window is a matter of bad design as well. Rather than carefully using the space in that huge window to make all relevant items that can be created clearly visible, you’ll easily find yourself in a situation where you see scrolling views and many subitems which makes things look much more complicated than they really are.

Other things that regularly drive my nuts in the new Interface Builder are that it seems even worse at dealing with localisations than its predecessors (easy switching between different language versions of the same UI element would seem like an obvious thing to do), that it makes it ridiculously non-obvious how to instantiate one of your own classes in a nib file and that its bindings editor still sucks full time (no good preservation of what you enter, no help autocompleting key paths and no friggin’ way to make the Inspector window wide enough to even see your full key path, FAIL, FAIL, FAIL) which makes other points like the new icons for nib and xib files that simply don’t stand out in your XCode file list and thus needlessly hard to find seem minor in comparison.

Uh, and Interface Builder’s pseudo HUD, small, overly auto-scrolling action method selection window is probably something for which you want a copy of the HIG engraved in some hard material just so you can slap the people who implemented this with it.

Documentation

Apple have a proud history of shipping all that nice Cocoa stuff and giving us sub-standard documentation and examples along with it. My impression is that this situation has improved a little but hasn’t changed fundamentally yet.

The problem with this is that it can waste a lot of time or create sufficient frustration to simply discourage people - particularly those of us who’re only in this for the fun - from trying things. In many places of Cocoa the behaviour of the framework is un(der)-defined by the documentation. Be it because Apple are too stingy to have more exhausting documentation written as they don’t care about their developers or because they don’t know themselves. Giving Apple the benefit of the doubt one could hope that the people working there and making the frameworks actually know those details and can explain them. Then please make them share that knowledge, it will save time over and over again around the world!

Then there are small things in XCode’s documentation browser which could be smoother. Say, when I’m entering NSMenuItem into the search field I keep getting the legacy NSMenuItem protocol as the first result. I am even warned about this being a deprecated way of doing things and (after scrolling) I’ll get a link to the new class, but wouldn’t things be, um, useful if the Help gave me the thing I’m looking for right away?

Another aspect of documentation - which is quite ‘in character’ for Apple, though - is that it’s not very candid about what doesn’t work. This kind of information can be tremendously helpful as it keeps people from wasting time trying things that are hopeless. What about a comprehensive guide to all Cocoa controls indicating which of them can realistically be used with Bindings, say?

The final point are examples. Many people, including myself, find examples a very valuable form of documentation as they don’t just theorise about the frameworks but show you how things are done. With a bit of luck you can even nick some code from them. In some areas (Quartz Composer, Audio?) the examples seem to be quite comprehensive and give you a good starting point for not just using those technologies in a simple way but really digging into them. In other areas, however (CoreData and Bindings come to mind), the examples are far from covering the whole problem space. I think that having more of them could make learning the underlying technologies much easier (particularly for ‘magic’ technologies like CoreData which you are not going to understand fully).

Code Signing

MacOS X.5 is full of technology for code signing. It has good and bad sides. Let me discuss what it does first.

The essential idea for code signing is to authenticate an application. A developer can create a certificate and sign his application. While - just as with encryption certificates - it takes quite a bit of extra effort to make sure such a signature really authenticates the developer, this at least gives you a way to determine whether two different versions of an application originated from the same developer (modulo the developer’s incompetence at handling his certificate, that is).

Mac OS X.5 already uses this in a number of ways. The best one of them is the keychain. If an application has been signed by the developer and you allow that application to access your keychain, the keychain will now remember not just the application and its version but the developer’s certificate. If you download an update for the application, the keychain will recognise that it has been signed by the same developer and the application will be granted access to the keys without further ado. This means there is one less annoying dialogue asking the user for permission to access the keychain. Which makes it more likely that users will take those requests seriously when they do appear. It’s a good thing.

Other areas where Mac OS X.5 uses code signing is for its firewall and its nanny mode. In the latter, code signing is used more destructively: Rather than letting people execute any code, only certain ‘trusted’ applications which are pinned down by the signatures of their developers (or the signatures OS X.5 adds if there is none) are allowed to accept network connections or to be used by the kids.

Obviously things easily become a bit creepy once we venture in this direction. For example because of the automated code signing by the OS, applications can be modified without you really knowing about it. And obviously once you have these technologies in place for the good intentions they are also easily available for bad ones.

I suppose the iPhone is a prominent example for this. Essentially it’s Apple’s decision whose code can run on the iPhone. That toy is not a computer at all but a highly controlled environment. Apple dictates its rules and it will be interesting to see when the first applications are found to be against Apple’s arbitrary terms and conditions and will be banned from iPhones. Would instant messaging be fine? Would route planning be OK for third party software on the iPhone or did Apple make an ‘exclusive’ contract with some software company for that already? Is porn in or out? And if it’s out, who gets to decide what porn is? Is VoIP OK? Or an application displaying The Guardian? Or one displaying Fox News for that matter? Do we really want someone else deciding which software may run and which doesn’t?

Well, the iPhone and Mac OS X.5 have the technology to censor away. Currently it’s a mere possibility but what will happen in X.6, .7, .8? We don’t know. The code signing doesn’t seem to be fully documented yet (another aspect which makes this a bit creepy) so the full power isn’t entirely clear at the moment. But it appears that using the codesign utility you can sign pretty much everything from a Unix command line tool to a OS X bundle. And the detailed settings appear to give finely levelled control about which parts of a bundle are covered by the signature. When will applications start trying to refuse running when their signature isn’t matched? When will frameworks which Apple ship but don’t want other applications to use simply fail to initialise when the application using them hasn’t been signed by Apple?

We’ll have to see how this plays out. If I am not mistaken they have done a similar thing on Windows for ages with drivers. Whenever a hardware company doesn’t cough up the money for Microsoft’s certification and a driver is installed some dialogue box comes up asking people whether they really want to continue installing an unsigned driver. I haven’t watched many Windows users but none of those I saw seemed to even register that warning, they just dismissed it because that’s the natural thing to do. Which to me means that if signing technology is to have any practical effect it has to be drastic. Things need to not work when they are not signed. And that’s a bit of a scary prospect as it makes a completely walled in computing experience - one that one wouldn’t consider a computing experience at all in some classical sense - seem quite realistic.

Interesting question: Does XCode support convenient code signing yet? There is a field for entering the key to use for singing in the build target inspectors. But entering something there never did much for me. When compiling the project with the xcodebuild command, however, it looked like a signature was created.

Backups

Backups are a sore topic. On the one hand, thinking about them is a painful and dull endeavor. On the other hand, not having a backup when shit hits the proverbial fan can be a disaster. The most important thing on a computer are your data. It’s not the machine itself. It’s not the operating system. It’s your files you will be worried about. Your work, your collected resources and work environment, your e-mail of the last decade or two, your photos and possibly your music collection or the playlists you created from it. Many of these things cannot be re-created if they were lost others would demand a massive effort to re-create which you wouldn’t undertake because it exceeds the ‘worth’ of the data in question.

Unfortunately many people aren’t aware of all this. They work with their data every day and spread it across machines and systems without much care. Parts of their e-mail will be at some web mail provider, other parts of their e-mail are still at their old web mail provider - each provider having a different state and subset of their address book. Phone numbers are stored in the mobile phone only. And documents reside in the folder the application creating them suggested as a default. The documents created after 2005, that is, because back then old computer with all the older documents on it was stolen…

Often people don’t worry too much about this, simply because they aren’t obsessive compulsive. But everybody seems to have weak points. Once the half finished dissertation or the wedding photos go missing because of a broken hard drive, careless operation or theft, panic strikes. And the count of people who become aware of backups increases.

Unfortunately, making backups is a bit of a hassle and thus of the few people who have a ‘backup strategy’ even fewer actually stick to it for a long time. It’s tedious, it’s boring and there is no day-to-day benefit from it as long as nothing goes wrong. Without ease-of-use and automation, backups don’t happen.

Once you start thinking about it you will discover many potential points of failure for a backup solution. Things you need are:

• Awareness of the need for backups.
• A mechanism that is easy and foolproof to set up.
• Automatically scheduled running of the backup that works reliably.
• The ability to read and recover your backup when your data are lost.
• Bonus: The ability to restore your data in a quick and foolproof way to a working condition. By Murphy’s Law you are going to be in a stressful tight-schedule situation when you need to do so.

None of these points is trivial to achieve. And - optimistically speaking - Time Machine is a brave attempt to address all these problems: Its mere existence raises awareness about backups. It creates and maintains backups automatically. It does so in a relatively simple way. And it makes restoring data easy.

Apple and Backups

All that said, Time Machine is also a product by Apple. And - as far as I can tell - Apple never went out of their way to protect their users’ data. The quality of their applications’ backwards compatibility varies widely (the OS X Address Book will export contacts [up to the photos associated to them] to the standard VCARD format, OS X Mail has no export feature at all, Mac OS X.5 doesn’t voluntarily play System 7 sound files, …). If Apple were self-confident enough to worry about their users’ data more than about locking their users in to peddle more machines, I’d certainly start off with a more positive impression.

While Apple’s software isn’t generally destructive, they have had their run of installers deleting users’ files and they also had the .mac Backup application gaining a reputation for not being something you’d want to entrust your data to. There were reports about people being not being able to restore data from backups (i.e. failure at its single most crucial function) a few years back. That seeds a bad reputation which is deadly for a backup tool - particularly for one by a tight lipped company which will neither admit errors nor document the changes they make in updates.

And from a good friend of mine I heard a possibly worse report about .mac Backup. He had a hardware failure of his hard drive. Which he wasn’t too concerned about because he had run the .mac Backup application all the time. He got a new drive and restored his data. Just to find that all his photos and iTMS purchases were there. But, the most essential data, his documents, were missing. Panic ensued and a large amount of money had to be spent for hard drive recovery to rescue his thesis.

Idiot! I hear you scream. Sure, he should have checked the recoverability of his data. But keep in mind that .mac customers are exactly the people more technical minded folks consider as ‘idiots’. If Apple tells them they have a backup and essential parts are missing at the end of the day, it doesn’t exactly build trust. It wasn’t possible to reconstruct how exactly that situation came to be. But even if he turned off backing up documents himself, the software should have told him about the implications. Just as it warns you before erasing a drive.

As a consequence of all this Apple will have to earn the trust needed for Time Machine as a viable backup software. And while a lack of actual experience with the technology as well as initial software problems still dominate the reports at this stage, I think they have a reasonable chance at succeeding in the long run.

Which I wish they do. Simply because it will help people’s data survive.

Background activity

There are two aspects to how Time Machine works: The user interface and the background activity needed to do the actual magic. Both of them are probably of equal importance as only a good user interface will entice people to create backups of their data and enable them to recover data which would be lost otherwise. But without the behind-the-scenes work nothing is going to happen. To keep up the good spirit, I will dig into the more obscure topic of background activity first before bitching about the user interface. This first part will be both lengthy and slightly technical. If you can’t take that skip it.

Ideas about what a backup is vary a bit. The most obvious - and very widespread - interpretation of the word is simply making a ‘clone’ of your hard drive which you update from time to time. Obviously that’s better than nothing but once you work with, or think about, backups, you soon realise that it’s really not enough. At least not if you want to be able to do more than just restore your system in the case of mass deletion or a hardware failure. Say, some data corruption happens to a file, but you don’t realise it at the time. You’ll run your clone ‘backup’ that evening and your only existing copy of the file’s previous version will be lost. Ouch! Suffer from this once and soon you’ll join the people who consider a backup to be an initial copy of all your data plus fresh copies of all files that have been changed since the previous backup run. Time Machine provides just that and I shall try to give a rough sketch of how that works (note for the overly optimistic: I have neither written Time Machine myself nor seen its code, so I can only go from observation of Mac OS X.5 and the documentation).

First, let’s look at the data storage: As things are, Time Machine requires a HFS Extended file system to work. This has been the default format for Mac hard drives for a long time, so it’s not a big deal. But if you don’t have a Mac OS Extended volume (but a drive shared by another Mac OS X.5 computer via Apple’s File Sharing or a drive in a different format), things will work as well as Time Machine creates a disk image in HFS Extended format and saves your data into that if necessary. In what follows I will just assume we are dealing with a HFS Extended volume.

At the top level of that volume Time Machine creates a folder named ‘Backups.backupdb’. And inside that folder there is a subfolder with the computer’s name - meaning you can back up different computers to the same Time Machine volume. Inside that folder there is a number of folders whose names are time stamps. These folders include the data of the drives Time Machine backed up at that time.

Hard links

Unlike many other backup solutions Time Machine doesn’t store backups in a single file backup database which may have an unknown format and may make it hard to extract data from it without the software that created it. Instead, Time Machine uses ordinary folders and files. That may be a bit less efficient space-wise but has the advantage of being simple as well as browsable on any machine that’s able to read HFS Extended volumes without Time Machine at hand.

Time Machine’s trick to avoid duplicates of unchanged files in its archives is to use hard links. Hard links are a very old and very basic concept. Yet they can be a bit mind boggling. When looking at how a file system works, you end up discovering that there are two things: The hierarchical folder structure and the files themselves. A folder essentially contains nothing but a bunch of references to the files themselves rather than the actual files: You can move even a huge file from one folder to another on the same volume in a split second because all that happens is that a reference to the file is inserted in the destination folder and it is removed from the source folder. The data themselves remain in the same place throughout.

In fact, that reference inserted to the folder is just a hard link. The new thing is just that once you have several such hard links to the same file, the file is listed several times in the file system’s folders but the actual data only exist exactly once. And that’s exactly what Time Machine uses.

Just to get a better feeling for this, let me answer a few questions I wished you had asked:

What is the difference between hard links and the ‘symbolic links’ Unixy people normally use?

The difference is that a symbolic link points to the path the file is located at. That is, it only points to the entry in the folder hierarchy rather than the ‘actual’ file. [Perhaps the image below or the next question clarifies this.]

Uh, what? You’re talking gibberish.

Just imagine the following situation: You have a file and a symbolic link to it. Then you delete the original file. As a result the symbolic link will point to nothing. And then you move another file to the same path that the original file was at. The symbolic link will then open that file. None of that could happen with a hard link. It always points to the same ‘actual’ file. Essentially it ‘is’ the file.

If ‘hard links’ are so great, why aren’t we using them all the time?

While they are great for many purposes, hard links also have a number of drawbacks. Most notably they can only point to things on the same volume.

How do Mac aliases figure in this game?

Mac aliases are more like symbolic links as they are separate files containing data about the location of the file they reference. However, aliases can contain information about the location of the ‘actual’ file and thus be much better at finding files even after they have been moved than symbolic links storing a path reference can hope to be.

One way in which Time Machine goes beyond what is usually done is that it makes extensive use of hard links for folders. These are usually avoided as they require very careful handling in situations when multiply hard linked folders are nested within another. But the feature is essential for Time Machine so it can just use a single hard link as a reference for a folder and everything it contains instead of having to create actual folders for everything and hard links for every single file in each snapshot.

Having multiple hard links to a folder turns out to be problematic once you start thinking about circular references and how deleting files should work: currently you can only remove a folder from the file system if it is empty; if you have multiple hard links to the folder it wouldn’t seem unreasonable to remove one of them without the folder being empty. But then imagine the only remaining link to that folder being from an item that is contained in the folder itself: You wouldn’t be able to reach that folder from the file system hierarchy anymore! In fact, the System’s ln tool doesn’t let you create hard links to folders yourself.

Perhaps a little demonstration will be a good way to finish this off. You can play around with the relevant commands in the Terminal without much effort, creating files, creating hard links with the ln command as well as soft links with the ln -s command. And keeping a close eye on the output of the file listing command ls -l. Its output contains a mysterious number after the initial permissions (e.g. drw-r--r--) bit. That number is exactly the number of ‘hard’ links to the file in question. Looking at these a bit you quickly realise that most files have a number of 1 and a folder has a number of at least 2 [the parent folder pointing to it and the folder pointing to itself - the ‘.’ item in the listing of the folder. Every additional item inside the folder will increase that number by 1].

[kalle:~] ssp% mkdir linktest
[kalle:~] ssp% cd linktest
[kalle:~/linktest] ssp% echo "test" > originalfile
[kalle:~/linktest] ssp% ls -la
total 8
drwxr-xr-x   3 ssp  ssp   102 17 Feb 20:25 .
drwxr-x---@ 41 ssp  ssp  1394 17 Feb 20:25 ..
-rw-r--r--   1 ssp  ssp     5 17 Feb 20:25 originalfile

Up to here we created a file originalfile containing the string test and looked at the folder contents. There is a link count of 1 for the file and of 3 for the folder. Next, create both a hard link and a symbolic link to that file:

[kalle:~/linktest] ssp% ln originalfile hardlink
[kalle:~/linktest] ssp% ln -s originalfile softlink
[kalle:~/linktest] ssp% ls -la
total 24
drwxr-xr-x   5 ssp  ssp   170 17 Feb 20:25 .
drwxr-x---@ 41 ssp  ssp  1394 17 Feb 20:25 ..
-rw-r--r--   2 ssp  ssp     5 17 Feb 20:25 hardlink
-rw-r--r--   2 ssp  ssp     5 17 Feb 20:25 originalfile
lrwxr-xr-x   1 ssp  ssp    12 17 Feb 20:25 softlink → originalfile

And observe how the numbers change: The file with the hard link to it has an increased link count of 2. Finally, we delete the original file and convince ourselves that it survived thanks to the other hard link to it:

[kalle:~/linktest] ssp% rm originalfile
[kalle:~/linktest] ssp% ls -la
total 16
drwxr-xr-x   4 ssp  ssp   136 17 Feb 20:36 .
drwxr-x---@ 41 ssp  ssp  1394 17 Feb 20:25 ..
-rw-r--r--   1 ssp  ssp     5 17 Feb 20:25 hardlink
lrwxr-xr-x   1 ssp  ssp    12 17 Feb 20:25 softlink -> originalfile
[kalle:~/linktest] ssp% cat hardlink
test

Once such hard links come into play, you lose the sense of which one is the ‘original’ and which one is ‘just being pointed to’ - as both of them are the original. And thus, when browsing your Time Machine folders on the backup drive, all files will look like proper files and not carry the little superimposed arrow icon which Mac OS X normally uses to indicate aliases or symbolic links. But with the Terminal commands above you can hope to get an idea of what is going on.

When peeking around your backup you can learn a few things about it if the numbers are interpreted correctly. For example for files at the top level of your startup volume the number tells you in how many backups the file has been included. E.g.

[kalle:~] ssp% ls -l  /Volumes/Snowman/Backups.backupdb/Kalle/Latest/Kalle/ | grep mach
-r--r--r--@ 43 root  admin    616052  6 Nov 06:40 mach.sym
-rw-r--r--@ 30 root  wheel  10272820  6 Feb 01:15 mach_kernel

Suggests that the ‘mach_kernel’ file has been replaced 30 backup runs ago. The same heuristic doesn’t work as easily for folders as you’ll have to subtract the number of items in them before knowing the number of hard links around. Also be careful when examining folders lower down in the file system. You will frequently see link counts of 1 there. That happens when no item in that folder or in any folder contained by it has been changed. Then Time Machine only created a hard link for the enclosing folder and thus every file inside it has only been backed up and hard linked to once.

Extended Attributes

But there’s more to see for those who like poking around. Apple introduced extended attributes in Mac OS X.4 and Time Machine uses them a lot, both inside the backup copy and on your usual files. Mac OS X.5 comes with the xattr command line tool for reading and writing them.

Poking around the ‘Backups.backupbd’ folder you will find these attributes:

Machine Name - a top level folder with the name of the machine backed up, e.g. ‘Kalle’

• com.apple.backupd.BackupMachineAddress — containing a string of the Ethernet port’s MAC address on my MacBook and that of the Airport card on a MacBook Air .

Timestamp - a folder with the date of the backup it contains as its name, e.g. ‘2008-02-15-000458’

• com.apple.backup.SnapshotNumber - a string representing the index of the backup, increasing by one in each iteration.
• com.apple.backup.SnapshotVersion - I have only seen the string ‘1’ here.
• com.apple.backupd.SnapshotCompletionDate - sounds clear, though I haven’t tried figuring out the number format yet.
• com.apple.backupd.SnapshotStartDate - dito.
• com.apple.backupd.SnapshotState - gives the string ‘4’ for all snapshots I have.
• com.apple.backupd.SnapshotType - gives the string ‘3’ in the time range when snapshots are daily, ‘2’ in the time range when they are hourly (with a little glitch where daily and hourly meet) and ‘1’ for the latest snapshot. I’m guessing a bit here, to be honest.

Volume-Name

• com.apple.backupd.SnapshotVolumeFSEventStoreUUID - as the name suggests the UUID of the current fsevents store (see below); this matches the string in the file /Volumes/Volume-Name/.fseventsd/fseventsd-uuid.
• com.apple.backupd.SnapshotVolumeLastFSEventID - the ID of the last event reflected in the snapshot, not sure how to find that one in the files.
• com.apple.backupd.SnapshotVolumeUUID - the UUID of the volume in question, matching the one you get with diskutil info /Volumes/Volume-Name.
• com.apple.backupd.VolumeIsCaseSensitive - naturally 0 on sane volumes.
• com.apple.metadata:_kTimeMachineNewestSnapshot - a binary property list containing a Date as its root object. The date is that of the newest Snapshot containing the file or 1.1.4001 if the file is still present in the newest snapshot.
• com.apple.metadata:_kTimeMachineOldestSnapshot - a binary property list containing the date of the oldest Snapshot which contains (or once contained) the file.

Further subfolders

• com.apple.metadata:_kTimeMachineNewestSnapshot - as above.
• com.apple.metadata:_kTimeMachineOldestSnapshot - as above.

With all these extended attributes in place, Time Machine’s archive is more than just a copy of the files. It also contains guides to match the backup up with local data and to see where the previous and next versions of a file are stored.

Time Machine also uses extended attributes on your working volume to mark files and folders as excluded from the backup.

Exclusion

In the way Time Machine currently works, exclusion is a vital concept for it. The software defaults to backing up a complete volume and you can control things a little by excluding folders from the backup. Say, to ensure that huge but frequently changing files - like disk images from a virtual machine or a large database - don’t clutter up the backup too much. There are various ways files can be excluded from the backup. (Don’t read too much into the order of that list as I haven’t checked which of these items takes precendence in case of conflicts. That might be an interesting thing to do.)

StdExclusions.plist in CoreService’s ‘backupd.bundle’

This file lists many paths which are excluded from backups, both globally and in user folders. These are mostly folders for temporary files or files which can otherwise be recreated like the Spotlight index. As of Mac OS X.5.2 this file excludes the content of the folders /Volumes, /Network, /automount, /.vol, /tmp, /cores, /private/tmp, /private/Network, /private/tftpboot, /private/var/automount, /private/var/log, /private/var/folders, /private/var/log/apache2, /private/var/log/cups, /private/var/log/fax, /private/var/log/ppp, /private/var/log/sa, /private/var/log/samba, /private/var/log/uucp, /private/var/run, /private/var/spool, /private/var/tmp, /private/var/vm, /private/var/db/dhcpclient, /private/var/db/fseventsd, /Library/Caches, /Library/Logs, /System/Library/Caches, /System/Library/Extensions/Caches, the files and folders /.Spotlight-V100, /.Trashes, /.fseventsd, /.hotfiles.btree, /Backups.backupdb, /Desktop DB, /Desktop DF, /Network/Servers, /Previous Systems, /Users/Shared/SC Info, /Users/Guest, /dev, /home, /net, /private/var/db/Spotlight, /private/var/db/Spotlight-V100 and for each user account the files and folders Library/Application Support/MobileSync, Library/Application Support/SyncServices, Library/Caches, Library/Logs, Library/Mail/Envelope Index, Library/Mail/AvailableFeeds, Library/Mirrors, Library/PubSub/Database, Library/PubSub/Downloads, Library/PubSub/Feeds, Library/Safari/Icons.db, Library/Safari/HistoryIndex.sk. I knew you’d find that interesting.

com.apple.TimeMachine.plist in global preferences

This property list contains two items of interest: ExcludeByPath which seems to accumulate items like ‘Library/Calendars/Calendar Cache’ or ‘Library/Mail/Envelope Index’ for users that used them (why isn’t the first one included in the previous list? Why is the second one in both?) and SkipPaths which contains a list of all items the user dragged to the exclude list in Time Machine system preference pane.

extended attributes

Applications can also mark files as not-to-be backed up. Apple’s Backup Core provides an API for that. The effect that seems to have is adding an extended attribute “:EA:com_apple_backup_excludeItem” to the file with the content “com.apple.backupd”. A Spotlight query for com_apple_backup_excludeItem = "com.apple.backupd" should reveal all files excluded from backups in this way. Currently it seems to be mainly used for the iTunes Music Library.xml file.

While there certainly is a point to many of the exclusions made by Time Machine, I do find them slightly problematic. Particularly as many people will not know they exist and it is non-obvious how to find out which files will not be backed up. Even more so, as applications can easily remove your documents from future backups by accident (or, maliciously, on purpose). It’s a shame that there isn’t an easily accessible list of files which aren’t backed up. I guess the sufficiently paranoid may want to keep an eye on the property lists and attributes mentioned above.

A point that is rather sore as well for me is the fact that Caches folders are completely excluded from backups. By Spotlight’s poor design the Caches folder contains a Metadata folder which applications have to store their data in if they want it to be found in the index. Those metadata files will not be in a backup. As a consequence restoring your system from a backup will leave you with an incomplete Spotlight index until you run all the applications which stored data in the Caches/Metadata folder and make sure they re-create them. Ultimately this is a Spotlight issue, I think, but with Time Machine being made by the same company, they should have had an eye on it.

Spotlight

Time Machine also ties in with Spotlight. Items in the backup are indexed by the Spotlight engine to help you find stuff when needing it. The index for the Time Machine backup is stored separately from the remaining index of the volume, indicating that Spotlight has become a bit more flexible in this regard. I suppose separating things in this way may also improve performance for everyday searches in the non-Time Machine content of the volume.

An interesting aspect is that extended attributes beginning with com.apple.metadata can also be found in the Spotlight index. In particular, this gives you a way to look for all files added to the backup in a certain time range. So if you sometimes wonder ‘what the heck caused my last backup run to take so long and to contain that much data, Spotlight might be helpful.

If Spotlight’s graphical user interface weren’t a complete POS, this could even be fun. But currently it’s a bit painful to do. And given Apple’s past speed of improving the Spotlight UI, I won’t expect considerable improvements there before 2050. But still, we can attempt to do it. Just be sure to take this with a lump of salt. I have seen loads of strange behaviour when testing this. ‘Strange’ behaviour included the Spotlight server process mds crashing and happily starting a complete re-index after being automatically restarted by the System. It also included recently backed-up files not being in the Spotlight index and thus not appearing in the results for these searches. All that seemed a bit random, so while in principle possible, I wouldn’t rate these steps as reliable but more as an interesting exploration at this stage.

1. In the Finder, navigate to the ‘Backups.backupdb’ folder on your Time Machine backup volume.
2. Use the Find command
3. Switch the scope of the search to the ‘Backups.backupdb folder. This switch will ensure that the Spotlight index for the backup is used instead of the general one.
4. Add a ‘System Files’ search criterion and set it to include System files. This ensures that Mail messages and files from the Unix innards which the Finder prefers to ignore are displayed as well.
5. Add a ‘Raw Query’ search criterion (still localised as Reine Daten in German - whatever that is supposed to tell me).
6. Enter a raw search query string like _kTimeMachineOldestSnapshot > $time.now(-3600) into the field in the Finder window. [This didn’t work when I initially tested it and the text used to contain a more complicated way to create a query. Thanks to tinbert for pointing out that this does work after all and encouraging me to give it another try.]
5. Determine the number of seconds between an hour ago and the beginning of 2001. Unless you have something like an NSDate handy this will suck. Luckily thanks to Cocoa’s new ‘scripting bridges’ in Mac OS X.5, you tend to have an NSDate handy. So I invested two lines of Python to get this number [note that (a) I don’t know any python, (b) probably some smartass can do the same with less resources in bash or ed or whatever and (c) Google Calculator apparently can’t evaluate ‘seconds since 1.1.2001’]

   #!/usr/bin/env python
   from Foundation import NSDate
   print NSDate.timeIntervalSinceReferenceDate() - 3600
   

   [Download a version for the current time or one for an hour ago if you’re too lazy to copy and paste.]
6. Enter a raw search query string like _kTimeMachineOldestSnapshot > 225450653.661 into the field in the Finder window. Where the lengthy number has to be replaced by the number you determined in the previous step. This is yet another good opportunity to curse the people who thoughtfully ‘designed’ the Finder to not extend the width of that query text field to grow with the window’s width.

The Finder lets you save elaborate search queries like this one as a ‘smart folder’ for easy accessibility later on.

In case you want to edit it at a later stage you need a Finder window with a toolbar and an action/cogwheel menu in there. That menu contains a command for editing the search query. Thanks to Dirk and S for pointing this out.

If you are happy using the Terminal, running a command like

mdfind -onlyin /Volumes/Snowman/Backups.backupdb/Kalle/ '_kTimeMachineOldestSnapshot >  $time.now(-3600)'

may be more convenient in practice.

fseventsd

One thing that hasn’t been looked at so far is how the System knows which files need backing up. The traditional way of just scanning all the folders and subfolders on the attached drives would be quite an undertaking with hundreds of thousands of files hiding on a normal Mac OS X startup drive these days. Particularly if you want to update your backups hourly.

But luckily Apple gave the System the ability to just know when a file has been changed in Mac OS X.4 where Spotlight needed it to know which files require re-indexing after being modified. This approach has now been extended in a way that the fseventsd background process monitors all file changes on the system and. It notes the folders containing files which were changed on a volume in a file inside the .fsevents folder at the top level of that volume. A more detailed description of how that works can be found in the Ars Technica X.5 review.

For this to work reliably it is crucial that the fseventsd logs are complete as otherwise files could accidentally be omitted from a backup. And thus if you ‘improperly’ remove a drive, say, because of the system crashing or being too daft to properly unmount it, fseventsd creates a fresh catalogue of changes the next time you use that drive. In the logs that will look like this:

23.02.08 14:47:38 fseventsd[45] event logs in /Volumes/Bender/.fseventsd
    out of sync with volume.  destroying old logs. (2394 0 2433) 
23.02.08 14:47:38 fseventsd[45] log dir: /Volumes/Bender/.fseventsd
    getting new uuid: 6AF6C56A-3E96-4D99-89F6-BB60256E0C97 

backupd

Finally, there is backupd, the workhorse of Time Machine. It uses the information collected by fseventsd to zone in on the files which have been changed since the previous backup. If that isn’t possible because there has been a problem with fseventsd, that will be recognised [see that the fsevents store gets a new UUID in the log message above and that the extended metadata for the previous backup include the UUID of the fsevents store used] and kicks off a ‘deep traversal’ of the file system. In that case it will go through all files to ensure it is aware of all changes. In the logs that’ll look like this:

23.02.08 14:48:08 /System/Library/CoreServices/backupd[26227] 
    Event store UUIDs don't match for volume: Bender 
23.02.08 14:48:10 /System/Library/CoreServices/backupd[26227] 
    Node requires deep traversal:/Volumes/Bender 
    reason:kFSEDBEventFlagMustScanSubDirs| 
    kFSEDBEventFlagReasonEventDBUntrustable| 

Once the list of files needing backup has been collected, the process will copy them over to the Time Machine volume, creating all the metadata described above. And of course making sure there is enough space to accomplish the task before starting it. It also purges old backups when more space is needed on the backup volume as well as those which aren’t needed anymore (that is, hourly backups which are older than a day as well as daily backups older than a month).

To do its job particularly well, backupd will check for any files that may have changed during a lengthy backup run and add those files to the backup at the very end.

Weakness

All this sounds fine and dandy. But you really have to keep in mind how it works: fseventsd only notes which folders have been changed. Then backupd scans them for files which have changed. And backupd doesn’t make a big effort when doing that job. Two obvious tests for its cleverness are failed with ease.

The first test is renaming. Renaming a file only changes its name in the folder but not its contents. Thus - keeping in mind all the hubub about hard links above - I’d expect Time Machine to create a hard link with the new name to the same old file in the backup when running the next time after renaming the file. So let’s try just that. Create the file

[kalle:~] ssp% mkdir /Volumes/Bender/renametest
[kalle:~] ssp% touch /Volumes/Bender/renametest/file

Run the backup and have a quick look at it:

[kalle:~] ssp% ls -l /Volumes/Snowman/Backups.backupdb/Kalle/Latest/Bender/renametest/
total 0
-rw-r--r--@ 1 ssp  admin  0 23 Feb 15:07 file

The file exists, now rename it

[kalle:~] ssp% mv /Volumes/Bender/renametest/file /Volumes/Bender/renametest/file-2

Run a backup again and check once more:

[kalle:~] ssp% ls -l /Volumes/Snowman/Backups.backupdb/Kalle/Latest/Bender/renametest/
total 0
-rw-r--r--@ 1 ssp  admin  0 23 Feb 15:07 file-2

The file with the new name exists in the backup. But its link count is just 1. Thus it is a new file. The harm done by this varies. For a tiny file like this one it’s hardly worth mentioning. For a big video project where renaming may mean a wastage of many gigabytes of storage space just because it has been renamed, that may be more painful.

What is potentially even more harmful is the following behaviour: Time Machine considers two files to be identical if their path and dates and size are the same. I.e. it relies exclusively on metadata to determine whether or not a file changed, which ironically means it checks everything but the only relevant information. Luckily this isn’t too much of a problem in most practical situations, but it still strikes me as ironic. Having something like a hash of the file’s content could improve this a lot.

And although the last changed date of a file is supposed to change when that file is edited, that isn’t necessarily the case. (In fact, with the shambles that are metadata on computers the ability of GraphicConverter to not modify the last changed date of a file when saving it is convenient as it will not cause converted versions of an image to have a date that differs from the original file’s.)

And thus I can create a file (I used a 1 by 1 gif image for a simple test), back it up, edit it (with that file not changing its size and dates in the process), run the backup again and have only the first version of my file in the backup. If my hard drive breaks now, the changes to that file will be lost.

This may seem like a tiny point and a situation that shouldn’t happen. I think it’s good to have a clear sense of what the words we use are. And in the way Time Machine uses the word ‘changed’ it means that a file that has been moved to a different location has been ‘changed’ and that a file whose content has been replaced by other content of the same size in a way which didn’t modify its other metadata is not ‘changed’. As my usual understanding of the word ‘changed’ differs from that, it’s worth noticing and keeping in mind to avoid disappointment.

User Interface

Not last but least there is the graphical user interface for Time Machine. Like many recent GUI ‘innovations’ by Apple it leaves me with a ‘WTF What were they thinking?!’ expression:

While not the biggest Bauhaus fan, I am a ‘form follows function’ type. And that concept certainly didn’t play a role when ‘designing’ the GUI for Time Machine. There are so many things wrong with it that I don’t know where to start or how to make sense of it. So I’ll just give a semi-random list of points to think about.

• Shouldn’t tasteless Sci-Fi backgrounds be an exclusive feature of Linux distributions?
• If you need to have moving stars in the background image of your backup application, can’t I expect them to move smoothly no matter what I do? Particularly if I’m using a ‘supercomputer’? With an OS that has been ‘crafted’ to perfectly match the hardware? [This can be worked around using the _FXShowBackgroundAnimation hidden preference of the Finder.]
• Wouldn’t it be helpful to have a real timeline on the side? One with a meaningful scale?
• Wouldn’t it be helpful to see which files changed in a particular snapshot? (Rumours are Apple thought about that, but obviously they didn’t ship it. The Spotlight comments above point at how doing that could be reasonably easy.)
• Wouldn’t it be helpful to be able to see the development of a folder or file rather than just having a 3D arrangement of older versions of the window behind the front one, all of which are covered up so much that they are completely useless?
• If you start Time Machine from a lean Finder window without the sidebar, is there any chance that the user doing it may want to use the Find command? (The only workaround I currently see for that is quitting Time Machine turning on the sidebar in the Finder window and turning Time Machine on again - no kidding!)
• I am not a fan of contextual menus, but shouldn’t those work in the Time Machine environment as well when it pretends to be just like the Finder?
• Is a feature like the remove from backup command really ‘discoverable’ when hidden away in a cog wheel menu that may or may not be visible on screen.
• If you try to give users the illusion that they really are working with exactly the same window they fiddled with in the Finder while using Time Machine, wouldn’t it be a good idea to actually do this in a way that scroll bar positions remain constant as well?
• … and in a way that the Show Package Contents command works?
• If you need a full screen UI for restoring and you even hide the Dock when showing it, wouldn’t it be great if the ‘shadows’ of F11-Exposé went away as well?

… and so on. When playing with just the Finder’s Time Machine interface you quickly get the idea that some Sci-Fi geek had an UI idea which they then implemented but never really tried out. If you are using a large sidebared Finder windows in column mode, it will work all right for recovering a single file. In any other situation things quickly appear a bit broken and not exactly well planned and thought-through.

Other aspects of the UI such as the navigation arrows could also be better in that what happens when you click them would be predictable. Right now clicking the backwards arrow will slide things around but I have no idea how far they will slide before seeing it. Even if it slides just ‘the right’ distance, why not indicate that in the UI beforehand by omitting the superfluous layers or greying them out? It’s not really clear to me anyway what all those Finder windows actually represent on screen. Usually they do not represent the state of the file system at the time the window in question represents. They just seem to be a copy of the frontmost window and will only be updated when they come to the front themselves.

Time Machine also comes with two UI elements: An icon which appears in the Dock by default in X.5 (Apple’s tendency to clutter your Dock with stuff starts reminding me of the Windows ‘Start Menu’ and Desktop which are littered by every installer - Apple just have a better monopoly to get more exclusive spamming opportunities for themselves) and a menu bar item which was added in the X.5.2 update. It lets you run a backup and launch the Time Machine UI. And it rotates like hell, the wrong way round, while files are being backed up. (The icons the Finder displays in its sidebar next to the icon of the Time Machine volume rotate the right way round, btw.)

Restoring Files

The most important thing about a backup is that you can get your files back if you need them. As a Time Machine backup is nothing but a folder with files in it, you always have the ability to just browse into the Backups.backupdb folder in the Finder and get the file you want.

Or you can use the ‘official’ starry sky Time Machine GUI described above and grab the files from there if you have the nerves for that. You will even be asked whether you want to use the backup file to replace the newer version of itself or whether you’d like to have both files next to each other. With the files I recovered so far, both methods worked fine.

Both these methods use the Finder for copying the files. And the Finder will strip the extended attributes relevant to the backup from them when doing that. Which is probably a good idea. Copying the files using the cp command leaves the extended attributes in place and you’ll probably want to remove them afterwards.

The Mac OS X.5 installer also offers an option to restore the system from a Time Machine backup rather than setting up or migrating accounts. That’s a very appealing option if the whole hard drive breaks. I haven’t tried it myself yet but reports I read so far seem to be mixed. While no harm is done to the backup data, doing the full restore doesn’t seem to be terribly reliable yet. It may be a good idea to keep those cloned drives handy for a while if you need to be able to recover quickly.

Other applications

Another feature of Time Machine is that applications are encouraged to integrate with it. Some Apple applications like the Address Book, Mail or iPhoto offer such integration (and applications like iTunes, iCal or Safari lack it). But I fear that many people won’t even notice that as you have to invoke Time Machine while the application is active to get the Time Machine interface for that application. And there isn’t an out-of-the-box way to invoke Time Machine for most of those applications if you haven’t cluttered your Dock or menu bar with its icons. Only iPhoto seems to have an explicit menu item for accessing backups at this stage.

I haven’t really needed this so far, so I can’t say terribly much about it. The idea behind it seems to be a laudable one. As many applications store their data in an opaque format which consists of many little files and perhaps some database keeping them together, it’s not feasible to do fine-grained restores for these applications. You can either revert the full database to a previous state - undoing many other, wanted, changes in the same move or you can’t undo anything.

Letting the application access its old database and restore the little bits and pieces from it is a sweet idea. But I wonder how many applications will offer that. I am sure that doing so may be quite a bit of effort. And doing that in a correct way for the rare situations when a user both needs it and knows the feature is there, seems to be a lot to ask.

As things are, the integration with Apple’s non-Finder applications isn’t first class anyway. Not only is half of the date at the bottom of the screen covered frequently covered by a button on my ‘small’ MacBook screen (a problem so obvious that it suggests nobody at Apple ever bothered to think about this properly or to try it out on such a machine), the general behaviour also seems to be quite erratic (iPhoto windows showing up empty at first and only filling eventually after I scrolled back and forth a bit, iTunes playback skipping while the computer made that ‘effort’, the lack of displaying actual differences in the Address Book, to give a few impressions I collected when trying things out for a few minutes).

The integration with applications also makes me slightly uneasy. The beauty of standard Time Machine backups is that they are a-file-for-a-file. It’s easy to understand what goes on there. And recovering a file from the backup is essentially a copy operation. The more ‘clever’ behaviour applications show here means that the transparency of what is going on is lost. I will have to hope that the software does the right thing.

Preferences

Time Machine also comes with its own preference pane in System Preferences. It features a huge logo with the name beneath it in Myriad which doesn’t really belong to OS X. It also features a huge ‘slider’ On/Off switch that is in no way a standard UI element and in no way behaves like the real world slider it’s supposed to represent. It works like a checkbox, so why isn’t it a checkbox?

With space being so generously used for the ‘pretties’, little remains for actual preferences. You mainly get a display of the current backup status, some information about how backups work and two odd looking buttons which in open sheets containing the actual settings. Just get that: There are a mere five settings they let you make for Time Machine and you need a whole preference pane and two sheets for them.

In fact those sheets are quite anaemic. They are tiny and all they let you do is select the backup volume and exclude stuff from backups. The excluded items from various volumes are then displayed alphabetically in a single list with no sense of their location. So if you excluded a folder named ‘Pictures’, good luck finding out which one it actually is. You’ll have to hover over the line in the list to get a ‘tooltip’ with a Unix style path to the folder in question. Very user friendly and Mac like!

And that’s before I get into real bitching mode, my friends! Let’s say you have a hard drive and a single folder on that hard drive which you want to include in your backup. If you naïvely assumed that this should be a ‘super easy’ single drag-and-drop operation in Apple’s OS, you’d be wrong. As far as I can tell you are expected to add all other folders on the disk in question to the exclude list and of course update its state whenever you add another folder. Excellent!

Security

Security seems a topic which they mostly ignored when designing Time Machine. If you consider your data to be private, you can be screwed in two ways by Time Machine.

If you really are more concerned about your data remaining private than about it being unaccessible, you are probably using FileVault on your machine. It stores your home folder in an encrypted disk image (these started off with a reputation for easily being irreparably broken in system crashes or so - I heard the format improved since, but never really checked that). And you get plenty of inconvenience for that - such as your home folder not being available via file sharing and possibly a bit of a speed hit. Oh - and you will only get very limited Time Machine goodies as well. Because, rather than saving the files in your account, the computer will only make a backup copy of your complete encrypted disk image if you are using File Vault. To make that a bit more efficient, Apple introduced a new disk image format, the ‘bundle image’ which splits its contents up in several ‘slices’. Meaning that Time Machine only has to back up the slices which changed rather than the complete image. This isn’t exactly neat but - assuming that they got the encryption right - it will keep your data encrypted even in the backup. But your only option for restoring will be to restore the whole image in one go, rather than the individual files contained in it.

The other aspect is encryption of backups. Usually backup software offers it as an option. And it makes a lot of sense to me. Even if you say that someone has to break into your home to steal your data and that you don’t need local encryption because of that, it seems entirely possible that you would (or rather, should) store your backup somewhere else. Put differently: There will be another copy of all your data now which you rarely need to access. Thus the inconvenience of the encryption plays less of a role and it would be nice to have.

Unfortunately Time Machine doesn’t offer to encrypt backups. Which is a shame - it will use disk images anyway and encrypted disk images do exist, so it seems like this would be an easy thing to do. Of course that extra level of encryption would require a bit of careful design about where the password for the backup image is stored on a shared computer. And it could lead to agony if someone ran their backup for years and ended up needing it without remembering the password. It’d still be a nice option.

Once you start thinking about network backups - possibly storing your data on servers of some third party on the internet - which hopefully becomes an option in the future, encryption of backups would become pretty much mandatory. Call me paranoid, but at least I wouldn’t want all my data to be stored in clear text on their machines.

What Else

Even though this text is both wordy (for sure) and rich in information (hopefully), it still doesn’t present a full coverage of Time Machine. Mostly because I am running a single machine in a single environment which means that I am not able to know about the full range of usage cases, the opportunities, the conceptual problems, the practical problems. All I can go by is word of mouth. Where word of mouth isn’t even that but just what I picked up from the garbage dump known as the internet.

The impression I get from this - my limited experience spanning a few months and the reports I read - is the following: Time Machine is worth having because it does make an additional copy of your data and is unlikely to accidentally destroy that additional copy.

On the other hand there are still plenty of practical issues at this stage. Ranging from the - not necessarily Time Machine intrinsic - fact that OS X.5’s support for external drives could be much better performance-wise and even seems kernel panic prone (from my own experience). Then there’s the sub-par UI, both for setting up the backup (no way tell Time Machine ‘include this specific folder in future backups’) and for recovering files (which is both tacky and giving much less information than would be helpful and possible). I’d say that technically those UI points aren’t biggies. And perhaps we can hope for someone givng us a better UI for it. With Apple’s personality being a bit too far on the self-consumed and narcissistic side to admit they shipped something less than ideal or - gasp! - poorly designed, I wouldn’t hold my breath while waiting for them to improve things.

And then there are future questions, most of which were touched somewhere above: Quotas trying to fairly split backup space among several users? Dealing with small changes to large files (frankly, I consider Apple’s idea of simply splitting them up - as demonstrated for bundle disk images - to be a cheap hack rather than a good solution)? The apparent problems with applications like Aperture (no first hand experience)? Getting the ability to easily revert system updates you didn’t like with Time Machine? The numerous glitches which have been reported by people trying to do full restores (i.e. your data are saved but recovering them may not be totally fool proof). The whole issue of encryption. And in a next step file systems (there will always been someone mentioning ZFS which isn’t a reality yet but sounds like it might be a good match for Time Machine although it may require a number of changes to the concept). And in yet another step, networked file systems for backups to another location (with encryption being a must-have feature there). Which leads to more questions about which file server volumes Apple support (currently OS X.5 computers and Time Capsule devices by default and give or take a hidden preference for everybody who likes doing unsupported things with their backups). This is far less than they initially promised and it’s usually said technical rather than marketing issues cause the scrapping of that feature. Of course with Apple always achieving all their goals we will never learn what the real reason was. And while them limiting the features of their software just to shift more Time Capsule hardware would be a shame and really do injustice to a topic as important as backups, it wouldn’t seem completely along the lines of Apple’s ‘personality’ to do exactly this.

It appears that Apple’s mid-March 2008 Airport/Time Machine update at least enables volumes connected via their own Airport Extreme base stations to be used for backups.

Fin

So here we are - I hope you enjoyed the trip. Time Machine looks like a great step forward for preserving data. But we will have to keep an eye on Apple whether they are serious about it or not.

Want some Time Machine goodness?
[Buy at amazon .com, .uk, .de]

NAME

pictd — general helper tool

SYNOPSIS

pictd

DESCRIPTION

pictd is a helper tool for the Cocoa frameworks.

There are no configuration options to pictd. Users should not run pictd manually.

Strings like Unable to register connection for PICT renderer. or _NSPICTRenderer in the tool suggest that it is related to handling old-school PICT images. But it’s not really clear to me what causes the tool to be launched.

Just being curious…

I know that nobody asked me, and thus I will have to do that myself. During the interview I answered each of the following questions with a firm no

• Should the machine be sluggish just because Spotlight wants to re-index an external drive for whichever reason?
• Should Time Machine need an hour to backup 100MB to another drive at the same time?
• Should copying files be slooow while that is going on?
• Should typing in TextEdit stall for many seconds at a time just because I am copying files to an external USB drive and burning a DVD with data from another external USB drive?
• Should the mouse cursor stop moving for a full second just because the machine is busy updating its Spotlight index and I dare to use Quartz Composer at the same time?
• Have I been a bit optimistic about OS X.5’s ability to use arbitrarily high process IDs? (They seem to cycle round somewhere in the 100000 range.)
• Should my Airport connection break down a few times an hour?
• And generally be unreliable as it has been for well over a year? (Which I am fairly sure by now is an Apple software problem as I am seeing the same problem on another machine and yet non-Apple machines can use the network just fine.)
• Should the machine magically increase the current PID by a few thousand each hour without me doing anything to justify that?
• Should it be impossible for me to find out what is happening there? (Some process which quickly launches and then dies I suppose, but how can spot it if it’s short-lived?)
• Should Safari stall for 20 seconds just because I turn off ‘Private Browsing’?
• Should I need ‘Private Browsing’ mode at all?
• Should iTunes stall for 15-30 seconds every other time I try to interact with it? (This makes using it almost impossible, particularly for the obsessive-compulsive types who need to fix typos in metadata when they spot them.)
• Should I really be required to close all my applications and restart (which takes around 20 minutes until everything is up again) just to work around these shortcomings temporarily?
• And do I need to do this in the firm knowledge that the wireless networking problems will persist after that?

Unfortunatlely Mac OS X.5.2’s opinion to each of these question is yes. And, yes, compared to all these problems, my remarks about the menu bar icon for Time Machine rotating too obviously and in the wrong direction are utterly irrelevant.

Aaargh!

But before doing that let me just mention the newest nuisance they introduced with Mac OS X.5.2: A menu bar icon for Time Machine. That in itself is a good idea as it means you have the option of kicking out the space wasting Time Machine icon from the Dock. – And replace it by a space wasting one in the menu bar, that is. (Note that, as far as I can tell, there is no way to fully use Time Machine without one of these space wasting gadgets.)

But the icon itself is friggin’ silly. Not only does it rotate and distract you whenever the magic Time Machine does its wonders (thus ruining the whole beauty of Time Machine’s transparent operation), but the damn thing rotates the wrong way round! It rewinds the clock, thus giving the impression that, well, it’s rewinding your data to some previous state, i.e. erasing your recent work. Obviously I wouldn’t even expect Apple to be negligent enough to let such a bug slip through, but my first impression when seeing this was a shock of horror because the damn machine was deleting my data. I was about to shut down / kill / unplug things before I calmed down.

I’m afraid I remain a believer in boring design. No such excitement for me, please.

I suppose now we can try to turn this into a sport to figure out an expression which fools that cleverness…

While that is clever and suggests the patch makes an effort, I do wonder whether it actually is a good thing. It keeps you from trying out things by mathematically ‘commenting out’ variables in the equation that is being computed. If you do so, the corresponding inputs vanish. And the connections hooked up to those inputs vanish with them. Which means that restoring the previous state will require extra effort. Which in turn makes it less likely that you’ll try things out.

Another thing are loops. Some kinds of loops are easily done in Quartz Composer. With the Replicate in Space patch, for example. But other things just seem rather hard to get right and you quickly reach a complexity where you wish you had tried programming a custom patch with exactly the feature you want to begin with rather than fiddling with Quartz Composer’s built-in patches.

One thing I tried to do was the following: I wanted to monospace a string. That is, I wanted to throw in a String, and, as an output get an image where the string is written in an arbitrary font with the space used being the same for each character. In a way that should have been easy. But things didn’t quite work out as I wanted them to and I ended up with a monstrous composition for a limited length string instead. As a bonus it even contains a feedback loop (I have to draw all characters to find out which one is the widest and then I’ll have to draw all characters centred on that width). The composition seems to do the job but it’s just an awful ‘solution’:

When looking at compositions like this one you start thinking whether you couldn’t try to create compositions just for the æsthetics of their components…