How To Fix Xcode 4

People are sort of tired of me complaining; I’ve been saying I hate Xcode 4 since the first time I used it. “You’ll get used to it!”, “Learn Behaviors and everything will be well”, “Embrace the new UI model”, they said. I’ve been using it exclusively for several months now, trying to explore and use every feature of it; make sure to try to find a way in which I can confidently work in it. I failed. Launching Xcode 3 is a fresh breeze every time.

Just a note up front: I’m not saying Xcode 4 sucks. It is light-years ahead of Xcode 3 in many ways, particularly autocompletion and the IDEs idea of what the code contains (inline warnings and errors as you type is wonderful). This is just about the way it manages windows and tabs. And even then, maybe there is some mode or usage pattern that I don’t know about and would fit me better — if so, please shout in the comment or to my twitter!

Spatial Thinking

Xcode 4 works fantastically for a lot of people. How am I special? Why do I have to try to be so different all the time? Apart from just generally being an annoying sod, I imagine myself being a spatial thinker; I like to think in terms of spatially organizing whatever concepts I’m working with. I loved Classic Mac OS’ spatial Finder, where windows always appeared at the same location were it did last time, and there was a one-to-one mapping between folder and window.

The same goes for me and source code files. I used to work in Condensed Mode in XC 3; a single thin window with a project file listing, all source windows separate, and then dedicated slots for Build and Run windows on a separate screen (together with a terminal or two). I would work with a dozen source files at the same time, placing them around the screen. When I needed one I used recently, I would know where it would be spatially, and could just click a tiny corner or edge peeking out beneath whatever I was working on(1). This way I could also layer other applications in between windows, and have a rather deep pile of things I’m working on. It’s the classic desktop metaphor, and the main argument against Multiple Document Interfaces (MDIs) that we Mac nerds would ridicule Windows users for having to use every day(2).

You might be able to imagine why XC4 would then be a problem for me, with everything in a single window, everything appearing in the exact same location on screen as you instead switch between modes of operation in one screen area, rather than switching between locations and zones on the screen. It was — and is — a complete disaster for my productivity. I spend so much time switching between contexts nowadays, trying to reorient myself.

Staying In the Past?

Trying to work in an Xcode 3 manner in Xcode 4 is worse than just trying to adapt.

One of the best features of Xcode 3 is that it would remember all the open windows of a project when you closed it. In Xcode 4, closing a window is terminating it and its layout forever, and the last window you happen to close will become the main project window. Oh, you accidentally closed the window *you* consider the main window first, and then you had a reference header in a window configured for reading reference headers still open on another screen? Well, I guess that’s your main window now, and you’ll just need to reconfigure it as your main. (Update: There is in fact “Close project” in File which will remember all the windows. This doesn’t help in the situation where you close one window forgetting that you had two open though, which is quite common.)

Want a separate window for your build progress so you can keep track of what’s actually happening in your builds? That’s in a sidebar now, and if you actually want build details, you have to manually choose the latest build results in *another* sidebar, and then switch back to the first sidebar.

Want a separate window for your console output? Hey, there’s this hack where you can name a tab “console”, bring up the debug lower-pane, set that pane to console output, make the pane fill the window, turn off the left-and-right sidebars, drag the tab out to its on window. Remember to put it back in its tab before closing the window though, or it’ll become your project!

OK, this is not working.

Trying To Adapt

The way I work now is to have everything I used to have in separate windows, in separate tabs; and have Behaviors set to switch to them at appropriate times. I have one Build tab and one Debug tab, set up in a pretty nice way. I mean, this is okay, I can work this way. However, I do lose my focus a lot of the time, and I get lost trying to context switch between tabs. And I need to set this up for every project I start working on. And I can’t really use my secondary monitor for the things I like and used to do. And it’s just a pain.

Fixing Shit

Why alienate all spatial thinking developers from the Mac and iOS developer communies? I can absolutely imagine Condensed Mode working in Xcode 4. The concept of optional sidebars everywhere could still work. I’d just want to pull out the Project navigator sidebar into its own window which becomes the ‘project main’ (closing it closes all related windows); allow some of the more awkward sidebars like the Object Library to be a panel like in IB; have dedicated Build and Console windows; and have the IDE try to suggest the “one file per window” concept (but possibly allow browsing different files within one window is the user really wants to). I mocked up a horrible concept in Acorn, for your viewing pleasure:

This would make me very, very happy. Sure, it’d be half as “fast” as Xcode 3, it’d still forget how to autocomplete once a day, and crash pretty much every minute, but at least I could work with it.

Thanks to Brent Simmons for writing “The part of Xcode 4 that tires me out”, finally triggering me to write this post.

“Software today is not engineering. Building the Empire State Building with three thousand people in a year is engineering. Programming today is more like the Egyptians piling blocks on each other and hoping it doesn’t fall over.”

— 

Paraphrasing Dr Alan Kay in his talk “Programming and Scaling”. I got the same sense of depression over the state of our industry when I read about Doug Engelbart’s amazing NLS system from ‘68 and the Hypertext Editing System in ‘67. Yes, we have millions and millions times faster computers today, but what we’re doing today is not news. They realized already in ‘68 that you want direct manipulation and collaborative computing.

Every day, I spend so much time building almost trivial functionality in thousands of lines of code, with brittle APIs that hook into other people’s brittle APIs, because that’s the code base we have, and every other code base seems to be roughly looks like that, too. I spend my weekends learning about C and C-like systems because that’s what I know; I don’t go around learning about compiler-compilers, I’m stuck in what my brain already “knows” that programming is like. I didn’t know twenty minutes ago that you could write a complete compositing 2d graphics environment with 26 compositing method in under 400 lines of code.

My very good friend Martin once told me how different it is to even think in Japanese and Chinese, how those thoughts can’t even be translated into English. Kay’s realization that the same holds true for programming is profound in a way I didn’t realize before.

Back in uni I unwittingly participated in an experiment. One of the teachers arranged two lab sessions with the exact same content, and put Java programmers in one and C++ programmers in the other. We were given some UML and were asked to implement it. The real task had actually been to see how fast we could build it (but were of course not told in beforehand that we were being timed); and while I was fastest in my group and it felt like the C++ came out pretty naturally, the result showed that it took me twice as long to write the code than it took on average for the Java coders.

Ever since then, I’ve been worried that even though I feel like I’m really fast with my tools, those very tools are holding me back, and I’m *really* just a tenth as fast as I could have been with another set of tools. I love ObjC, but it’s still C, it still got non-object types, there’s still manual memory management, everything is viciously verbose, and meta programming is still more limited than in some non-C languages.

Low Verbosity KVO

DRY Cocoa: SPLowVerbosity

I love Objective-C. It’s a very explicit language: there is no magic in the language. If you want such magic as distributed objects, model classes generated at runtime, generic change notification, or heck, even referenced counted memory management, you can implement it yourself in a library. It’s also by convention a very verbose language. If the method name doesn’t say exactly what the method does, the author is doing it wrong.

However, there are some things we type over and over again every day, whose verbosity does not make the code more readable but only serves to pad the code file with useless characters. These are in particular array and dictionary literals, and formatted strings.

ObjC wizard Jens Alfke wrote MYUtilities quite a while ago, and in particular CollectionUtils. This header defines a few very handy macros such as $array(...), $dict(...) and $sprintf(...). $array is simply a shorthand for [NSArray arrayWithObjects:..., nil] (note the extra nil, freeing you from typing that yourself every time, and also avoiding a possible crash). $dict is NSDictionary’s constructor, with the arguments in the right order (key, value, key, value, …). $sprintf is [NSString stringWithFormat:]. It may seem trivial, but this simple header has saved me tons and tons of typing, and surely from some silly bugs as well.

For more C++ friendliness, and to not have to depend on the rest of MYUtilities, I wrote my own, called SPLowVerbosity, with mostly the same things. The details are not interesting, so I won’t list the code; click through if you are curious.

This is a hack, and while it is very convenient, it is not pretty. Objective-C needs in-language literals for arrays, sets, dictionaries and number objects.

I do believe that such literals are now inevitable, though. With ARC, what I wrote in the first paragraph is no longer entirely true. Dealloc now magically calls super. Memory management is implicit and part of the language, not explicit and manual. The sanctity of Objective-C’s simplicity has been violated, parts of Cocoa has been moved into the language. Moving more parts of Cocoa into the language (e g exposing APIs saying ‘this is the class that should be instantiated when I use a dictionary literal’) no longer has a high religious price, and I bet we’ll see it at next year’s WWDC.

Object-Oriented KVO: SPKVONotificationCenter

During the past few years, my nose has started to really pick up on a code smell I didn’t feel before: unencapsulated concepts. What I mean by that is an API concept or contract that is only visible in comments and documentation, thus only implicit by knowledge of that documentation when you read code using the API, instead of exposing the concept as a construct the language can help you manage.

Examples:

• Mutex locking as two separate functions that must always be paired. This is an easy to make mistake, and can be very difficult to fix in complexly branching code. Static analyzation can find such errors, but is in my opinion the wrong approach.
• UITableView’s beginUpdates/endUpdates must always be called in a pair, and all the updates you make in that pair must match the changes you are doing to the data source.
• NSNotificationCenter requires you to register for a notification with a {sender, receiver, name} triplet, and reuse that triplet to destroy the subscription. In practice what you have here is an implicit object that you must manage using an awkward object triplet, rather than an actual object.
• Key-Value Observing, with the same defect as with notifications, plus that callbacks are dispatched to the same instance method, forcing you to use the triplet plus a context pointer as the identifier again.

Instead of the above concepts being implicit and only visible in documentation, several of them can be represented as objects and closures, thus making it impossible to do wrong.

The realization is not new, of course. It’s quite common to use RAII patterns in C++ to encapsulate concepts in code, giving them an explicit start point and a managed (and deterministic) end point.

But this is Objective-C, not C++, so we don’t have RAII. We do have objects, however, and if we use them to actually represent our objects, we can clean up so much code.

SPKVOObservation is an object that represents a KVO subscription. When this object is created, you know you have your subscription. When this object disappears, you know the subscription is gone. By managing instances of this class like I would any other instance variable, I know I’m following KVO’s subscribe/unsubscribe contract. (With ARC, I don’t even have to manage the ivar).

SPKVONotificationCenter also has dispatch to a given selector instead of the catch-all -[NSObject observeValueForKeyPath:ofObject:change:context:].

(As for solutions for the rest of the examples: a closure can be used for the mutex case if in the current scope, otherwise you might want RAII (which is possible with GNU extensions even in C). For the table view, I’d prefer bindings, but doing to-many KVO and bindings can be really hard, so I doubt we’ll be seeing it on iOS in the near future.)

DRY KVO: SPDepends

I love KVO. Basically every library I’ve ever used has their own implementation of notifications on attribute changes. Such an implementation is of course not difficult, but because there is no generic system for it in other languages/platforms, you can’t build a framework for working with change notifications generically. Recognizing that this is a generic concept, and then as good as building it into the language is genius, and has given us wonders such as bindings.

Outside of their use in bindings where you have tool help, KVO’s API is pretty bad, and working with it is almost painfully verbose (made worse by the single callback point, spreading your code out all over the file). What I really wanted to do was to unify the two above sections for a very simple, non-verbose and object oriented approach to KVO. The result is SPDepends. Magic is taken to the next level; we are approaching dark arts. Whether what I’m about to present is actually a good idea or not, I leave as an exercise for the reader. First, the header:

tl;dr: SPDepends lets you give it a list of {object, key path(s)} pairs that one of your properties depend on. When the value of any of the given key paths changes, a given closure is called, letting you recalculate the dependent property (or whatever you want to do). It’s about as far as you can go with KVO without actually building bindings.

There is one additional piece of magic related to memory management (this was before ARC). By providing an owner and associationName, the dependency is assigned as an associatedObject, so that it will automatically be cleaned up when the owner object dies. Defining a dependency again with the same name will also throw away the old dependency. If this is not desired, you can just not provide the association name, and instead manage the returned SPKVOObservation object on your own.

Example:

Output:

Note how short the $depends macro makes all your KVO work. The macro also defines a block-safe self variable selff that won’t create a reference cycle.

The equivalent classical KVO code would be many lines longer. However, this macro takes several steps away from how Cocoa code is normally written. Is the syntax too obscure? Is the code still readable for normal human beings?

Media keys hook in Mac OS X

There is no nice (or is it even official?) way of detecting and handling the media keys on the user’s keyboard. One can intercept events with type NSSystemDefined and subtype 8 in -[NSApplication sendEvents:], but this has major problem: any other applications that listen to the media keys will receive these events too. This means for example that if only Spotify is running and you want to pause your music, iTunes will start when you press play/pause. If you have VLC running in the background, it will also start playing.

Apple has solved this problem internally by having their media key using applications cooperate and resign media key controls to the application that was in the foreground most recently. However, there is no way for third party applications to join this cooperation.

I have implemented a workaround that does this using a CGEventTap in the Cocoa class SPMediaKeyTap. The advantage of using an event tap instead of just intercepting the events in -[NSApplication sendEvent:] is that you have the power to throw the event away. This way, we can decide to “own” the media keys and be the only app that listens to them.

This class is smart enough to resign the media key event tap whenever another application that we know will want to use the media keys becomes active, and keeps track of which media key using app was recently started.

If everyone uses this class, and everyone add each other’s bundle ID to that list of whitelisted bundle ID’s, we’ll get nice behavior from all apps. An even better solution would be if Apple provided a way of acquiring the media keys.

One-line closures in ObjC without blocks; or, NSInvocation fun

Objective-C is wonderfully dynamic. One of my favorite wonders is the invocation grabber. I never paid them much attention at first, only using them when I explicitly needed an NSInvocation for some API. Then, I started working at Spotify and found a certain header file; suddenly, I saw the light.

If you know what that an invocation grabber is, you can just skip down to the code. For the rest, I’ll elaborate.
Shortcut: Just gimme the juicy bits, I’m impatient!

NSInvocations

An NSInvocation is a standard Foundation class that wraps four pieces of information all related to calling a method:

• target, the object on which to call the method
• selector, the name of the method to be called
• methodSignature, a class that describes the return type, and the type of all the arguments, of a method.
• … and finally it contains all the arguments to call the method with.

Or, in the form you are used to see these parts in code:

Normally, these are kind of tricky to construct, particularly the method signature. However, Objective-C’s message forwarding mechanism practically gives them to us for free!

Message forwarding?

(Short clarification detour: In Objective-C, you almost never call methods directly; instead, you send messages to objects, and the receiver almost always responds to the message by calling the method corresponding to that message on itself. This is why the terms ‘method calling’ and ‘messaging’ are often used interchangeably in ObjC, but they are not the same thing.)

When you call a method on an instance whose class does not define that method, you don’t get a compiler error as in C++. When you then run your code, the runtime assumes that you’re trying to do message forwarding. Message forwarding means that the receiver of the message will not itself handle the message, but is only a proxy for some other object. One common example is Distributed Object, where you’re sending messages (calling methods) on objects in other applications, or even other applications on other computers.

Writing an invocation grabber

This is where the invocation grabber comes in. We can create a class that does nothing but listen for message forwarding requests, and when it receives one, asks the target what it would respond to such a request. Given this answer, an NSInvocation can be created which matches the invocation of the message we sent to it.

Consider the following example:

We define a MyClass, create an instance of it, then create an invocationGrabber with that new MyClass as its target, and then try to call -[MyClass areTheNewViewersGoneYet:] on the grabber. Finally, we fetch the result of the invocation grabbing.

Note that line 9 does not call areTheNewViewersGoneYet: on your new MyClass. Instead, the grabber intercepts that call and gives us an NSInvocation that would have called that line just as we wrote it, but some time in the future instead of now. This is like a one-line closure, as the code, state and variables you send to the grabber are saved until later, just waiting to be executed.

Our invocation grabber does not implement a method with the selector areTheNewViewersGoneYet:. Given just the invocation grabber and that selector, the runtime would have no idea how to call it. Does it return a bool, in which case it will need to reserve four bytes for the return value? Or maybe a CGRect, in which case 16 bytes are needed.

So to figure this (and related things) out, the runtime will begin by asking for the method signature for the selector being sent with the message with -[NSObject methodSignatureForSelector:(SEL)]. Our SPInvocationGrabber itself has no idea, so we implement that method to ask the target:

- (NSMethodSignature *)methodSignatureForSelector:(SEL)inSelector {
	return [_target methodSignatureForSelector:inSelector];
}

With the signature in hand, the runtime will now give you an NSInvocation and ask you to take care of invocation with -[NSObject forwardInvocation:]. In our case, we’ll just stash away the invocation for later use. In the example above, we created a yellowDuck and sent it as an argument to areTheNewViewersGoneYet:, and if we delay our execution of the NSInvocation, that duck might be released with the nearest NSAutoreleasePool. Thus, in forwardInvocation, we ask the NSInvocation to retain all its arguments. Don’t worry, when we release the NSInvocation, it will release those arguments too.

- (void)forwardInvocation:(NSInvocation *)anInvocation {
	[anInvocation retainArguments];
	anInvocation.target = _target;
	self.invocation = anInvocation;
}

That’s basically all there is to it! Some memory management and accessor methods, and we have our own invocation grabber.

So what’s all the fuss about?

Let’s start out with some code.

That’s the header. It has a few categories on NSObject, which is where it’s at. (A category on NSObject means that we are adding methods on almost all other classes; these three methods can be called on any instance of a class inheriting from NSObject.)

-[NSObject grab] gives you an invocation grabber for the receiver, so that instead of [[[SPInvocationGrabber alloc] initWithObject:foo] autorelease] we can just say [foo grab]. It doesn’t sound like much, but it goes a long way toward taking the hassle out of doing magic.

-[NSObject invokeAfter:] will give you a proxy object that, instead of performing the method you call on it right away, instead invokes it later. Again, doesn’t sound like much, but man, -[NSObject performSelector:withObject:afterDelay:] is a mouthful, and it doesn’t work with non-object arguments or calls that take more than two arguments. Suddenly delayed execution is simple.

-[NSObject nextRunloop] is the same as invokeAfter: with a zero second delay. This is the common idiom for having something executed the next runloop (or at least as soon as possible after the current runloop).

I’ll show you what it can do with some examples:

A: Yeah yeah, we could have set an animation delegate, and animation did finish selector, and from there called removeFromSuperview. That’s at least six lines of code. Now we did it with one: [[flash invokeAfter:duration+0.1] removeFromSuperview].

B: Some things work better if you call them very soon instead of right away. Sometimes it’s an animation that needs to have started before we can begin modifying it. Other times it’s a timing issue and creating a delegate or notification is a lot of hassle for something that will just work one runloop from now.

C: Hey, who thought animations could be so simple? Don’t worry if the view controller that owns that table goes away before the animation finished: the invocation grabber retains both the table view and the arguments, so for 450 milliseconds, those objects *will* remain.

There’s more magic, but I’ll go through that after the implementation:

First, stack trace saving. You have no idea how useful this is. If you do the old -[NSObject performSelector:withObject:afterDelay:] trick, you’ll note that sometimes you crash. You’ll also note that below the method you just delayed execution of, you have no useful backtrace. Fret not, for execinfo.h is here to save your day! When an SPInvocationGrabber is created, the names of the symbols on the stack are saved away with -[saveBacktrace]. Then, if you use the grabber’s invoke instead of -[NSInvocation invoke], the invocation will be wrapped in a try-catch, and in the event of an exception, the old backtrace will be printed, showing you the culprit that scheduled this broken future invocation.

As for those convenient categories on NSObject, they are ridiculously simple. All -[invokeAfter:] does is to create a grabber, schedule it for execution later with an NSTimer (before it has grabbed anything, even!), and then returns it, ready to grab something for later execution.

I’ve tried to create -[NSObject invokeOnMainThread] and -[NSObject invokeOnBackgroundThread], but those are trickier, because you can’t just schedule the invocation and then return the grabber, as -[NSObject performSelectorOnMainThread:withObject:waitUntilDone:] would perform -[invoke] immediately, giving us no time to do the invocation grabbing, as with -[invokeAfter:]. It’s easy to do on your own though, even without a convenience method:

  id grabber = [foo grab];
  [grabber doSomethingFancyThatMustBeDoneOnMainThread:object someArg:wohoo];
  [grabber performSelectorOnMainThread:@selector(invoke) withObject:nil waitUntilDone:NO];

Enjoy, and do post your own invocation tricks in the comments and reblogs.

Update 20100828: As you can see from the gist, I’ve updated it with background and main thread invocation (example usage below); thanks to Adam Crume in the comments for the idea and half of the code.

Japan, in 1991 and 2008 respectively. Same train, same platform; very timeless. And VERY crammed!

Growl 1.1.4 with 0.1s transition animation

Just a little bit of sanity. It would drive me batshit crazy that the growls would animate in just so slowly, covering whatever you needed to interact with right now, and just when it starts to fade out you mouse-over it to click it to remove it, but it fades in on mouse-over! And then you click it, move your cursor away, and then have to sit through an additional 0.7s of fade out transition, and then you could interact with whatever it is you needed to interact with. Unless another growl decided to appear just then, and into the vacant spot. GRARGHHHH!

I never understood how LanOSD could die while Growl prospered; LanOSD was pure grace.

Funniest part of this hack: Just above the setTransitionDuration method, there’s a comment: “Maybe there should be a preference setting for this” (or something to that effect)

Mutable Adventure, Pedro and Erlang Text Processing

As some of you might know, I’m currently writing a game called Mutable Adventure, a 2D sidescrolling platformer MMOG with the editability of Second Life.

The thing about Mutable that interests me the most, however, is the networking. I’m using a library/protocol called Pedro, by Peter Robinson, which I found when Keith Clark presented it at work. Before I go on, I need to explain why this protocol is so goddamned cool.

Pedro and software agents

It’s based on Prolog, more specifically, his own implementation of QuProlog. It’s sort of a blackboard system, with a central messaging server that everyone connects to. It’s string-based, and what you send over the network are prolog fact-style messages, called a notification, for example:

  playerWasHit(152, 12388)

This is then broadcasted to everyone to subscribe to this message. A subscription might look like this:

  playerWasHit(PlayerID, ObjectID), PlayerID = 152

Notice that the expression contains Prolog variables, and a guard expression. This means that several clients may subscribe to the same notification, but with a guard expression that says they only want player info about their own player.

Each subscription is accompanied by a number, so that you on the client side can redirect the incoming notification to the right place. In Python I’ve implemented this so that individual objects may subscribe, and that individual objects may receive notifications. For example, say a ball is spawned in the game world. At the instance it’s instansiated, it may subscribe to information pertaining to this specific instance, by giving the subscription a guard expression with its own ID number. Boom, automatic network message propagation within your game client or server. As an example, in the client, the Tilemap class is appended with a category/mixin with the following code (notice the third argument, the guard):

  gClient.net.subscribe(self, "tilemap(TilemapName, NameOfTilesetTilemapUses, ScrollX, ScrollY, AutoScrollX, AutoScrollY)", 
                              "TilemapName = \"%s\""%self.name)
  gClient.net.subscribe(self, "tileInMap(TilemapName, TileX, TileY, TileIndexInRoomTileset)",
                              "TilemapName = \"%s\""%self.name)
  gClient.net.subscribe(self, "tilesInMap(TilemapName, FromIndex, ToIndex, TileIndexArray)",
                              "TilemapName = \"%s\""%self.name)

(Voxar has significantly improved the syntax in the Pedro python wrapper since then)

Suddenly, your objects aren’t mere objects; because they can communicate on their own with the outside world, and respond on their own, they’re now software agents. Also, because the messaging medium is now separate from the server instance, your game server may now be freely split as you see fit into several processes, both for load balancing and for division of labor into discrete entities, with higher cohesion and lower coupling.

The downside

Peter Robinson’s implementation of this concept isn’t perfect, however. First off, it’s written in plain c, and it depends on glib2. GObjects. I can’t stand GObjects. I can’t stand manual memory management. And it’s 5000 lines of code for a relatively simple concept.

Secondly, you just traded yourself simplicity for the price of a single bottleneck. A buggy, unstable, memory leaking single bottleneck with no means of load balancing or distribution. Written in C with a single maintainer, based on glib2 which is pretty hard to get running under Windows, if you would want to.

(I attended a 24 hour game development challenge a while back, where I wrote a networked 3D racer. Because I couldn’t get pedro running under Windows, I had to run it on my own server at home, while the judges tested the game from the other side of the country. >1sec lag and no lag compensation or similar whatsoever in the code = I certainly didn’t win that competition :( )

Erlang, The Savior

After reading Joe Armstrong’s book on Erlang, I was itching to write something. I tried writing a Twitter clone, thinking that Erlang’s highly distributed nature would come to great use there, but every line took a minute to write (because I’m so new to Erlang), and Twitter just felt too big and too difficult to write as a first project, so that got abandoned.

So, the next thing I’m trying is to write a Pedro clone in Erlang. It’s a good match, since the hardest part of of Pedro is the pattern matching, and Erlang got that as a part of being a functional programming language. I’ve set out to get it to work in 100 lines or less. So far I’m up to 60 lines, and I think I have a chance of making it.

In Erlang, processes are cheap and abundant. You spawn a lot of processes and then do all communication through erlang messaging. A common pattern is the middle man pattern. While cleaning the dishes I remembered this pattern, from its usage in an IRC client built in the book, and tried to apply it to Erdro (Erlang Pedro :P) conceptually in my head (Sorry, the following is a bit hazy as 1) I haven’t implemented it yet 2) it contains a lot of erlang terms). One problem with Erdro is that in its current implementation, even if you have a supervisor process watching the server and respawning if it does, restoring all the stack state, the process would be dead and the sockets meaningless, and all clients would have been lost and restring state would be pointless. However, if each client is abstracted away with a middle man process (meaning all socket communication goes to and from a separate process, and communication with the server is handled through erlang messages), you could store all the state including process pointers to these middle men in a mnesia database or similar, and if the server dies, respawn the server and its state and everything you’d have lost was a single message (the one that made the server crash); all socket connections would be alive and all clients just continue communicating.

Put the middle men on a cluster of machines away from the messaging server, each with its own IP and bandwidth, and you’ve distributed your network load. The machine containing the messaging server could self-immolate, still only a single message would be lost (given that there is another computer that could act as messaging server).

Process load balancing of the server, however, is left as an exercise for the reader.

The nitty gritty details of text processing in Erlang

There’s a slight mismatch between Erlang and Prolog, given that they’re completely different languages (one is functional, the other is logical). In the context of Pedro, however, I could only think of a single difference that mattered, and had to be changed.

In Prolog, the term “myFunctor(atom, anotherAtom)” defines a fact. In Erlang, it’s a function call. So, for this to work, I’d have to convert that term to something equivalent that could be used in Erlang pattern matching: a tuple, like so, “{myFunctor, atom, anotherAtom}”. Since Pedro doesn’t have tuples, the transformation is reversible and fully equivalent (I hope! It’s not like I’ve tested my code yet…). How would you do this? The first thing on my mind, being a ruby coder, was regular expressions. My first realization was the erlang’s regexp support is really, really, really horrible. Not only is it slow, its syntax support is so basic you might as well do without. My second realization was that regular expressions were a really bad match for the task at hand anyway. So my first real piece of Erlang code was a text search-and-replace implementation with pattern matching specific to my task:

functor_to_struct(PrologString) ->
  [First|Rest] = PrologString,
  fts("", [First], Rest, false, 0).

fts(BeforeFunctor, Functor, [], _, _) ->
    BeforeFunctor++Functor;
fts(BeforeFunctor, Functor, Rest, InStringEscapeMode, Prev) ->
  [Next|Rest2] = Rest,
  case Next of
    34 when (Prev == $\\) and InStringEscapeMode -> % \"
      fts(BeforeFunctor++[34], "", Rest2, true, Next);
    34 -> % "
        fts(BeforeFunctor++[34], "", Rest2, not InStringEscapeMode, Next);
    Char when InStringEscapeMode ->
      fts(BeforeFunctor++[Char], "", Rest2, true, Next);

    $( ->
      fts(BeforeFunctor++"{++Functor++", " , "", Rest2, false, Next);
    $) ->
      fts(BeforeFunctor++Functor++"}", "", Rest2, false, Next);
    Char when ((Char >= $a) and (Char == $A) and (Char == $0) and (Char =
      fts(BeforeFunctor, Functor++[Char], Rest2, false, Next);
    Char ->
      fts(BeforeFunctor++Functor++[Char], "", Rest2, false, Next)
  end.

Half-way through the code, it felt so horrible, I felt like I was writing the worst code in the universe. Now that it’s done, though, I find it pretty nice. It’s relatively short for what it accomplishes (replaces function calls with tuples, being careful not to interpret parens inside a quoted string) imo. However, after a good night’s sleep, I realized that this was a really stupid way of doing it. Why? Because Erlang has a code parser in its standard library.

  {ok, Scanned, _} = erl_scan:string("foo(bar)."),
  {ok, Parsed} = erl_parse:parse_exprs(Scanned)

yields [{call,1,{atom,1,foo},[{atom,1,bar}]}], a perfectly fine nested Erlang data structure that can be traversed and parsed. So that’s what I did!

  transform_calls_to_tuples(ParseTree) ->
    tctt(ParseTree, []).

  tctt([], Collected) ->
    lists:reverse(Collected);
  tctt([Token|Rest], Collected) when element(1, Token) == call ->
    {call, 1, FunctorNameAtom, ArgumentList} = Token,
    tctt(Rest, [{tuple, 1, [FunctorNameAtom | tctt(ArgumentList, [])]} | Collected]);
  tctt([Token|Rest], Collected) ->
    tctt(Rest, [Token|Collected]).

… which yields the same result, but in a parse tree (which you can turn into a string again with erl_pp, the pretty printer).

That’s it! I’ll get back to you when Erdro is done.

Stump The Experts 2008 (Audio only)

I recorded the Stump The Experts session at WWDC 2008 with my MBP. I missed the first ten minutes or so, unfortunately.

Adding a player here as soon as I’ve uploaded the audio in mp3…

Weekend Hacking

I had a lot to do this weekend and decided to do none of it, and instead ported sfxr to Cocoa, with a native UI and proper save/load :) My version’s called cfxr (as in “Cocoa sfxr”) and is available at http://thirdcog.eu/apps/cfxr . I’m not saying it’s better than sfxr, only different and more native. If you’ve got a 10.5 Mac, check it out and let me know what you think :) It’s basically just an experiment to learn Core Data and Bindings (Thanks, Scott!), and a reason to make save/load work better on the Mac.

The sfxr code (more specifically, the sdl port), when I first saw it, seemed like the worst mess I’ve ever seen. Sure, it’s a quick hack, but not even keeping state in a struct? OMG. But after working with this code for a weekend, it’s surprisingly good for what it is! “Porting” it to Cocoa was as easy as finding out which globals were properties of the sound (that is, attributes), and adding “sound.” before all accesses to those, and #defining objc syntax as C syntax for the four major playback methods. I guess I could’ve #defined the sound attribute accesses as well, making an upgrade as easy as a copy-paste, but I felt I had already done enough code generation for one day :P (Check out the legacyAccessors.m in the source to see what I mean :P Not very good looking code but it got the job done.)

It’s a rough 0.1 and might need some work. I was also thinking about making the playback part an AudioUnit or VST (just for fun) to make the playback more flexible. It works pretty nice as it is though, so do check it out :)