We Need a Hero

September 27th, 2006

You’re a good developer. You’ve tackled lots of tough problems they’ve thrown your way. You are respected by your peers. You’ve done a great job with your life! And then there’s that nagging in the back of your head. You can’t kill the guilt of that glaring omission – the horror of horrors. The place where your code just doesn’t stack up.

You’ve got sucky AppleScript support.

Don’t beat yourself up. Apple is to blame for this, and they should step up and start taking responsibility. Adding AppleScript support to an application, even with the pleasure/pain of Cocoa scripting, is as much of a rite of passage for developers as learning to write AppleScripts is for users. That is, it’s a nearly universally agreed upon royal pain in the ass. And that’s wrong.

Why is it so painful? With Cocoa Scripting it’s common to hear that adding support is almost as simple as flipping a switch, and it sort of can be. But it’s also true that skydiving is as easy as finding something tall to jump off of. Now how do we go about ensuring a safe landing?

I talk to a lot of developers through mailing lists, IRC, and even live in person. I’ve never met one who was completely confident about adding scripting support to an app. Even the smartest Mac programmers I’ve ever met tend to confess that they “just tweaked things until they magically worked, then stopped touching it.”

This mirrors my own experience. So let me vent some things that I think are wrong with Apple’s handling of “the AppleScript situation.”

  1. Inadequate Documentation. The Cocoa Scripting Guide is pretty frickin’ good but it falls short on some frustrating levels. The step-by-step is encouraging, but it fails to mention key points like “for some features, an sdef just totally won’t work, and you’ll want to resort to using old-fashioned script suites or even an ‘aete’ resource.”

    Worse are the misleading details about indexed array operators. I ran into this with FlexTime. I need the scripter to be able to manipulate the list of activities associated with a given document. My code-level name for these objects is “TimedEvent.” The documentation suggests that methods of this form should do the trick:

    - (void)insertObject:(TimedEvent *)newEvent
    		inTimedEventsAtIndex:(int)index
    

    But in practice, I had to google my way through blood, sweat and tears to discover that this form was required for scripting:

    - (void) insertInTimedEvents:(TimedEvent*)newEvent
    		atIndex:(int)index
    

    Ugh! Kill! Kill!

  2. Buggy Examples. Like any good programmer, you’re looking to stand on the backs of giants. In the case of scripting, you’re going to want a good “scripting definition” file, consisting of all the standard Cocoa-supplied handlers, and any custom goodies you add on yourself. So that “standard handlers” stuff should be easy to copy and paste, right? If you look for such a starting point, you’ll find several options. And they all seem to be buggy. It’s not easy for me to advise a developer starting with AppleScript whether they should use Skeleton.sdef, NSCoreSuite.sdef, or some other less scientific approach. The documentation is equally vague:

    If your application supports any of the commands, classes, or other scriptability information defined in the Standard and Text suites, as most do, you should copy the suite elements for those suites into your sdef from an existing sdef file, such as Sketch.sdef.

    It doesn’t matter which path you take, you’ll end up banging your head bloody against the keyboard as you try to decipher endless, frustratingly nuanced bugs. Script results will come back with confusing chevron values instead of class names:

    «class » «constant ****all » 
    	of script item "Text Input - 2 Btns"
    

    Confusing console log messages will appear, complaining that methods like “scriptingAnyDescriptor” are not implemented in your objects. You’ll wonder aloud what the hell that method is for, and even ask around on mailing lists and search google, only to find the plaintive cries of other frustrated developers.

    There should be no “hunt-and-gather” phase in adding Scripting support to an application. Apple should include a perfectly functioning standard sdef as part of the Xcode templates for all application targets. If they want good scripting support, they need to throw us a bone.

  3. Spotty e-Mail Support. There’s a mailing list dedicated to allegedly solving the problems of frustrated scripting support implementors. The problem is, nobody from Apple reliably steps up to answer the most vexing questions posed there. Inevitably the tough questions either go unanswered, or are eventually acknowledged by Dustin Voss or Bill Cheeseman. The two most reliable sources for AppleScript implementation support are non-Apple employees who usually have to painfully admit they only figured it out after hours of scientific experimentation. When Dustin or Bill doesn’t chime in, occasionally an Apple representative does, but it’s highly unpredictable, and more often than not silence reigns. This sucks!

    OK, I know, it’s not their “job” at Apple to monitor and reply to issues on the free mailing lists. But maybe it should be. The fact is, if there was anybody on that list replying to questions with the accuracy and authority of somebody like Eric Schlegel on Carbon-Dev, then AppleScript support would be a lot better. We’re treading water, here!

    Other groups at Apple also provide stellar support through their lists. Core Audio leaps to mind. USB, Quartz Composer, Cocoa, and Networking also have real experts – the developers themselves – providing quick and friendly support to developers on a daily basis. Most of these groups in fact have 2 or more team members actively tackling problems on the lists. And when something is undocumented or broken, they admit it and we move on. I’m sure there are others. At Apple, helping developers is the rule rather than the exception, and I’m grateful for that.

    Thank you, Eric! Thank you Jeff Moore, Bill Stewart, Fernando Urbina, Barry Twycross, David Ferguson, Pierre-Olivier Latour, Chris Kane, Douglas Davidson, Ali Ozer, Becky Willrich, and Quinn! I’m sure there are dozens more that don’t spring to mind or whose lists I don’t read. You’re all heroes, and we appreciate it. Truly.

    But we need a Cocoa Scripting hero.

Contained By What?

One of the biggest hurdles in adding Cocoa Scripting support to an application seems to be coming to terms with AppleScript’s “containment hierarchy.” It requires that any scripted object be able to fully specify its location in the containment hierarchy. For instance, in MarsEdit this text paragraph I’m typing right now needs to know that it’s the:

last paragraph of current text of post window “We Need a Hero” of application “MarsEdit”

See, that’s how AppleScript refers to things. And Cocoa Scripting exposes objects directly to AppleScript, so that scripters can manipulate them directly. Long story short, is your Cocoa objects need to know how to tell AppleScript where they live. The way this happens is by way of the “objectSpecifier” method, which most of your scriptable objects will end up needing to implement.

This was a big mental block to me, because I’m not used to “back-referencing” from my objects to their containers. Maybe it’s my application design naiveté, but I find it awkward that in FlexTime for instance, my “cue action” object should know that it’s owned by a particular activity in a particular document. This just seems clunky to me. But as far as I know we’re stuck with it.

But just because the object exposes the objectSpecifier method, doesn’t mean it has to be in charge of it. I’m more comfortable in general with the containing object being responsible for claiming and disavowing ownership of a given object. So I came up with a generic solution that works pretty well for me.

RSContainableObject (Free, MIT License) is a generic NSObject subclass that can have its owning object and relevant key set on it. Then, when somebody asks it for its objectSpecifier, it uses that information to provide the required directions. The actual specification is accomplished by asking the containing object to reckon the object relative to itself, so the parent must itself respond to “objectSpecifier,” either because it’s also an RSContainableObject, or because it implements the method itself.

This approach works for me because I discovered that the vast majority of scriptable objects in my applications are in fact direct subclasses of NSObject. Those that are not, like my NSDocument subclass, inherit the objectSpecifier magic directly from Cocoa. So my typical set up is an NSDocument subclass that contains trees of various RSContainableObject subclasses, which all know how to reckon themselves recursively back up the chain in terms of their container.

These custom model objects end up with declarations that look something like this:

@interface TimedEvent : RSContainableObject <NSCopying>

Then when such a model object is set as a property or added as an element of a containing object, the container takes responsibility by setting the pertinent ownership terms on the object:

// Associate the object with us
[newEvent setObjectContainer:self 
			containedByKeyName:@"timedEvents"
			asToManyRelationship:YES];	

The above example is for a “to many” relationship, meaning it’s a member of a list of items. But the same event object could be associated as a property relationship with a similar type of pattern. Here I pretend there’s an attribute of this object called “best damn timer” with an associated Cocoa method of “bestDamnTimer”:

// Disassociate from the old object
[mBestDamnTimer setObjectContainer:nil
		containedByKeyName:nil
		asToManyRelationship:NO];

// Retain & Release
[mBestDamnTimer release];
mBestDamnTimer = [newTimer retain];

// Associate
[newTimer setObjectContainer:self 
		containedByKeyName:@"bestDamnTimer"
		asToManyRelationship:NO];

By factoring the messy “objectSpecifier” code into one place for a wide variety of use scenarios, I avoid having to engage too often in the (for me, at least) bug-prone ritual of writing that method from scratch.

You might observe that a problem with this approach is that objects can’t be owned by more than one object. That’s true, but that’s also a basic limitation of AppleScript. Objects have “one true specifier” even if there are multiple legal specifiers that lead to it. For instance “default timer of document 1” might resolve to the same object as “timer 3 of application ‘ClockThing'”, but the RSContainableObject relationship should only be between the application and the object. A given object at one time can have only one “canonical” object specifier, and that is the relationship that this class affords.

Hope this helps…. but we still need a hero!

Update: A couple readers have pointed out that I might as well link to some good resources for AppleScript debugging and design:

TN 2106 – Scripting Interface Guidelines. This has a good discussion of the containment hierarchy, what it means for your application, and how it relates to the world-view of the scripter. Thanks, Erik Wrenholt.

TN 2124 – Mac OS X Debugging Magic. This talks about general techniques for debugging AppleEvents and also reminds us of the Cocoa Scripting debugging “defaults” setting NSScriptingDebugLogLevel. I always forget about that and maybe I wouldn’t be quite so tightly wound up about AppleScript if I remembered it! Thanks, Jonathan Wight.

59 Responses to “We Need a Hero”

  1. Andrew Says:

    Oh my, I can only imagine the uproar from the prepress industry if Applescript was abandoned for something else. The number of man hours that have been put into Applescript workflows within the industry is simply astonishing. I have personal experience of this both as an application developer looking to add Applescript support to applications and as a user trying to script Applescript workflows. I wholehartedly agree that Applescript needs an overhaul but backwards compatibility is a must.

  2. Guy O'Rojo Says:

    Hi:

    As a non-programmer bridging the design and database worlds in a production environment, I NEED APPLESCRIPT! Why? Because I don’t think with arbitrary symbols or in dot syntax. I am a logical fellow, I think in English, and I don’t have a degree in Computer Science or Mathematics.

    But I do know what I want, and I know how I want it to happen. Applescript is the only thing that lets me automate my processes easily, quickly, and flexibly.

    Thanks,

    guy

  3. Allan Odgaard Says:

    Peter Hosey: At first I was certain that it was an Adium bug, but I later ran into a very similar problem with another application where exactly the same workaround fixed it.

  4. James Hober Says:

    Yes, AppleScript implementation is too hard.

    Since pre-history (CodeWarrior PowerPlant) it has always been so.

    When OS X was first released it came with a cool program called Interface Builder. Wow, I could wire up a GUI in no time! I told Apple, this is the way you should do AppleScript implementation. Let me wire up a Scripting UI the way I can wire up a Graphical UI.

    Well, we’ve since gotten bindings, Core Data wiring of the model, KVC (all great stuff), but still no way to wire up AppleScript support. (I don’t doubt that the problem is hard but solving it could mean an explosion of AppleScript support by developers.)

    Suggestions to Apple:

    – offer an IB-style way to implement AppleScript.
    – offer extensive sample code illustrating anything that can’t be wired up.
    – offer great debugging for the implementer and for the scripter (even if it lifts the curtain a little on opaque AppleEvent structures).
    – continue to encourage English-like termininology that is as consistent across applications as possible.
    – make Cocoa applications automatically recordable with respect to moving, closing and minimizing windows, etc. This means taking some of the recordability implementation for the Finder and moving it into NSApplication or some other Cocoa implementation class. Then we can get base recordability for free and implementers need only make their specialized events recordable.
    – fix bugs.

    Suggestions to implementers:

    – always include example scripts. I stick mine right in the Help documentation that the user can pull down from the Help menu.

    – make your terminology as English like as possible and as consistent with other apps as possible.

    – make your custom AppleEvents recordable (without trying to do what should be Apple’s job of making moving windows, etc. recordable.) I gave a simple example of how to do that here: http://lists.apple.com/archives/applescript-implementors/2006/Mar/msg00024.html where I construct the AppleEvent in Cocoa and use a single Carbon call to send it. Another example can be found here: http://lists.apple.com/archives/applescript-implementors/2004/Nov/msg00014.html

    – communicate with scripters and find out what they really want. The whole point of AppleScript implementation is for the scripter. We must listen.

    Finally, I want to send huge thanks to all who contribute to the AppleScript implementers mailing list. The Apple engineers who contribute there are enormously helpful and do so on their own time! If they haven’t been around as much lately, we can only hope that they are very busy and hard at work making AppleScript better.

  5. Brennan Young Says:

    Very interesting discussion here. How wonderful to see Sal responding to it so positively.

    I am a scripter rather than an ‘implementor’ but I have worked closely with various developers as they struggled to implement AppleScript in their apps. It has always been a world of pain.

    Here are a few points I’d like to raise:

    AppleScript implementation needs to be a ‘no-brainer’, and preferably something which happens ‘automagically’ in Xcode (at least for Cocoa apps).

    Apple should lead the way with its own AppleScript implementations. There are a few good ones (e.g. QuickTime Player), some really bizarre ones (TextEdit) and plenty of quirky, abandoned monsters (e.g. AppleWorks).

    Far too many important (and expensive) Apple apps (e.g. pro audio/video) have no AppleScript support at all. This is a very poor show. The pros that need this stuff as much as the amateurs!

    Some of Apple’s own scriptable apps go directly against the design guidelines described in the famous TN2106. (e.g. iTunes). This should be fixed, even if it breaks older scripts. I think at least we should hear some good excuses for why the iTunes dictionary has both ‘updatePodcast’ and ‘updateAllPodcasts’. (And what’s with the ‘camel case’? What a howler!).

    Applescript is deliberately designed as a ‘little’ language (i.e. it’s mostly just the core syntax, with the API provided by app developers), but there are a small handful of half-implemented ‘core’ features that for over a decade have shown tremendous promise but never been fleshed out.

    I am thinking (for example) of the ability to use the filter reference form (‘whose’ clauses) on scripters’ own lists, records and objects, or the ability to work with multiple text item delimiters. These things would be a tremendous boost to both scripters and implementors and while such features may not sell any additional iMacs, they might shift a few more high-end machines, and besides surely not rocket science. These features aren’t just scripter fantasies, they are tantalizingly half-implemented. Something about spoiling the ship for a ha’pe’th of tar.

    And Sal, or anyone at Apple, can we PLEASE get the Cocoa ‘default’ scripting implementation tidied up and properly documented. (Mostly the ‘Text Suite’) It promises much but in most cases it’s really not doing what anyone wants. Some developers even have the gall to say their app is ‘AppleScriptable’ when in fact it merely has the ‘default’ cocoa dictionary, which is often not even connected up to anything. Apple could take steps to make this less ‘acceptable’.

    I respect the need to keep details of unreleased products but we still need to be reminded that ‘something is happening’. That means decent, up-to-date docs, example projects which compile and run ‘out of the box’, and active participation from Apple Engineers on the mailing lists and other online fora. Thank you, Sal, for participating here. It means a great deal more than you may imagine!

    And when are we going to get AppleScript on the iPod? ;)

  6. nik Says:

    And now there also is a Ruby/Apple-event bridge by Hamish Sanderson:

    http://rb-appscript.rubyforge.org/

    It’s already used by the safariwatir project, btw!

  7. Steven Says:

    Sal wrote:
    Regarding Python: I’m curious, has anyone here tried the Python OSA component at:
    http://www.python.org/download/mac/
    Would that address your issues if it, or something similar, was included in the OS?
    ##############################
    ############################
    Yes!! Please!!!
    I saw this statement and I jumped for joy.

    Something like appscript and Pyobjc that you guys would fold into the os and mantain, not instead, but in addition to AS. That would provide the best of both worlds for everyone, AS for the novice and for those expert users with legacy needs and Python for those of us who prefer a more traditional programming aproach . But documentation and examples are paramount. How about getting that in 10.5 ?
    Best regards,
    Steven Quinones-Colon

  8. Mark Myers Says:

    I “cut my teeth” in programming using AppleScript to automate prepress workflows for the newspaper industry, and have since added perl and am currently adding Cocoa/Obj-C to my toolset. While agreeing with the frustration levels many in this post have experienced in writing AppleScript, and understanding the headache it must be to add AS support to Cocoa applications (I have yet to cross that bridge), I see AppleScript as a continued necessity for good Mac apps. Why? Because it is an end-user, doesn’t-have-to-know-computer-programming, language for “the rest of us.” If I had not had AppleScript as a gentler learning curve into a broader world, I don’t think I would have bothered trying. And I know of many others whose experience was similar. The truth is, abstracting the complexity of controlling applications to the end-user level necessitates a language that doesn’t require a complete shift in syntax rules from what the average Joe already knows — i.e., English.

    I do agree that AppleScript needs some work. Better list manipulation, a regular-expression syntax, stronger support in the finder, more recordability, and a LOT more speed are things that I would give my right arm for. As for standardization, I think much of the problem is the result of third-party developers not putting in the effort to compare their applications’ AS behavior with existing applications that have rich, intuitive scripting dictionaries. It would help if Apple did some work on standardizing and documenting the way dictionaries should be written, but in a somewhat obscure way standards do exist. Back when I started, QuarkXPress was considered a standard bearer in scriptability (although InDesign has far surpassed it), and there are many apps today (thanks, OmniGroup, Adobe, and others!) that other developers should be looking to in order to see ways most AppleScripters expect their precious language to be implemented. Perhaps we need an Indie AS Implementation guide similar to the Indie HIG.

    I have two suggestions, from an end-user’s perspective, to developers when contemplating adding AppleScript support:

    1) Expose as much of the functionality of your application as possible to the OSA engine. Please don’t leave needed functions “in the dark” from an AS point of view. A good bad example of this is the more recent iterations of QuarkXPress (I’m thinking of 6.5) where they added a wonderful PDF Export ability but did not make that function scriptable, forcing us AppleScripters to continue to rely on printing or exporting as EPS, and then passing the resulting file into Distiller. And anyone who has ever tried scripting QuarkXPress printing can attest that it is wildly unreliable at best.

    2) Use terminology in your dictionary that mimics the same concepts reflected in the GUI. In other words, write your AppleScript syntax in such a way that if I understand how to do a particular process from the GUI, I can easily map that knowledge to the identical feature in AppleScript. This does not mean scripting button clicks and palette openings, but if I want to, say, create a 2″ by 2″ text box 1″ from the side of the page and 3/4″ from the top, I shouldn’t have to learn an obtuse reference form to do so. AppleScript implementation should be as elegant and well-thought-out as the GUI.

    Lastly, I should note that I strongly disagree with posters suggesting this or that framework for adding support for specific scripting languages to applications. The LAST thing we need is for every developer to decide what scripting language their application is going to use and essentially isolate it from the rest of the scripted-application world. One of the most powerful features of the OSA is its ability to pass instructions from one application to the next. Most AppleScript workflows I and others have created use at LEAST two applications. Even if you prefer to use a different scripting language yourself, please take the time to implement the OSA in your app. This way, using an OSA component, anyone can write their scripts in whatever language they prefer. And us AppleScripters can continue on in our blissful, English-like world.

  9. flight16 Says:

    I just logged in at http://developer.apple.com and as of 3-5-2007 they have a few new sample AppleScript apps with code, complete with detailed(?) readmes. I haven’t sat down with any yet, but they look promising… at least for an AppleScript newbie like me.

Comments are Closed.

Follow the Conversation

Stay up-to-date by subscribing to the Comments RSS Feed for this entry.