Scripting Word



Scripting Word

Now, this is weird. My more recent tasks have involved working with both
the Word testers and the automated testing group to iron out some ways that we
can improve on testing. In particular, now that Word has a more complete
AppleScript implementation than previous versions, we can move from automated
testing based on VBA (Visual Basic for Applications) to automated testing based
on AppleScript.

Up until now, I’ve dabbled in AppleScript, but as someone accustomed to more
conventional programming languages, I’ve found AppleScript to be a wee bit
inscrutable. It’s unfortunate that there really is no standard implementation
of, say, the Text suite such that one way of doing things with, say BBEdit,
might also work with something else, like CodeWarrior’s IDE. Alas, AppleScript
dictionaries don’t quite measure up to VBA’s object browser.

This is particularly difficult for me, because I have some preconceived
notions of how a text editor should be automated. More than simply the fact
that I’ve worked on Word for a few years, I was one of the team members who
implemented the original object model for Word’s implementation of VBA. I did
all of the table-based objects, and the paragraph formatting object (including
some work with styles). I also did a good deal of work on the underlying
plumbing that routes certain methods and properties through Word’s normal
command dispatch mechanism (i.e. the code that runs when you do something like
select “Font” on the “Format” menu).

Anyway, when it comes to AppleScript, I’m definitely in learning mode. So,
I’ve been googling a bit about AppleScript, and what do I run into but John
Welch’s very
excellent piece about
AppleScript and KeyChain, which happens to begin with
a link to my href="http://blogs.msdn.com/rick_schaut/archive/2004/05/19/135315.aspx">
Anatomy of a Software Bug. And, yes, John, Paul Berkowitz is a genius.
There’s a reason Matt Neuburg devotes almost an entire paragraph of his
acknowledgements in AppleScript: the Definitive Guide to Paul.
(And, no, Paul, I haven’t forgotten the question you asked about typography in
Word and the possibilities for improvements. I’m wrestling with just how much
I can talk about at this stage of the game. As my father would say, when in
doubt, mumble.)

But the genius I want to call out today is Jim Murphy. While the rest of us
were busy enhancing our applications from the user’s point of view, Jim, almost
single-handedly, put together the AppleScript implementations for Word, Excel
and PowerPoint. I’ve spent the past couple weeks sanding off some of the rough
edges (particularly the Style object), but the sandpaper I’m using is very
fine-grained due in no small part to just an incredibly outstanding job that
Jim did on this enormous effort. While I shan’t fail to acknowledge the role
Paul Berkowitz played in helping define how the implementation should behave,
for which AppleScript programmers around the world owe an eternal debt of
gratitude, Jim is the person who made it happen.

If you’re doing anything having to do with trying to script Word 2004, I
want to hear from you. You can add comments here, but you can also get in
touch with me by clicking on the “Contact” link in the “My Links” list in the
left-hand column of the web page that has this article. I’ll also be attending
WWDC this coming week, and you’ll probably find me in some of the scripting
sessions.

I want to hear both the problems and issues you’ve encountered as well as
some of the successes you’ve had. I want to know both the things you’ve found
difficult and the things you’ve found to be easy.

 

Rick

Comments (19)

  1. Dale Gillard says:

    AppleScript is different because of its English like syntax. But as with anything, you get used to it.

    I’m waiting to get my Office upgrade (the Pro update is delayed), so I can’t have a play with the new AppleScript implementation. One thing I’d love to see is some online info about AppleScript in Office. For example, making the application dictionaries available online in a xhtml/PDF/.doc form, and maybe getting Paul to write a few examples for each dictionary to kick start things.

    I went Googling last night and could only find scant mention in MS press releases.

    Cheers.

  2. Rick Schaut says:

    Dale,

    Perhaps you can download the test drive version of Office 2004? I just checked the setup build, and it has the full dictionary.

    And, I don’t mind the English-like syntax. What I mind is how difficult it is to figure out what that means for each scriptable application. There really seems to be a lack of any sort of standard by which to measure what should and shouldn’t work.

  3. There technically is. But as you’ve seen, no one really follows it.

    john

  4. chris says:

    True Im a sun solaris engineer and shell script prog, most of what I do baffles the mcse’s. What you use most always seems better.

  5. Dale Gillard says:

    Hi Rick

    I’ll wait for a magazine CD to try the Office 2004 demo. I’m using a 56K dialup connection so the 186MB/16 hours is too long. (This was reason for seeking out the documentation.)

    Thanks.

    Dale

  6. Paul Berkowitz says:

    That’s really too kind of you, Rick, to keep mentioning me this way. I did make some (pretty important, true) requests to Jim Murphy to help get Word’s AppleScript functioning in a way that scripters would appreciate and would expect of a well scriptable application – ways which differ from the object model that Word’s new AppleScript mostly shares with its VBA model. (I, too, am not sure, how much I’m allowed to say.)

    But Jim is simply amazing. I don’t think many people can appreciate what he did. Word’s dictionary is, on its own, much larger than any other I’ve ever seen – there are some 520 events and classes! And some of those have 50-100 enumerations, 10-20 optional parameters, and so on. Then there’s Excel, which has to be more than half as big, and PowerPoint on top. Murf (that’s Jim Murphy) did ALL of that. Plus implemented plurals-as-list, properties of ‘every element’, whose clauses, default properties for all classes…and on and on. It’s simply amazing what one man – Jim Murphy – accomplished. And it seems very solid too – some of that must be due to the testers assigned to it, who were exhaustive. Great credit must go to MacBU itself – the program managers and development managers who authorized that precious commodity – developer time – pretty well the full time of one of your small set of developers – to this project. Who would have believed that a few years ago, when Word’s AppleScript was languishly in a paltry state and no one seemed to care? I have a feeling that a certain amount of credit must go to someone who has since left MacBU – Michael Connelly – but who authorized the project a few years ago and got the ball rolling. I’m sure I must be omitting others, too, whom I simply don’t know about. But Murf is the one who did almost all of it.

    It’s a little ironic that you say "It’s unfortunate that there really is no standard implementation of, say, the Text suite such that one way of doing things with, say BBEdit, might also work with something else, like CodeWarrior’s IDE." Actually, there _is_ a fairly standard way of implementing the Text Suite: see Tex-Edit Plus for a model version, with a few of its own additions. It’s the essence of AppleScript that applications should mirror their own object models – and all applications differ from each other or they’d be the same application – while attempting to use the structure of the Standard Suite and – yes, the Text Suite – as much as possible. The diversity, and relative freedom, is what AppleScript to be a universal language on the Mac – with a far larger percentage of major non-Apple applications implementing it (their own versions) than non-Microsoft apps implement VBA on Windows, after all.

    It’s Word 2004 which departs far, far more than most apps in its implementation of the Text Suite – and for good reason. The Text Suite of the old (pre-2004) Word looked like a model AppleScript Text Suite – it shared the terms, look and feel of other script editors. And that’s precisely what played a large part in its instability – all the crashing – I’m quite sure. The core Text Suite was designed for single-byte text editors like Tex-Edit Plus and BBEdit. Word is a much, much more complex binary application. It has oodles of hidden chartacters and who knows what else (well, you do) beneath the surface. In the old Word, you couldn’t even rely on selecting word 4 of paragraph 9: it was always ending up in the wrong place, before crashing. I’m sure some of that had to do with trying to squeeze this complicated double-byte structure into the confines of the basic Text Suite. (There must have been a lot more, too.)

    I was originally disappointed to hear that the AppleScript object model would be "parallel" to the VBA model, because I was aware of how much that would differ from conventional AppleScript. But Murf has done a really great job, and I’m now convinced that this was the right way to go about it: we get this immense powerhouse of automation – virtually the entire application and all its preferences (who ever heard of another app scripting preferences like this?) in a structure that’s solid because it’s based on the same model that underlies the tried and true VBA.

    However, it does mean a lot of work for scripters. Word’s model is like a whole new dialect of AppleScript. Everyone had better start bonong up espcially on ‘text range’ class and the associated ‘text object’ property; that’s where all text editing takes place. and get used to ‘find’ as a class, not an event. There’s just so much hidden there.

    I do kick myself for not thinking of one thing to suggest to Murf, although it would probably have already been too late by the time I came on the scene. AppleScript does not have any punctuation, like VBA’s ":=" to link parameter labels of commands (events) with their values. They just "run on" one into the other. That’s because it’s meant to read more like English. But Word has far, far more command parameters than most scriptable apps. And these parameters, and most properties of classes, often take explicit ‘enumerations’ – unique named constants. These enumerations often have the same words as the parameters themselves repeated again perhaps in a different order. So you can end up with a statement such as:

    set myTable to convert to table myRange separator separate by tabs table format table format simple2 with auto fit, apply heading rows, apply borders and apply font without apply first column

    This is a real line (yes, all one line) from one of my own scripts adapted from its ‘do Visual Basic’ version for Word X to the new AppleScript in Word 2004. Now that’s _not_ very English-sounding, is it? What I should have suggested was to make all non-Boolean parameters begin with a preposition. That’s how to make it sound like English. (Again, check Tex-Edit Plus’s dictionary.) And it’s what’s been done in a few case, e.g. expand. ‘expand by a character item’, etc., reads much better than ‘expand expand a character item’ :would have.

    When all is said and done, however, this new dictionary is so enormous that it would be quite unwieldy no matter what style its enumerations took. It will bear verydetailed study. We’ll all be the better off when the Office AppleScript Reference is ready. Until then, I advise any putative scripter of Word to consult the Visual Basic Editor’s Help: it’s the only way you’re going to be able to figure out most of what’s going on: the terms are very similar: the arguments of VBA Methods correspond to the parameters of AppleScript events. People like you, Rick, who are already familiar with Word VBAhave a great advantage (except that most VBA programmers are content to stay in VBA and not venture into AS). When the Reference is out, the real scripting will begin in earnest.

    As Rick says, we should all be grateful to Jim Murphy. I think the scale of hos achievement will only begin to be fully appreciated years from now when people really get up to speed on the object model.

    That’s more than enough for now. If I don’t get some sleep I’ll never make it to WWDC tomorrow. See you there, I hope…

  7. has says:

    > I’ve dabbled in AppleScript, but as someone accustomed to more conventional programming languages, I’ve found AppleScript to be a wee bit inscrutable.

    First thing is to learn to distinguish between the AppleScript language and application scripting. The AS language tries to hide the distinction, but doesn’t do it very well and generally creates more confusion than it solves.

    > It’s unfortunate that there really is no standard implementation of, say, the Text suite

    [This is part of application scripting, not the AppleScript language.]

    There is… well, a bit anyway. :)

    The classic Mac OS defined some standard terminology (sans implementation) for common functional areas like text, database and graphics that application developers were encouraged to follow. You can find this stuff buried in the AppleScript component’s own dictionary, still causing confusion amongst users to this day who can’t understand why AS defines all these keywords that normally don’t do anything.

    In OS X we now have Cocoa Scripting’s Standard and Text suites. The former is slightly ill-formed, defining more terminology than it really should (e.g. non document-based applications don’t need a document class and the events that operate on it), while the latter is extremely basic, lacking even some essential functionality (e.g. that ability to write a reference like ‘text 3 thru 7 of text of document 1’ to get a contiguous range of characters), never mind the bells and whistles that heavy-duty text editors and word processors will need.

    So, not much, but it’s something. This said, there’s probably not much point in Apple defining lots of bells-n-whistles suite definitions for application developers to follow, since every application has different needs and features. What they do provide is the Scripting Interface Guidelines, which should encourage different applications to at least ‘feel’ much the same as one another. Which might coincidentally increase the frequency with which different apps use the same terminology keywords to do the same things, which should at least give you a degree of polymorphic behaviour between apps from different vendors, even if it is completely ad-hoc. It’s something, anyway…

    > such that one way of doing things with, say BBEdit, might also work with something else, like CodeWarrior’s IDE.

    [This is really an AppleScript language issue.]

    And here we run into an even more annoying problem than the one just described. Deep breath now; here goes… :)

    One of the problems with Apple event-based IAC compared to other high-level RPC mechanisms like XML-RPC and SOAP is that Apple events use four-character codes rather than human readable names to identify commands, classes, properties, elements, etc. This is done for efficiency (the Mac OS uses Apple events for a lot more than just application scripting, and there’d be too much overhead in using arbitrary strings), but not very suitable for application scripting as you can imagine.

    Thus, Apple introduced application terminology (aete) resources as a means of translating between Apple event codes and human-readable names. Scripters use the long names in their scripts, e.g. ‘foobar’, and these are converted to Apple event codes, e.g. ‘Fobr’ at some point before use.

    Now, while application A and application B may both define the same long names, e.g. ‘foobar’, there’s no guarantee that they’ll both translate to the same Apple event codes; one might use ‘Fobr’, the other ‘fOOb’. This isn’t a problem when your scripting language waits until the last possible moment to perform the conversion, but AppleScript performs this conversion at compilation time for efficiency’s sake. So unless A and B by chance both happen to translate ‘foobar’ to the same Apple event code, then your AS code will only work with the application whose terminology it was compiled against; e.g.

    tell app "A" to foobar — [sends event ‘Fobr’]

    tell app "B" to foobar — [sends event ‘fOOb’]

    This makes it quite impossible to do something like:

    repeat with someApp in {app "A", app "B"}

    tell someApp to foobar

    end repeat

    Application scripting systems that leave the long name-to-AE code conversion until the code is executed don’t have this problem; they simply convert ‘foobar’ to whichever code is specified by the target application at the time:

    for someApp in [app(‘A’), app(‘B’)]:

    someApp.foobar()

    This is very nice, although you do inevitably pay a bit of a runtime performance penalty, especially if the name-to-code conversion is being done in interpreted code. But you pays your money and takes your choice. In an ideal world, Apple would provide a high-level ‘Application Scripting’ API that operated purely on long names and handled all the nasty AE code conversion stuff itself using highly optimised C code to provide both complete flexibility and good performance. Ah well, maybe some day…

    Anyway, I’d like to recommend some other languages for application scripting to you, but with the exception of UserTalk I don’t think any have mature, or even complete, application scripting support. Still, for what it’s worth, you might like to take a look at the application scripting support I’m working on for MacPython:

    http://freespace.virgin.net/hamish.sanderson/appscript.html

    It’s still got some way to go before it’s completely finished (offers of assistance gratefully received), but it’s mostly working already (it currently barfs on Word X’s terminology due to a known design flaw, but maybe Word 2004 will have more luck;) and is slowly getting there. It provides users with an interface that feels very similar to AppleScript’s, but with a number of advantages (sanity-checking, proper ‘polymorphic’ behavior, better application terminology renderer) and a very Python-like syntax that non-ASers will find much more familiar. Its stricter handling of application terminology also means it’s better at highlighting flaws in terminology resources than the rather more lax AppleScript; which could be helpful if you’re QA-ing the terminology definitions. Plus, of course, it lets you work in Python, which is one of the nicest mainstream scripting languages around and has lots of developer/tester-oriented libraries and tools available for it.

    HTH

  8. Paul Berkowitz says:

    Follow-up: an example of how a more readable Englisg style could have been applied. Instead of that fearsome:

    set myTable to convert to table myRange separator separate by tabs table format table format simple2 with auto fit, apply heading rows, apply borders and apply font without apply first column

    I quoted (note that only ‘myTable’ and ‘myRange’ are variables here; everything else is a parameter of enumerated value for the ‘convert to table’ command), a more user-friendly terminology making use of propositions for parameters and gerunds (‘…ing’) for enumerations that follow them might be:

    set myTable to convert to table myRange separating by tabs using table format simple2 with auto fit, applying heading rows, applying borders and applying font without applying first column

    That reads somewhat more like AppleScript is meant to. The underlying 4-character raw codes that has refers to would be exactly the same as they are now (Murf chose unique numbers for most of these), but the "surface" text would be more readable. It’s no easier, or different, to _write_ code this way than with the existing terms. But it is easier to read and maintain since it makes the AppleScript mostly self-commenting.

  9. has says:

    > But it is easier to read and maintain since it makes the AppleScript mostly self-commenting.

    Yeah. I think the key to picking good ‘English’ terminology is good ‘guessability’. And the closer the written command is to how you’d describe it verbally, the easier it is for the user to guess how to write it.

    BTW, not seen the Word 2004 dictionary myself, but just wondering if it really needs all the ‘applying’s, or would "…with auto fit, heading rows, borders and font without first column" be sufficient? Also, just mildly curious why the ‘separating by’ parameter takes an enumeration rather than a string?

    FWIW, while application terminology should be phrased to suit AppleScript (i.e. according to the SIG rules), seeing how commands look written in other languages might make for an interesting comparison, so here’s how Paul’s version would read in Python (assuming I’ve understood it correctly:):

    <code>

    myTable = myRange.convert_to_table(

    separating_by=k.tabs,

    using_table_format=k.simple2,

    auto_fit=True,

    applying_heading_rows=True,

    applying_borders=True,

    applying_font=True,

    applying_first_column=False

    )

    </code>

  10. timbo says:

    We don’t have 2004 yet in this office (we will soon), but I’ve been wanting to do the following with Word scripts for a while… but have never figured out how to do any of them… I don’t care if it is applescript or something else…

    1. To import an itunes playlist and format it to my preference (I have the playlists in a binder that I can look up)…

    2. To format page breaks in a screenplay… Basically it would look at the paragraph breaks at the bottom of each page. If a certain style of paragraph was broken up (ie dialog or scene description) and more than X number of lines long it would be broken and "(more…)" would be added to the bottom of the page. The top of the next page would say "(continued…)". There are lots of other factors to be considered, but this is the gist of what is going on… looking at page breaks, considering the styles of the paragraphs and either breaking or not breaking and formating properly.

    3. To use a batch of selected address book names in form letters…

    4. To take a batch of images selected in the finder and insert them into specific documents.

    5. To batch copy text an images from a word file into my blog…

    to name a few

  11. Peter Edman says:

    I started to play around with scripting today — I want to convert all the footnotes in a Gutenberg document into real Word footnotes, selecting all text between [ ] brackets and making into a new note. So far am completely stymied as Word tells me it "cannot make a new footnote" no matter how I try requesting things. My script compiles, but doesn’t run. But, I live in hope.

  12. Paul Berkowitz says:

    It’s tricky learning how to deal with the object model, especially things like text ranges, and footnotes all of whose properties appear to be [r/o] (read-only). But those properties may contain properties which are not read-only! Here’s how to make a new footnote at the cursor location:

    tell application "Microsoft Word"

    set oDoc to active document

    set f to make new footnote at oDoc

    set content of text object of f to "Here’s my first footnote."

    end tell

    It’s the ‘content’ property of text ranges (text object of footnote is a text range) which is what you usually want to set, and it’s always settable.

    It’s harder to figure out how to specify the location for the footnote if it’s not to be at the cursor. AppleScript dictionaries don’t tell you to which classes your class under investigation (‘footnote’ here) is an element. ‘document’ is obviously one such. But take a look at ‘text range’ itself. ‘text range’ can have footnote elements. And every sub-unit of a document – or most of them – has a ‘text object’ property of its own. So you could make a footnote at the end of the first paragraph of a document, no matter here the cursor might be, like this:

    tell application "Microsoft Word"

    set oDoc to active document

    set t to text object of paragraph 1 of oDoc

    set f to make new footnote at t

    set content of text object of f to "Here’s another footnote."

    end tell

    And you’ll see a new footnote number at the end of the first paragraph with your new footnote at the bottom of the page. Word is very clever about re-indexing the footnotes. If this new footnote is actually earlier in the document than the other one you made, this new one becomes footnote 1 and the other one becomes footnote 2. If you get the ‘entry index’ (definitely r/o) property of the footnote you can find that out too.

    ‘note reference’ property is a bit peculiar. It too is a text range, but if you get its content the result is "". I think that’s because footnote references are set to AutoNumber by default and these are considered to have no content, just an index. If you set the content of note reference of f to "•" for example, it does replace the number by this character but suddenly the footnote disappears. ‘footnote options’ lets you set the footnote numbering style, and that includes a ‘note number style symbol’ type. When I have more time I’ll look into that. Usually you’ll want numbers, in some style, for footnotes so probably the note reference remains "".