How to get full text specifier of an AppleScript object?

I am looking for an easy (or even not so easy) way to get the text specifier of an object in AppleScript.

Consider the following:

tell application "System Events"
	tell process "Terminal"
		set theContents to entire contents of window 1
		set theObject to item 5 of theContents
	end tell
end tell

Using Script Debugger I know that the specifier for theObject is:
scroll bar 1 of scroll area 1 of splitter group 1 of window "title" of application process "Terminal"

I am looking for a way to recover that specifier from within AppleScript. In other words, given theObject, I want to recover the full text of the object specifier.

The obvious approach is a simple recursive routine that starts with theObject and follows attribute "AXParent" back until arriving at the application process. The difficulty is that while the class of each object going back up the chain is available, I'm not sure how I'd get the full specification.

For example, I could tell that an object has class button, and has a parent with class splitter group. It's less obvious how to tell the difference between button 1 of splitter group 1, and, say, button 7 of splitter group 43.

Thoughts? If somebody is sitting on a piece of code to reconstruct the whole tree of an application, that would certainly serve.

(The use cases would include things like a KM macro that reads the location of the mouse, looks through entire contents of a window for the object with the most restrictive frame that contains the desired location, then returns the AppleScript specification of that object. Yes, I know about UI Browser. I also know that UI Browser is no longer supported, and in any event would like to be able to grab the specifier directly into a variable.)

You're mistaken. Mark Alldritt has taken up supporting UI Browser.

See this thread and pay especial attention to @cjk's object-specifer macro.

You can't directly (unfortunately)...

But you can wrap it in a try-on-error handler and use the error reporting mechanism to create some parseable text.

tell application "System Events"
   
   tell application process "Script Debugger"
      set theObject to first button of front window
   end tell
   
   try
      theObject / 0
   on error errMsg
      return errMsg
   end try
   
end tell
1 Like

Supported, trial version available -- unfortunately no longer available for purchase.

UIElementInspector is still available from https://developer.apple.com/library/archive/samplecode/UIElementInspector/UIElementInspector.zip, and still works in Ventura -- you'll have to right-click Open it the first time because of security changes, and the alert about System Preferences/Assistive Devices should point you to Privacy & Security->Accessibility.

Very clunky compared to UI Browser -- but it does work.

1 Like

Whups, I misread – I thought the source was going to be released as open source...

Well drat! :frowning:

You can always use the demo and then use an uninstaller like AppDelete to remove it when it times out. (I think that still works, but it's been some time since I tested.)

1 Like

You gave me the answer that I was expecting, I suppose.

When Bill Cheeseman announced that he would no longer be supporting UI Browser, he said that he was making the source available. When Alldritt picked it up he reversed that and said that the source would not be available. When I said "no longer supported," I meant just that. The page on Late Night Software says that they will continue to make it available to those who already have licenses, but that they will not make those licenses available and they offer no support. → Unsupported.

Agh! The UIElementInspector is awful, but it does have its uses.

The Accessibility Inspector.app bundled with Xcode is far better but still falls far short in terms of usability.

1 Like

This macro works with macOS Mojave – I'm not sure about later versions of macOS.


Download: Identify UI Element v2.00.kmmacros (29 KB)

Macro-Image

Keyboard Maestro Export

Macro-Notes
  • Macros are always disabled when imported into the Keyboard Maestro Editor.
    • The user must ensure the macro is enabled.
    • The user must also ensure the macro's parent macro-group is enabled.

System Information
  • macOS 10.14.6
  • Keyboard Maestro v10.2

1 Like

The problem is easy to tackle from the other direction. Mapping from the top down is straightforward. A simple recursive handler that follows the AXChildren attribute does the job. It might be fun/useful to write such a thing that returns a map of an application in JSON format.

As a proof of concept, the following generates a tree-ordered list of the class of every object. It's not useful as such, and takes forever to run.

set allElements to {}

tell application "System Events"
	tell process "Touch Portal"
		set w to window 1
		my getChildren("Touch Portal", w, allElements)
	end tell
end tell

on getChildren(procName, uiElement, allElements)
	tell application "System Events"
		tell process procName
			set theChildren to value of attribute "AXChildren" of uiElement
			repeat with theChild in theChildren
				try
					set end of allElements to class of theChild as text
					my getChildren(procName, theChild, allElements)
				end try
			end repeat
		end tell
	end tell
end getChildren

The results seem to match the order of items in entire contents. Not sure it would be faster but one could use the contents of AXChildren and class for each item in entire contents to reconstruct the hierarchy, then pull out position or whatever else one might be interested in for each element.

Might make a handy macro. I'll add that to the "Round Tuit" list.

Thanks for the macro! I didn't see it pop up while I was playing with my idea. I'll give it a look!

O Caveat Emptor understood, but when I run the macro the AppleScript has a compile error.

/var/folders/8_/dcx58qzd3yb2t9st4h6rbtmm0000gn/T/Keyboard-Maestro-Script-624D688F-3985-42A4-B454-F0F4C404A5C6:3851:3856: script error: Expected end of line but found identifier. (-2741)

Looking at it with Script Debugger, the error occurs in the line change _find into _replace in _data with regex without case sensitive in the definition of on cng().

I'll wait until I upgrade to Ventura later this week and try it again.

Do you have the Satimage OSAX installed?

Unlikely, but I do of course...

I'd intended to rewrite that script to remove the dependency and forgot that I hadn't done it yet.

It'll take me a bit to get it working with other tools.

AppleScript isn’t my go to for that sort of computation, but it might be handy. How useful have you found Satimage OSAX?

If you're past High Sierra it's probably not worth your while – I had to play some games to use it on macOS Mojave.

Mark Alldritt created a work-around for later versions of macOS, but there's no telling how long this will continue to function. (As long as Apple continues to support Carbon Libraries I think.)

I've used the Satimage.osax since 2003 and have found it utterly indispensable. Apple should have bought it and officially rolled it into AppleScript.

Mark's version actually includes ALL of the various Satimage Osaxen, and they add a whole lot of functionality to AppleScript.