Front Window Analysis Tool Using AppleScript System Events (List UI Elements)

ccstone · February 24, 2021, 11:04pm

Window Analysis Tool for System Events v1.5

Hey Folks,

This macro gets a bunch of information about the front window in the front application (specific to System Events UI-Scripting).

~~At present this is raw information that won't be very readable by the uninitiated, although I'll probably change that in time by finding and replacing the chevron-codes with their English equivalents.~~

Fixed in version 1.5 below.

The purpose of the macro is to discover whether System Events can see UI-Elements in the front window. This is very useful, because System Events can see and do things Keyboard Maestro can't.

-Chris

Updated: 2021/02/24 17:02 CST by @ccstone

Please see: The latest version (v1.5) uploaded by Chris on 2016/06/13.

Also see: The Hierarchical UI-Element View using Script Debugger.

Also see: Alain's mod to enable code folding in BBEdit.

Window Analysis Tool for System Events 1.00.kmmacros (3.5 KB)

DanThomas · June 11, 2016, 11:37am

How is this different from your other macro (which I use, by the way)?

ccstone · June 12, 2016, 11:36am

Hey Dan,

Front window only in the Front application. No need to designate an app – just have it frontmost.

I’m also getting the entire contents of the window – not just its properties – so we’re looking at all the UI-Elments of the window

-Chris

[ Edited: 2016/06/12 06:36 CST ]

JMichaelTX · June 11, 2016, 9:03pm

Thanks for sharing, Chris.

You can count me in the group of “uninitiated”.
Your report reminds me of the first time I did a hexadecimal dump – OK, now what?

I can understand the potential. We will probably need some help in translation and usage.

Since you can get all of this info in a better format from UI Browser, do I assume correctly you are just trying to offer an alternative?

Oh, one thing. It opens the text file in TextEdit, even though my default app for “.txt” is TextWrangler.

DanThomas · June 11, 2016, 9:18pm

What, you can’t read hex? Come on!

I still remember laughing when I saw a Z28 with a license plate that was (I think this is right) “5AF2F8”, which again IIRC is EBCDIC for “Z28”. The EBCDIC numbers were the giveaway, since they start with “F”. Back in the day when EBCDIC was still a thing.

JMichaelTX · June 11, 2016, 9:23pm

It’s like some of my other skills – use it or lose it.

DanThomas · June 11, 2016, 9:25pm

Uh, that’s not the only difference. The other macro produces results that make sense to me. This, as @JMichaelTX said, takes me back to reading HEX dumps. No, that’s not right - reading assembly language. I’ll bet there’s some NOP and HCF codes in there somewhere!

ccstone · June 12, 2016, 11:55am

Hey JM,

I’m not offering an alternative – more like a litmus test (although there is potential for more than that).

99.9% of people won’t use the UI Browser utility, because it’s too expensive and/or too intimidating.

This macro gives me a diagnostic tool that anyone can use by pressing a keyboard shortcut.

By looking at the report I can immediately have a pretty good idea of whether or not GUI-Scripting is possible for the given window in the given app.

That’s because there’s a big difference between the -f and -t options in the shell’s open command.

-t looks at Launch Services to figure out what editor is set for text files “in the Finder”.

It also takes input from STDIN, which is why I used it in this instance.

-f looks at the default editor in the shell’s setup files. By default that’s TextEdit.

You change it you need to add a line to your ~/.profile (or other shell configuration file).

export EDITOR="/Applications/Applications_Chris/Text_Editors/BBEdit/BBEdit.app";

open -f "/posix/path/to/the file.txt" # Input is a posix-path *not* STDIN.

-Chris

JMichaelTX · June 12, 2016, 6:26pm

Chris, I’m glad you can, but I’m not sure the rest of us could.

Any chance you could work your RegEx magic and produce a simple statement at the top as to whether or not the window is GUI-scritable?

If so, a report like the one UI Browser produces would be really helpful.

Here’s an example:
EN Mac 6.6 UI Elements.txt

JMichaelTX · June 12, 2016, 6:39pm

Here’s a related script from MacScripters.net that I have used a few times.
The output is readable.

Coerce GUI scripting information into string?

ccstone · June 13, 2016, 1:59pm

Hey Folks,

Updated to version 1.01.
Now requires Yosemite or greater.

It's faster and many of the chevron-codes are now replaced with human-readable names. (There are a lot of codes, so I certain to have missed some.)

I've added a property at the top of the script that allows switching the output between TextEdit and the default text editor set in the Finder's Get-Info dialog. (If you haven't changed it on your system then the default is TextEdit.)

-Chris

Window Analysis Tool for System Events v1.01.kmmacros (7.3 KB)

DanThomas · June 13, 2016, 2:06pm

Sow how long before it can generate AppleScript or JXA code to manipulate these things?

Just kidding. Well, kind of, anyway. Welcome to Peter’s world of “it’s never enough”.

alain · June 13, 2016, 8:05pm

«class splr» 1 of splitter group 1

is one of this in Path Finder

Chris, do you look for such feedback?

–Alain

ccstone · March 2, 2017, 5:59pm

Hey Folks,

Changes to this version (1.50):

Faster.
No more chevron-codes to worry about.
Hierarchical view.

In my opinion BBEdit (or its freeware sibling TextWrangler) is a better vehicle for viewing this output than TextEdit. One reason for this is the ability to turn on “Show Tab Stops”, which makes the view of the hierarchy clearer (some other programming editors also have this feature).

-Chris

Window Analysis Tool for System Events v1.50.kmmacros (7.6 KB)

ccstone · June 13, 2016, 8:13pm

Hey Alain,

Sure.

But that version is obsolete now.

-Chris

alain · June 13, 2016, 8:44pm

I cannot stop me to show an example of use of this so nice tool (I have needed) for UI players:

--Alain

ccstone · June 13, 2016, 8:52pm

Here's what it looks like with “Show Tab Stops” turned on instead of “Show Invisibles”.

DanThomas · June 13, 2016, 9:43pm

You know, it's funny you posted this. I was wondering today if there's any sort of native way to display hierarchical data on the Mac. For example, here's the plist view from Xcode:

Another possibility would be to output XML. Most browsers support reading XML and collapsing nodes, etc. I could probably shake the dust off my XML skills if you needed help.

In any case, great work!

DanThomas · June 13, 2016, 10:29pm

Scratch that. Atom has a decent collapsible view. Good enough.

ccstone · June 13, 2016, 10:42pm

Hey Dan,

Interesting thought.

Here's how it looks in Script Debugger.

tell application "System Events"
   tell application process "TextEdit"
      front window
   end tell
end tell

I'm not going to fiddle with transforming it to XML at the moment, but I have an osax that might make that easy – so I'll think about it.

I'm not aware of any freeware hierarchical viewers for the Mac other than Xcode. I have a great utility called PlistEdit Pro that's much lighter weight than Xcode (but also more expensive).

-Chris

Front Window Analysis Tool Using AppleScript System Events (List UI Elements)

Options