I downloaded the app, and my analysis tool works fine with it.
UI Browser has some problems with it though and can crash when trying to get certain information.
-Chris
I downloaded the app, and my analysis tool works fine with it.
UI Browser has some problems with it though and can crash when trying to get certain information.
-Chris
Hi ccstone, thank you for your response, how do I overcome the problem that I am facing?
Hey Ali,
How could I possibly know?
If I can't reproduce exactly what you're doing and test – then I can't help much.
You could join the newly formed UI Browser Users Group and provide more information about what you're doing.
Perhaps someone in the group has used Daylite.
-Chris
In fact using BBEdit as editor folding is just around the corner... the simple tuning is to change ".txt" file name extension in ".py" in this AppleScript handler:
Explanation:
The conventions to encode in a compact way the hierarchy of the GUI elements are identical to that of the Python code: it is therefore enough to take full advantage of this homology by transforming the temporary text file into a pseudo Python file, by changing the extension, so that BBEdit automatically activates the folding
Before:
After
For example, "scroll bar 1" elements are folded:
-Alain
Indeed!
Somehow I always knew that Python must be good for something.
Hey @ccstone sorry for maybe asking super basic questions about this macro... but I am like the rookie of the rookies of the rookies when it comes to this subject.
So my question: this macro only shows a list of elements, right? That's the sole purpose or am I missing something else?
Would it be possible for the script to identify the cursor's position and based on that, either show the information for that element only (similar to UI Browser) or even if it shows the long list, maybe add some unique information next to that element that we could easily search for using the Search bar inside TextEdit?
Again, sorry for maybe some silly questions, but my knowledge when it comes to this is like -1000, but I would like to start using more AS when it comes to KM instead of using Found Image actions whenever possible.
Thanks!
I hope I'm not stepping on any toes. I remember posting my handler that does exactly this on this forum a couple of years ago, but to save you (and me) from finding it, here's its most recent incarnation:
use framework "AppKit"
------------------------------------------------------------------------
return the infoForUIElementUnderMouse()
------------------------------------------------------------------------
### HANDLERS:
on mouseCoordinates()
tell the current application to tell mouseLocation()'s {x, y} ¬
of its NSEvent & NSHeight(its NSScreen's mainScreen's ¬
frame) to return the {(item 1), (item 3) - (item 2)}
end mouseCoordinates
on infoForUIElementUnderMouse()
return infoForUIElement at the mouseCoordinates()
end infoForUIElementUnderMouse
on infoForUIElement at __Ref as {list, record, reference}
local UIElement
tell application id "com.apple.SystemEvents"
if {__Ref}'s specifiers = {} then
tell __Ref to if its class ≠ list ¬
then set __Ref to its {x, y}
set __Ref to click at __Ref
end if
set UIElement to __Ref
if the UIElement = missing value then return {}
script Object
property parent : UIElement
property AXAttributes : a reference to (the ¬
attributes in me whose (name is not ¬
"AXURL") and (name is not "AXPath"))
property AXValues : value of AXAttributes
property AXList : the name of AXAttributes
property AXRecord : a reference to the the ¬
{«class usrf»:my AXList}'s contents
end script
set my text item delimiters to linefeed & linefeed
tell (a reference to the Object's AXList) ¬
to set the contents to paragraphs ¬
of (it as text) & ""
tell the Object to repeat with i from 1 ¬
to the length of its AXValues
set its AXList's item (i * 2) to ¬
item i of its AXValues
end repeat
return {UI element:the Object's contents} ¬
& the Object's properties ¬
& (the Object's AXRecord as any) ¬
& the {_AXActions:the name of ¬
every action in the Object}
end tell
end infoForUIElement
-------------------------------------------------------------------❮END❯
The handler you being called for your purposes is infoForUIElementUnderMouse()
. That handler first retrieves the mouse pointer's coordinates by calling the handler mouseCoordinates()
. This is passed to the handler that does the real work, infoForUIElement
, which enumerates and evaluates all properties
, attributes
, and actions
that belong to the UI element
identified as occupying the same coordinates directly under the mouse pointer.
A couple of points:
Should you ever try to call infoForUIElement
yourself, and wish to pass it a reference to a UI element
, this must resolve to a single UI element
class object belonging to System Events, and not a collection of objects.
There are two points in the script where it can occasionally hang, neither of which are the fault of the script per se, but rather caused by System Events stalling. This most likely occurs when the element underneath the mouse belongs to an application that hasn't exposed its object hierarchy very well (or at all): therefore, it can happen with Electron-based applications, which will often stall at the start of infoForUIElement
when trying to perform a click
; and I've noticed it happens with some objects in Monterey's System Preferences app despite passing a perfectly good direct reference to infoForUIElement
.
But, on the whole, it works very well for me in Monterey, and I use it regularly.
Note that @CJK's script fails in macOS 10.14.6 Mojave.
Although there is a version which does work on that OS, which I have and use now and then.
Any details on the nature of the failure ?
This line fails to compile:
set the UIElement is missing value then return {}
Presumably should be:
if the UIElement is missing value then return {}
Thanks.
Right you are. I've made the correction and a few other edits, mostly stylistic, with the exception of the class list for the at
parameter of infoForUIElement
where I've swapped out point
for list
.
Hello @ccstone I followed your advice. Is there any reference you could suggest to help me convert the output to an AppleScript
For example, simply to press the Notes Menu Item, how would i write the AppleScript based on the following
1- I imagine that you use a standard AppleScript wrapper
2- how do you add the info below into the wrapper ?
thanks very much
application Process "Bear"
window Bear
splitter group 1
scroll area 1
outline 1
row 1
UI element "Notes"
static text "Notes"
image 1
UI element 3
What notes menu item? The one in the menu bar?
I will make my question more generic.
You offered a solution for non expert users in order to avoid having to resort to UI browser.
With UI browser, the user records the screen → is given the AppleScript on a silver platter (the report to which one has to add the wrapper).
What I am asking is what the non expert should do with the output of your macro.
Let's say that I am in Bear (could be any app), and I want to write an AppleScript to click on Note in the Menu, how do I go from the output of your macro to an AppleScript ?
thank you Chris
Screenshots?
At a complete guess: The simplest would be that each indent is a tell
bock until the last, which is the action to take. So try this:
tell application "System Events"
tell application process "Bear"
tell window Bear
tell splitter group 1
tell scroll area 1
tell outline 1
tell row 1
tell UI element "Notes"
click UI element 3
end tell
end tell
end tell
end tell
end tell
end tell
end tell
end tell
But I'm no UI-scripting guru -- I'd take the above as the start point and try various tweaks until it works!