Recommendations for image matching with a dialog box?

Questions & Suggestions
#1

Hi all,

I'm trying to automate checkbox selection/deselection in this dialog box:

image

It comes from the Draw.io app, which I think is basically a webpage hosted inside an app Window. i.e. The dialog isn't a real window.

I can tab across and press space to toggle the checkbox, but there's no key that I know of to only "turn on" or "turn off" checkbox. So currently what I'm doing is matching on an image that includes the empty checkbox, then clicking on the checkbox. I'm using the "Move and Click from Found Image" action.

The problem I'm having is that if I don't use a little fuzziness, then the image won't match. I guess this is due to the anti-aliasing of the text. But if I use a little fuzziness, then the image incorrectly matches if the checkbox is already selected. If the checkbox is already selected, then I don't want it to match, so that it will be left selected instead of being toggled off.

The only solution I can think of is to use a if-then where I match fuzzy on the text, then use a non-fuzzy match for the region to the left of the text. that's pretty cumbersome. Are there any better options?

#2

Well, let's find out how much we can learn about this window.
Try running this script, and see what window elements it finds:

Front Window Analysis Tool for System Events

and post the results here in a Code Block.

#3

My approach is different from JM's. JM uses in-depth knowledge of HTML to properly solve these things with AppleScript. My approach is to change what you are searching for. In this case, search for a blue box labelled export (and INCLUDE some of the white on all four sides) then click on an offset of -X, -Y from the upper corner of that box. That should work peachily.

#4

Front Window Analysis Tool output:

application Process "draw.io"
	window 
		button 1
		button 2
		button 3

Front Application – Window Properties Report:

-------------------------------------------------------------------------------------------

draw.io -- Window Properties Report -- 01 Windows Listed

-------------------------------------------------------------------------------------------
Window 01                  :  missing value
-------------------------------------------------------------------------------------------

title                      :
size                       :  2481, 1490
position                   :  40, 62
subrole                    :  AXStandardWindow
role                       :  AXWindow
focused                    :  false
role description           :  standard window
description                :  standard window,
class                      :  window
accessibility description  :  missing value
enabled                    :  missing value
help                       :  missing value
maximum value              :  missing value
minimum value              :  missing value
orientation                :  missing value
selected                   :  missing value
value                      :  missing value

I tried replacing the images with ones that include as much surrounding white space as possible. It works okay when I test it in the editor using the "Go" button, but when I run it, the macro is de-selecting one of the checkboxes that it should leave selected.

#5

I've implemented the "annoying" way that I mentioned, and it seems to work well.