AppleScript and Navigating Mail

[Joel]

I am currently having Keyboard Maestro navigate the Mail response window with tabbing as there are no Menu commands, shortcut keys, etc. that I can find (I have searched extensively with Claude, Perplexity, etc.) and am therefore wondering:

1. Is there a way to construct an Apple Script that will move the cursor from "top portion of the window" (i.e., where the To, From, etc.) are located to the body of the window (i.e., ideally top left corner where one would write a response)?

2. Is there a way to construct an Apple Script that will move the focus from the search box to the found word in the body of the response window?

I have tried with AI but cannot get the result that I am looking for as the scripts either:

1. Use GUI scripting and simulate repeated tabbing (which I already have via Keyboard Maestro).

2. Use ⌘J or other keystrokes which are supposed to work but don't (and I have tested on different MacBooks)?

Much thanks.

Joel

[griffman]

1. Given the body will always be a fixed distance from the header, it seems the easiest way to get the cursor there is with a Move and Click Mouse action—just set the proper offset from the top left corner of the window, and you should activate it every time, with the cursor there ready to type.

2. I've looked for this in the past, but never found a way. The search box makes things behave weirdly when you have an active search. Command-G will highlight each match in turn, but I've never found a way to move the cursor there. The cursor is a tricky thing anyway, as it's not the mouse's location but another, but there are no APIs that share that location that I've ever heard of.

-rob.

[Joel]

@griffman

1. Given the body will always be a fixed distance from the header, it seems the easiest way to get the cursor there is with a Move and Click Mouse action—just set the proper offset from the top left corner of the window, and you should activate it every time, with the cursor there ready to type.

Appreciated.

I never noticed that the distance is always the same even when the window size changes. I will give it a try! One down!

@griffman

2. I've looked for this in the past, but never found a way. The search box makes things behave weirdly when you have an active search. Command-G will highlight each match in turn, but I've never found a way to move the cursor there. The cursor is a tricky thing anyway, as it's not the mouse's location but another, but there are no APIs that share that location that I've ever heard of.

-rob.

Arrrggg! At least I am not alone in my quest. This is a PITA!!

Thank you!

[Nige_S]

@Joel

Is there a way to construct an Apple Script that will move the cursor from "top portion of the window"...

Why not change your order of operations? The insertion point is always at the point you want immediately after you "Reply" to an email, so do that stuff first and whatever with reply's fields later.

Otherwise...

With "native" KM Actions, "Click at" just inside the bottom-left corner of the email window then "Keystroke: ⌘↑" to force the insertion point to the start of the reply's text.

Or run this -- tested on Sequoia but you'll probably have to change the element hierarchy for Tahoe (and with every OS update after that )

tell application "System Events"
	tell process "Mail"
		set focused of UI element 1 of scroll area 1 of group 1 of group 1 of window 1 to true
	end tell
end tell

It does assume your reply is the frontmost Mail window...

Is there a way to construct an Apple Script that will move the focus from the search box to the found word in the body of the response window?

I was going to reply with a definite "No" -- but Mail seems a bit weird... I don't know if this will work for you, but give it a go.

It appears that when you have a successful "Find" in Mail a new "overlay" window is created with bounds slightly larger than the found selection. You can get those bounds, find the middle, double-click there:

Mail Select Found.kmmacros (3.3 KB)

That should work for single-word searches. Multi-words you can't select with double-clicks, while matches that spread across the end of one line and the start of the next will have unusable bounds.

But it does suggest another method. Finding the selected text by image detection is difficult -- even restricting it to the front window can lead to false positives when just looking for a couple of pixels of your highlight colour. But now we have a way to restrict the search area -- much better!

Mail Select Found by Highlight.kmmacros (9.9 KB)

You may have to play around with the colour and the down-offset, depending on your settings. This even works for multi-line selections, playing on the fact that (I think!) image detection "scans" left-to-right, so the first match will be the left-most of our explicitly-set "Topmost".

This is more fragile (image detection!) but has the advantage that the selection is maintained so you can Cut, Copy, move to before or after with arrow keys, and so on.

Once more -- choices, choices!

[Joel]

@Nige_S

@Joel

Is there a way to construct an Apple Script that will move the cursor from "top portion of the window"...

Why not change your order of operations? The insertion point is always at the point you want immediately after you "Reply" to an email, so do that stuff first and whatever with reply's fields later.

I considered that at the outset but I need to end up there at the end so this is not possible.

@Nige_S

Otherwise...

With "native" KM Actions, "Click at" just inside the bottom-left corner of the email window then "Keystroke: ⌘↑" to force the insertion point to the start of the reply's text.

Or run this -- tested on Sequoia but you'll probably have to change the element hierarchy for Tahoe (and with every OS update after that )

tell application "System Events"
	tell process "Mail"
		set focused of UI element 1 of scroll area 1 of group 1 of group 1 of window 1 to true
	end tell
end tell

It does assume your reply is the frontmost Mail window...

Appreciated.

Love the AppleScript route but not crazy about "upgrade fragility".

Will likely go with the Mouse Click idea suggested by you and Rob.

Will test over the next few days and report back!

@Nige_S

@Joel

Is there a way to construct an Apple Script that will move the focus from the search box to the found word in the body of the response window?

I was going to reply with a definite "No" -- but Mail seems a bit weird... I don't know if this will work for you, but give it a go.

It appears that when you have a successful "Find" in Mail a new "overlay" window is created with bounds slightly larger than the found selection. You can get those bounds, find the middle, double-click there:

Mail Select Found.kmmacros (3.3 KB)

That should work for single-word searches. Multi-words you can't select with double-clicks, while matches that spread across the end of one line and the start of the next will have unusable bounds.

But it does suggest another method. Finding the selected text by image detection is difficult -- even restricting it to the front window can lead to false positives when just looking for a couple of pixels of your highlight colour. But now we have a way to restrict the search area -- much better!

Mail Select Found by Highlight.kmmacros (9.9 KB)

You may have to play around with the colour and the down-offset, depending on your settings. This even works for multi-line selections, playing on the fact that (I think!) image detection "scans" left-to-right, so the first match will be the left-most of our explicitly-set "Topmost".

This is more fragile (image detection!) but has the advantage that the selection is maintained so you can Cut, Copy, move to before or after with arrow keys, and so on.

Once more -- choices, choices

I will similarly test the first approach to see how well it works are also report back.

Huge thanks for the great solutions!

[peternlewis]

@griffman

• Given the body will always be a fixed distance from the header, it seems the easiest way to get the cursor there is with a Move and Click Mouse action—just set the proper offset from the top left corner of the window, and you should activate it every time, with the cursor there ready to type.

Sadly this is not entirely true.

The position is dependent on the enabling of the various View options:

And if you are like me, you occasionally need to change those view options (eg to do a Bcc - and why do I have the Reply-To field enabled??). And then you forget to change them back and the cursor ends up in the wrong place.

Just something to be aware of. Wouldn't it be nice if there was a good AppleScript dictionary to allow focusing on the various fields. Or even just a more generally good AppleScript support for controlling the text insertion point (or finding its location on the screen!).

[griffman]

@peternlewis

And if you are like me, you occasionally need to change those view options (eg to do a Bcc - and why do I have the Reply-To field enabled??).

That is really interesting: When I set up Mail many decades ago, or whenever I get a new Mac, I enable all those fields and just leave them on. I literally have never changed those settings once enabled.

I figure I might need them at some point, and it doesn't take up that much space to have them there, and I'd be more irked if I had to show them when I needed them…so I just let them be, even though I've used Priority Field maybe once or twice since it came into existence :).

But yes, a very valid observation that the body can move based on what's in the header. I just never considered that people might actively change what's there :).

-rob.

[Joel]

Appreciate the heads-ups noting "that sucks".

Completely agree that some consistent and reliable of navigation email -- which arguably is one of the most used apps -- would be welcome.

While the endless tabbing works reliably (even for the Peter's various view options), it is slow!

[peternlewis]

@Joel

While the endless tabbing works reliably (even for the Peter's various view options), it is slow!

Agreed. In one of my macros that I use frequently, there are 1 second pauses between each tab just to get Mail to move fields reliably.

[Joel]

@peternlewis

Agreed. In one of my macros that I use frequently, there are 1 second pauses between each tab just to get Mail to move fields reliably.

I managed to avoid pause of that length BUT because I am looking for the search result in the body I am continually tabbing → ⌘C → compare clipboard to image → repeat when not found.

I very much @Nige_S' AppleScript works because otherwise I am at a dead end!

[NaOH]

@peternlewis

And if you are like me, you occasionally need to change those view options (eg to do a Bcc - and why do I have the Reply-To field enabled??). And then you forget to change them back and the cursor ends up in the wrong place.

One way to address this risk is to have the cursor placed anywhere that will be in the message body and follow that by simulating Command-Up Arrow since that moves the cursor to the top of the message.

[Nige_S]

@Joel

Love the AppleScript route but not crazy about "upgrade fragility".

Almost all your automations can be borked by upgrades -- a change in field tab order, a renamed menu item, different colours or size for an UI element so it fails image detection, AppleScript dictionaries getting butchered (damn you, New Outlook!)... The list goes on and on.

The problem with System Events scripts is that breaking changes are less obvious, more difficult to fix, and more than likely "accidental" -- I'm betting that recent additional nesting of elements in Apple apps are because of the increased adoption of Swift UI and Xcode's generated user interfaces rather than a conscious developer decision, for example.

So more a case of "System Events is fragile, but not that much more fragile than other methods". And it's often more robust during execution than other methods when you're between of upgrades.

Don't be put of using System Events -- I've probably overstated the issue because of experiencing recent changes in Tahoe from Sequoia.

@Joel

Completely agree that some consistent and reliable of navigation email -- which arguably is one of the most used apps -- would be welcome.

Whilst maybe one of the most used Apple apps, I suspect it is also the one least automated in the way you want to do. For example, and as I said earlier, when you create a Reply the insertion point is already at the top of the content. Why do you need to change focus to the fields above then back again? Do what you need with those fields when you create the Reply.

This quickly knocked-up demo will create a Reply of the selected email to the original sender, CCing everyone in the original message's "To" field, adding a BCC, and still leave the insertion point in the body of the Reply so it's ready for your typing:

tell application "Mail"
	set theMsg to item 1 of (get selection)
	set outMsg to reply theMsg
	repeat with eachRecipient in to recipients of theMsg
		make new recipient at end of cc recipients of outMsg with properties {name:name of eachRecipient, address:address of eachRecipient}
	end repeat
	make new recipient at end of bcc recipients of outMsg with properties {address:"covermyass@example.com"}
end tell

[Joel]

@Nige_S

Appreciate teh clarification and understood.

I have a lot on the go at work thesis days but will find time to try ti implement solutions over the next +/- two days. I will let you know how I get on.

Much thanks!

[Joel]

@Nige_S

I got up early this morning to start implementing / testing your suggestions recognizing that my knowledge of AppleScript is limited.

I starting with the AppleScript that moves the cursor to the top left of the body / response area.

In working with Claude Code the results were:

1. I could not get AppleScript to physically move the cursor with the reason being " set focused and click at both failed in Tahoe is that Mail's body is a WebKit area that ignores synthetic accessibility events — only KM's native mouse click (which posts at a lower hardware level) actually moves focus into it."

2. I did get a combination of AppleScript + Keyboard Maestro Actions to physically move the cursor as follows:

Move To Response.kmmacros (24.8 KB)

Wondering whether you have any ideas or thoughts on how to get the AppleScript to physically move the cursor given Claude's response.

PS. Will move on to testing the Found AppleScript next chance I get.

Thank you!

[Nige_S]

@Joel

I starting with the AppleScript that moves the cursor to the top left of the body / response area.

In working with Claude Code the results were:

And what was wrong with the code I've already given you? Better to troubleshoot that, which makes the UI element active programmatically, than to start again getting coordinates and then clicking there.

...Mail's body is a WebKit area that ignores synthetic accessibility events...

Then I must have a very special copy of Tahoe, because the script works perfectly here.

But remember that any UI scripting is dependent on... the UI. What works for me in what I consider to be "standard" -- outgoing message at the front of Mail (Mail doesn't need to be the frontmost app) -- won't work for you if you are set up differently. You'll need to learn to dig around, which can be as simple as getting the UI elements of the front window:

tell application "System Events"
	tell process "Mail"
		return UI elements of window 1
	end tell
end tell

--> {static text "To:" of window "Re: Nigel... etc}

...then picking likely elements from that list and doing similar, eg for group 1:

tell application "System Events"
	tell process "Mail"
		return UI elements of group 1 of window 1
	end tell
end tell

--> {group 1 of group 1 of window "Re: Nigel...}

...and so on.

There are utilities to help, even macros that have been posted on this Forum, I just find it easier to dig in manually.

None of which answers the question of why you want to activate the "body" UI element. This appears to be a problem entirely of your own making, since the insertion point is at the right place when the reply first opens so you must be moving it elsewhere for $reasons. Sometimes the simplest solution to a problem is to not cause it in the first place

If this is for a new outgoing message the answer is just as simple. Even if you set fields and even content for the new message it will open with the "To" field selected -- a single Shift-Tab should put the insertion point at the top of the content area.

[Joel]

@Nige_S

@Joel

I starting with the AppleScript that moves the cursor to the top left of the body / response area.

In working with Claude Code the results were:

And what was wrong with the code I've already given you? Better to troubleshoot that, which makes the UI element active programmatically, than to start again getting coordinates and then clicking there.

...Mail's body is a WebKit area that ignores synthetic accessibility events...

Then I must have a very special copy of Tahoe, because the script works perfectly here.

As a start, the code that I tried was:

tell application "System Events"
	tell process "Mail"
		set focused of UI element 1 of scroll area 1 of group 1 of group 1 of window 1 to true
	end tell
end tell

When I executed the above, it did nothing (i.e., the cursor did not move).

@Nige_S

But remember that any UI scripting is dependent on... the UI. What works for me in what I consider to be "standard" -- outgoing message at the front of Mail (Mail doesn't need to be the frontmost app) -- won't work for you if you are set up differently. You'll need to learn to dig around, which can be as simple as getting the UI elements of the front window:

tell application "System Events"
	tell process "Mail"
		return UI elements of window 1
	end tell
end tell

--> {static text "To:" of window "Re: Nigel... etc}

...then picking likely elements from that list and doing similar, eg for group 1:

tell application "System Events"
	tell process "Mail"
		return UI elements of group 1 of window 1
	end tell
end tell

--> {group 1 of group 1 of window "Re: Nigel...}

...and so on.

There are utilities to help, even macros that have been posted on this Forum, I just find it easier to dig in manually.

Appreciated and noted.

I will dig around and see what I can find and figure out.

@Nige_S

None of which answers the question of why you want to activate the "body" UI element. This appears to be a problem entirely of your own making, since the insertion point is at the right place when the reply first opens so you must be moving it elsewhere for $reasons. Sometimes the simplest solution to a problem is to not cause it in the first place

If this is for a new outgoing message the answer is just as simple. Even if you set fields and even content for the new message it will open with the "To" field selected -- a single Shift-Tab should put the insertion point at the top of the content area.

Agree for Reply and Reply All but not for Forward where the number of tabs is dependent on the "fields" (i.e., Reply-To, etc.) as was noted by @peternlewis.

I agree with your comment "Sometimes the simplest solution to a problem is to not cause it in the first place " and have taken this away from teh first time you pointed this out. I will examine what can be done to further simplify the macro / workflow.

[Nige_S]

@Joel

When I executed the above, it did nothing (i.e., the cursor did not move).

But no error either?

What is the state of the outgoing mail window when you run it, and how (if you do) do you switch to Mail to see the results of the script?

It still works fine for me, even after updating to 26.3.1

As an added bonus (if you can get it to work!) -- when you are in "find" mode with some text found and selected, running this script will jump you to the beginning of the found selection. So it could solve that problem, too.

[Joel]

@Nige_S

I am still digging in, a few follow ups:

Move cursor to top left of body / responding:

It works when the cursor is the to top portion of the e-mail (i.e., To, From, Subject, etc.) but not when it is located elsewhere in the body. Is this teh correct / intense behaviour of the Apple Script or should it work regardless of where teh cursor is located.

Move cursor to found item:

I can get the version that searches for an image to work.

I can not get the version that click mouse on the results to work.

Just to be clear (and at the risk of asking a dumb question), the steps in running and testing the macros are:

• Open the Reply window
• Open the search "window" (⌘F)
• Search for the word
• Press escape to exit search window
• Run the macro

The above is what I am doing to the "image search" version working but it failed for the "word search" version.

Thanks.

[Nige_S]

@Joel

It works when the cursor is the to top portion of the e-mail (i.e., To, From, Subject, etc.) but not when it is located elsewhere in the body.

Well, no. If the insertion point (not the "cursor"!) is in the email's content then the email's content element is already active. The insertion point going to the beginning of the text is a happy accident of activating the previously-inactive element.

If you want to jump to the beginning of the content when already in the content, the usual way is to send ⌘↑. If you want to have a single macro that puts the insertion point at the start of the content, whether the insertion point is in a "header" field or already in the content then run the AppleScript and send the keystroke.

Or use the AS to activate a non-content element first, then the content element:

tell application "System Events"
	tell process "Mail"
		set focused of text field 1 of window 1 to true
		set focused of UI element 1 of scroll area 1 of group 1 of group 1 of window 1 to true
	end tell
end tell

Just to be clear (and at the risk of asking a dumb question), the steps in running and testing the macros are:

• Open the Reply window
• Open the search "window" (⌘F)
• Search for the word
• Press escape to exit search window
• Run the macro

Not step 4 -- run the macro with "Find" still active.

[Joel]

@Nige_S

@Joel

When I executed the above, it did nothing (i.e., the cursor did not move).

But no error either?

What is the state of the outgoing mail window when you run it, and how (if you do) do you switch to Mail to see the results of the script?

It still works fine for me, even after updating to 26.3.1

All good, I got it working when in the "Header Section".

Apologies for highlighting my lack of AppleScript knowledge / understanding and thinking it should work even when in the "Body/Response Section".

@Nige_S

As an added bonus (if you can get it to work!) -- when you are in "find" mode with some text found and selected, running this script will jump you to the beginning of the found selection. So it could solve that problem, too.

In terms of the added bonus, it is even better than that, it works on multi-word searches! In my testing it selects the entire phrase, be it single word or multi-word!

What a huge bonus and improvement for me!

With much thanks!!