Can KM be used to help a person who had a stroke on the RHS of their brain?

Questions & Suggestions
1

A person who has had a stroke on the right hand side of their brain cannot "see towards the left" easily or at all. People who have had strokes still like to use their computer (or their iPad) to use FaceTime. But FaceTime has its menu on the left hand side of the screen. However a person who has had a stroke on the right hand side of their brain needs menus to be on the right hand side.

This may be a bit of a long shot, but can anyone think of a way to use KM on a Mac so that the left side of the FaceTime app (the part where the menus appear) can be visible on the right hand side of the screen?

I'm toying with the idea of making a Custom HTML form that duplicates the left side of the FaceTime app (if it has a menu/pane visible) and displaying that in a window to the right of the FaceTime app (maybe the right 20% of the screen?) That shouldn't be a particularly hard macro to write. I think it would work. But then more amazingly, if someone could then make it so that clicking in the Custom HTML form would also send a duplicate click to the pane in FaceTime, that might be even better. I'm willing to put in the effort, but does anyone have any guidance? Are there better ideas?

1 Like
2

A New Macro Group available only to Facetime and set to "Shows a palette until" so you automatically get the palette when Facetime is active would be simpler than custom HTML unless you need more bells and whisltes.

As an example, a macro in that group with this applescript action would start a new FaceTime:

activate application "FaceTime"
tell application "System Events"
	tell process "FaceTime"
		click button "New FaceTime" of window "FaceTime"
	end tell
end tell
1 Like
3

Okay, I appreciate that tip very much, but that wouldn't actually show what's in the left window pane.

4

I rarely (never) use the FaceTime app, as we tend to start our calls from Messages. Looking now, is the left pane just a list of recent activity? If so, I don't know how you're going to get at that list, short of OCR, but that will only work on a section of the screen at a time.

There's no AppleScript dictionary for FaceTime.

-rob.

5

Thanks for the tip about AppleScript. It's amazing how many Apple apps don't support AppleScript.

I have indeed considered OCR, but I also suggested above just making a screenshot (my exact words were, "duplicates the left side of the FaceTime app") and then letting the user click on that duplicate image on the right hand side of the screen. I think this should work.

So what I'm thinking of is taking screenshots of the left pane of the FaceTime window, putting it in a new window on the right side of the screen, and "duplicating the clicks." I.e., any click on the duplicate window would also be sent to the FaceTime window.

I also submitted a feature request to Apple today, (asking for the user to be able to move the pane to the right side of the app) but I think we all know the likelihood of that being successful. I'd probably be more likely to succeed if I took a video (putting it on Youtube) of two people trying to use FaceTime, one with a stroke on the left side of their brain, and the other with a stroke on the right side of their brain. Once people see the difference, maybe Apple will act.

6

Oh I see. Even without an applescript dictionary, you could probably do some spelunking into the UI elements to return the menu items. This shows me the contact name of the last call:

activate application "FaceTime"
tell application "System Events"
	tell process "FaceTime"
		return value of item 1 of static text of group 1 of list 1 of list 1 of scroll area 2 of window "FaceTime"
	end tell
end tell

And this shows me the call before that:

tell application "System Events"
	tell process "FaceTime"
		return value of item 1 of static text of group 2 of list 1 of list 1 of scroll area 2 of window "FaceTime"
	end tell
end tell

The history is grouped by timeframes and the number of calls per group is not static, so it would be tedious to build out checks for those constraints on the fly, but I think it's doable.

Clicking on the elements is another story. We could look into it further if it's something you want to pursue.

7

I'm actively working right now on making a screen snapshot copy of the left pane into a new window to the right of the app. I'm excited by the progress I'm making, but I'm still not sure if I will succeed.

1 Like
8

I don't see why this would be a problem.

Periodic (every second) screen capture of the button area, and display that image in your custom HTML window, and then clicks in the window get translated to clicks in the corresponding area of the buttons on the screen.

You wont get anything useful as far as feedback from clicking the button (though you could do something in the JavaScript to indicate the click happened).