I'm not sure if this is even possible, which is why I'm asking. I would like KM to be able to read data from windows that aren't currently visible on the screen. Is that possible.
I think it depends on the app. For example, accessing menus probably requires the app to be frontmost. Some apps support AppleScript, but even then it probably depends on what the app's AppleScript supports. E.g. some apps have data dictionaries which are probably accessible even when the app is not visible.
Ideally I would want to be able to read the window's raw bitmap image (using KM) when the window isn't visible. I doubt that's possible, but I have to ask. But I'm worried that macOS itself doesn't know what's on the window if the window isn't at least partially visible on the screen, so this may not be possible. But I'm not sure.
I've heard there is some sort of optional add-on for macOS called XQuartz, which enhances the macOS GUI. Is this something that might help me? Does KM even work when XQuartz is installed?
Data is a pretty amorphous thing, but it sounds like you're talking about image data.
Apple's AppKit framework, after the tools in KM itself of course, is where I would look for native tools for messing with image data.
Here's a eureka moment worth looking at:
It shows how to convert the base64string data in an <"image"> key in an action dictionary to NSImage a native object in the Cocoa AppKit framework. Lots of other conversions are also possible with NSImage.
Stuff I am not particularly familiar with but could look up.
Of course, KM has a ton of tools for manipulating images.
As for getting info and data while an app is not front, absolutely. If the developer makes it available via AppleScript it's possible. Excel can do it all it the background.
Because the KM Editor's AppleScript makes the xml of an action available when the app is not front, accessing this data can happen when the app is not front.
The same goes for Engine's getmacros, gethotkeys plist data.
Preview is an app that doesn't require that it is front to get the path of a window's document's path.
tell application id "com.apple.Preview"
path of document 1 of window 1
end tell
Not raw in-memory data, but you get the drift.
As for XQuartz, after a brief web search, I'm still not seeing a relationship with image data.
If what you're after is PDF images, you might be thinking of Quartz, the native Apple framework to "Allow users to browse, edit, and save images, using slideshows and Core Image filters."
I know of it and that people have, I think, used AppleScriptObjC to manipulate Pdfs.
I have no idea if image extraction can be done with it.
You'd start by declaring this at the top of a script.
use framework "Quartz"
And that is as far as my Quartz scripting understanding goes.
I was open minded about any data, but indeed I was also thinking about image data. Thanks for your long, detailed response. In short, I'm inferring that any solution is app-specific but there's probably no way to get the full window pixel array of a hidden window.
