How to List All Windows of One App That Are Open in All Desktops

Hi Chris,

It appears to be a bit fixed in the current version of Typora, v1.1.5, out of Beta where it had been for years, and now costing $15 instead of free. (There is a free trial mode with a reminder/nag window that counts down the trial period.)

When I open Script Editor and look at the Typora Dictionary, all the items you listed above as "fails" are now defined in the dictionary doc. Does that mean they work now?

However, "selection" is not defined. Should it be? Or is it accessed another way? I don't find "selection" in the TextEdit dictionary either, so I think I must be missing a concept of operation here. When I google "applescript get selection" I get some advice on how to save the clipboard, send the ⌘+C keystroke, save that in a variable, and restore the clipboard. So it seems get selection isn't well supported in other scriptable apps.

If I create an AppleScript script something like this:

tell application "Typora"
    its visible
end tell

Is running that in the Script Editor enough? Do I need to make a KM macro to run it while Typora is frontmost? Putting it into the script editor with a delay 3 ahead of it lets me click the Run button and then make Typora frontmost.

When i use visible there I get an error and when I use frontmost I get true in the Results pane of the Script Editor. So I'm still not sure how to tell if the AppleScript support got fixed in the Typora 1.1 version. Can you tell?

Thanks for your help getting me this far.

Hey @ccstone / Chris,

I kind of got sidetracked into trying to make Typora work the way I would like it to, but that's not necessarily the direct path to my OP problem.

Were you ever able to get @CJK's function to work?

I'm realizing that one of the purposes I had for the original question (but not all) was to be able to tell one desktop from another, to tell when a desktop had been moved, etc. and I think I can do that using some one specific app that is well behaved that way. Maybe TextEdit, maybe something else. The idea here is to emulate what CurrentKey does invisibly by having some app that has one window on each and every desktop and using that to force window changes.

Maybe I build a small app, maybe I dedicate some app that I don't really use for anything else, so that leaves out TextEdit. Still sorting this one out.

About Mojave vs Catalina

I'm getting set to give up Mojave for Catalina. I don't use my 32-bit apps much anymore and there are free or affordable alternatives to the ones I do. Mostly the threshold for me is that sharing Notes between my iPhone and my Mac, on Mojave I can share only individual files with another person while on the iPhone I can share whole folders with someone else, and then they disappear from the Mac because Mojave doesn't grok shared folders. Moving to Catalina should fix this as well as give me updates to some other software that no longer supports Mojave.

I spent a day generating a list of all the 32-bit apps I had (there were 145 of them) and going through to see what I would miss if I deleted them (mostly Office 11 and a.bunch of Adobe stuff that I don't use or the licence has expired). The big one has been Acrobat Pro X, but some of the features I have been counting on are now in Acrobat Reader and others are available as free web utilites. Worst case is I subscribe for a month if there's something I HAVE TO have.

Sorry for the long Mojave/Catalina digression.

Did you test it for yourself ? I don't have Mojave, so I'm afraid I cannot, but I'm happy to report that it still works in Monterey. However, using the info that @ccstone provided about the possible error-prone statements, I've refactored the previous function to remove references to those elements:

Application.prototype.getWindowList = function() {
        ObjC.import('CoreGraphics');
        let pids = Application('com.apple.systemevents').processes
                   .whose({'bundleIdentifier':this.id()}).unixId();
        return  ObjC.deepUnwrap( ObjC.castRefToObject(
                $.CGWindowListCopyWindowInfo(16, 0) )).filter(e =>
                pids.includes(e.kCGWindowOwnerPID) &&
                              e.kCGWindowLayer==0  &&
                              e.kCGWindowAlpha==1  &&
                              e.kCGWindowStoreType==1).flatMap(x => 
                              [{ id     : x.kCGWindowNumber,
                                 name   : x.kCGWindowName,
                                 bounds : [ 'X','Y',
                                            'Width',
                                            'Height' ].map(z => 
                                          x.kCGWindowBounds[z]) }]);
}

As before, to use the function:

Application('<app name_or_id>').getWindowList()
2 Likes

Thanks @CJK, I appreciate your effort and I look forward to trying it. Unfortunately that could be a month away. Only being able to work on this every few months is frustrating. By the time I get back up to speed on the problem, my time window is nearly gone.

Could I ask a favor IFF you have time and interest? Would you be willing to wrap that function in a KM call to AppleScript (or whatever it takes) to demo it? Please don’t if it wouldn’t be fun.

I’m asking because I can get further faster if I can start with a working demo than if I first have to make something that works at all and then apply it to my problem.

Thanks!

On Big Sur at least, @CJK's very helpful (application prototype extension) code lets us generate a JSON listing of windows for named applications in this kind of Keyboard Maestro pattern:

Windows listed for named applications.kmmacros (4.0 KB)


Expand disclosure triangle to view JS Source
(() => {
    "use strict";

    // Rough sketch by Rob Trew,
    // using CJK's Application prototype extension code.

    // Ver 0.2 (fixed a Width ⇄ Height swap)

    const main = () => {
        const
            appNames = Application("Keyboard Maestro Engine")
            .getvariable("appNames");

        return (
            extendPrototype(),
            JSON.stringify(
                lines(appNames).map(
                    k => ({
                        appName: k,
                        windowList: Application(
                            k.trim()
                        ).getWindowList()
                    })
                ), null, 2
            )
        );
    };

    // ----------------------- JXA -----------------------

    // Using code by CJK

    const extendPrototype = () =>
        Application.prototype.getWindowList = function() {
            ObjC.import("CoreGraphics");

            const
                pids = Application("com.apple.systemevents")
                .processes
                .whose({"bundleIdentifier": this.id()})
                .unixId();

            return ObjC.deepUnwrap(
                ObjC.castRefToObject(
                    $.CGWindowListCopyWindowInfo(16, 0)
                )
            )
            .filter(
                e => pids.includes(e.kCGWindowOwnerPID) &&
                e.kCGWindowLayer === 0 &&
                e.kCGWindowAlpha === 1 &&
                e.kCGWindowStoreType === 1

            )
            .map(x => ({id: x.kCGWindowNumber,
                name: x.kCGWindowName,
                bounds: [
                    "X", "Y",
                    "Width",
                    "Height"
                ].map(z => x.kCGWindowBounds[z])})
            );
        };


    // --------------------- GENERIC ---------------------

    // lines :: String -> [String]
    const lines = s =>
    // A list of strings derived from a single string
    // which is delimited by \n or by \r\n or \r.
        Boolean(s.length) ? (
            s.split(/\r\n|\n|\r/u)
        ) : [];


    // MAIN ---
    return main();
})();


2 Likes

Thanks CP/Rob (what do you prefer to be called, @ComplexPoint ?),

This looks like a great start, not only for my project but as a trailhead into JXA. I’ve started learning AppleScript to use with KBM and I have some familiarity with shell scripts, I’ve used SED with KBM to massage text lists, but I’ve had no inkling that JXA was available within KBM. Thanks for your great Wiki articles on the subject. I’m in a similar position to where @JMichaelTX was:

I used Acrobat JavaScript 20 years ago to manipulate comments in PDF files, but I have no experience using it in browsers.

Again, thanks, I look forward to trying this out.

Nice! I like your formatting (indentation) much more than mine.

I've just noticed that, for whatever reason, I've weirdly ordered the ordinates of the bounds property as X, Y, Height, Width] instead of `[X, Y, Width, Height. I'll make the correction to my post.

1 Like

Thanks ! Switched above too, now (in the macro and the listing).

formatting

( It's just the ESLint and Prettier ESLint extensions in VSCode )

I got a chance to download an install "Windows listed for named applications.kmmacros" and what I get when I run it is:
image

I'm running on Mojave (on the verge of upgrading to Catalina). Would that make a difference?

How do I capture the rest of that error message? In a shell, I could redirect STDERR to a file. Should I have the shell execute the JavaScript to capture that, or is there a KBM-native way?

Ah HA! I found the logs folder /Users/amohr/Library/Logs/Keyboard Maestro/ and the Engine.log file.

Here's the result of running the macro:

2022-08-07 18:00:44 Execute macro “Windows listed for named applications” from trigger Editor
2022-08-07 18:00:45 Action 11176861 failed: Execute a JavaScript For Automation failed with script error: text-script:1137:1174: execution error: Error on line 44: Error: Ref has no type (-2700)
2022-08-07 18:00:45 Execute a JavaScript For Automation failed with script error: text-script:1137:1174: execution error: Error on line 44: Error: Ref has no type (-2700). Macro “Windows listed for named applications” cancelled (while executing Execute JavaScript For Automation).

It identifies line 44.

Here's lines 43-47 of the function text:

            return ObjC.deepUnwrap(
                ObjC.castRefToObject(
                    $.CGWindowListCopyWindowInfo(16, 0)
                )
            )

I'm not sure what "Ref has no type (-2700)" means, but I'll try running the script without the "strict" setting.

No difference. It's probably not just a JavaScript error then, I guess it's coming from ObjC. Stack Overflow has nothing for that error text.

Any suggestions?

It looks like it's taking issue with whatever gets returned by the function $.CGWindowListCopyWindowInfo(16, 0). Thus I'd want to know what value it's returning.

What gets returned by the following code:

ObjC.import('CoreGraphics');
let wInfo = $.CGWindowListCopyWindowInfo(16, 0);
wInfo;

// ObjC.castRefToObject(wInfo);

Subsequently, if you remove the two slashes at the beginning of the last line (this uncomments the line, which at present doesn't get executed), what does the code return now ?

[ I'd suggest running this in Script Editor: you can create a blank document and set the language to JavaScript at the top of the document (there's a dropdown menu). Also, from the bottom of the document, in the status bar, there are three icons on the left. Click the third one to reveal the event log. The reason to use Script Editor here is that we're dealing with rather opaque values that KM won't be able to display natively. The ObjC.castRefToObject function is integral to converting an opaque value into one that, firstly, JXA-ObjC can understand, before this gets unwrapped into something JS can understand, before it's manipulated to produce an array of dictionaries that KM should be able to understand. ]

1 Like

As a first try, I opened Script Editor, opened a new File window, set it to JavaScript, and pasted in the lines above. I opened the Events log and ran it (with the Play button). That produced 113038 characters of output in a single "line". The line starts out with "$([" and the matching "])" is at the end, 113035 characters later.

So I tried uncommenting that fourth (fifth) line above, When I did I got:
image

And in the Events pane I got:

Error -2700: Script error.

That looks familiar.

So, any idea what's going on? Why did you ask for that line? It's not in the original function, is it?

Back at the test script with the line commented out, all 113038 characters of it --
Within that overall ([...]), it looks like there are repeated blocks of

$({"kCGWindowLayer": ... }),

So for readability, I'll add linebreaks after all those commas, using a VIM global.

:g/}),$({"kCGWindowLayer":/s//}),^M$({"kCGWindowLayer":/g

That results in 337 lines, one per kCGWindowLayer.

Any idea what I should be looking for in there?

Aside:
97 of those lines include

"kCGWindowOwnerName":$("Logi Options Daemon")

That's for my bluetooth mouse, which isn't even on right now. There is only one entry for that in the Activity Monitor (All Processes), so why so many entries here?

Thanks!

I’m curious where you got those parameters, (16, 0).

The Apple doc for CGWindowListCopyWindowInfo describes them as:

static var optionAll: CGWindowListOption

var kCGNullWindowID: CGWindowID { get }

Where did you get numeric values?

A brief footnote or marginal scribble:

If automation is a significant part of what you need from macOS, then it may be prudent to lag a little (or even significantly) behind the newest versions.

osascript support has a very low priority at Apple these days, and every new version of macOS inevitably degrades or breaks it just a little, even if only inadvertently.
Sometimes fixes are made retrospectively, but typically with a lag of several months, rather than a few weeks.

Better to install a macOS version at the end of its life-cycle (when it's as fully cooked as it's ever going to get) than at the rough and creaking start of its life-cycle.

In short – macOS upgrades do have a cost (especially, now, for automation) and it's always worth pausing to seriously consider whether it's really worth paying that cost, and how exactly your work might benefit from it.

That's kind of the position that I'm in. I'm currently running Mojave, which Apple no longer supports, planning to upgrade soon to Catalina, a single-step upgrade. I'm looking for all the things that I can anticipate Catalina breaking -- I've removed all my 32-bit software. I'll miss being able to OCR directly in a PDF using Acrobat Pro X, but there are workarounds.

1 Like

IMO that's a little too at the end of it's life-cycle!

The OS should be well-baked by the time the next full version is publicly released. While I don't (can't!) stop our users jumping to the latest'n'greatest as soon as they feel like it, we generally recommend "production" machines hold back for a year, so they'd have upgraded to Catalina when Big Sur came out, Big Sur when Monterey dropped etc. (Assuming the hardware/software was compatible with the new OS, obvs.)

That keeps things reasonably up to date with a manageable risk of things going pear-shaped. More importantly for us it minimises the number of machines that aren't receiving security updates -- Catalina will probably go end-of-life a couple of months after you've installed it.

1 Like

I agree with your advice, but we have an odd situation here in that the problematic JXA/JSObjC is arising in earlier versions of macOS (Mojave), whereas it works seamlessly in Monterey. It's unexpected, to say the least.

1 Like

Thank you for the description of the output. You actually told me exactly what I needed to know here. That output is perfect, so we know that there's no problem with the actual function that retrieves the window data, $.CGWindowListCopyWindowInfo().

So the problem is arising when we try to take the opaque core foundation C-data types and recast it into more usable Objective-C foundation types.

I'm going to have to experiment tonight to see whether i can bind the opaque data type to something else that can act as an intermediary to bridge the values between C and JS. I just need to find out exactly how the $.CGWindowListCopyWindowInfo() function is declared.

I have to admit, something feels a bit off but i cant put my finger on it, but my attention is also divided right now, and im not at a computer.

Ha, this is because of how we conventionally use the term "window" to refer to a very specific type of element that we interact with, namely a framed, titled, content viewer that acts as a moveable container for other types of elements (documents, file icons, or whatever). But this is semantics, and in the language of Core Graphics and a session manager, a window is...well, actually, everything is a window: buttons, icons, labels with text, ... These are fundamentally all the same thing, with certain parts visible or invisible, and certain functionality added in or taken out. But they're all windows, including a lot that are completely invisible but are stilk very much present. I suspect your bluetooth device windows make up some of the invisible contingent, as well as the ones that will form part of the bluetooth menu bar menu (which is also a window).

If you examine my first script, those values were represented by constants that I was accessing the same way as I access the function. The 0 value is the value returned by $.kCGNullWindowID, representing a null value. 16 is the value of the constant $.kCGWindowListExcludeDesktopElements, which excludes desktop icons, the desktop itself, and some other "windows" that can't be seen. If you opt to include these, the list is absurdly massive, if you consider a single desktop icon is, itself, comprised of two or three windows—one for the icon, one for the label, and i think maybe one two group those two elements together.

So bear with me, and I'll get back to you with my success or failure.

I found the line in that script. Thanks.

With that line dumping all the window info, my own approach from here would be to massage that data dump with other tools:

  1. Break the data into separate lines, as I did above, probably doing it in a pipeline rather than with VIM.
  2. Filter for the lines that identify the given parent app, e.g. "kCGWindowOwnerName":$("TextEdit")
  3. Extract the name of the window from the lines that have one, e.g. "kCGWindowName":$("Xera Fluids Ordering, etc.rtfd")
  4. Format that for output.

I may be missing something, but that could be a minimal answer to my question in the OP. When I originally asked that question, it was just the the first part of my quest. I additionally eventually want to know which Desktop Workspace is each of those windows in.

I don't find anything about Desktops / Workspaces represented in the output of CGWindowListCopyWindowInfo. Is that tracked totally separately by Mission Control or is it in data that would be returned by a different first parameter (not 16, $.kCGWindowListExcludeDesktopElements)? Seems to me it might be a part of the "Desktop Elements" data.

Any ideas where to find that out?

Thanks!

You're welcome to use whatever method you wish, but there are good reasons not to leap straight into using other tools as a workaround for something. For one, other forum users will come by this post one day and it'll be benefit most people not only to see how the problems we encountered were solved, but also that it was solved and allows them to utilise the solution without third-party tools that they might not have nor might not wish to install. Secondly, debugging a solution that uses a number of different technologies is harder. Lastly, having a solution that is self-contained from beginning-to-end will usually (but not always) be the most performant.

My original contribution when I came across this post was simply to highlight a C API that enumerates windows regardless of which desktop (space) they are on. I later expanded this into a JXA function as the API call returns an excess of information (most of which is useless to most people), and of a data type a lot of people wouldn't know what to do with.

It wasn't intended to solve the exact problem you put forward, but I also knew (or was very confident in my belief) that there was no other means (that doesn't require installation of third-party tools) of obtaining window data for windows that weren't on the currently-active desktop. As far as I know, this remains to be true, and if anything, more so today than it was as Apple continues to tighten its security protocols.

I knew there was something off last time I replied. I couldn't put my finger on it, or must have been half asleep, but the fact that you're getting a data dump at all was all we needed. When I started to explain about data types and casting, etc., I was sort of doing this on autopilot, partly because what I was writing holds true when thinking about the problem in the context of my own system. On Monterey, this:

ObjC.import('CoreGraphics');
$.CGWindowListCopyWindowInfo(16, 0);

returns an opaque [Object Ref] that subsequently needs to passed to ObjC.castRefToObject() in order to get the "data dump" that you were already getting without this. This is obviously what changed between Mojave/Catalina and later version of macOS.

Therefore, the solution is simply to remove the call to ObjC.castRefToObject() so that this line:

return  ObjC.deepUnwrap( ObjC.castRefToObject(
        $.CGWindowListCopyWindowInfo(16, 0) )).filter(e =>

is now simply this:

return  ObjC.deepUnwrap($.CGWindowListCopyWindowInfo(16, 0)).filter(e =>

The rest is just regular JavaScript, so can be left alone.

I'm such an idiot.

Exploring this further, it doesn't look like information about which desktop a window is on is returned by the API. To double check, here's a list of all possible keys that can appear for any individual window (𝑡he first part of the list are Required Window List Keys that will always be included, and the latter part of the list are Optional Window List Keys that may or may not be included):

$.kCGWindowNumber
$.kCGWindowStoreType
$.kCGWindowLayer
$.kCGWindowBounds
$.kCGWindowSharingState
$.kCGWindowAlpha
$.kCGWindowOwnerPID
$.kCGWindowMemoryUsage

// $.kCGWindowWorkspace — deprecated in macOS 10.8
$.kCGWindowOwnerName
$.kCGWindowName
$.kCGWindowIsOnscreen
$.kCGWindowBackingLocationVideoMemory

I wonder whether $.kCGWindowWorkspace used to hold the specific information about which desktop a given window was on, but it no longer works. However, what will provide a little more help for you is the key $.kCGWindowIsOnscreen, which does at least inform you whether a given window is on the currently active desktop or not.

Here it is implemented in the previous function declaration:

Application.prototype.getWindowList = function() {
        ObjC.import('CoreGraphics');
        let pids = Application('com.apple.systemevents').processes
                   .whose({'bundleIdentifier':this.id()}).unixId();
        return  ObjC.deepUnwrap($.CGWindowListCopyWindowInfo(16,0))
                .filter(e => pids.includes(e.kCGWindowOwnerPID)  &&
                                           e.kCGWindowLayer==0   &&
                                           e.kCGWindowAlpha==1   &&
                                          !e.kCGWindowIsOnscreen &&
                                           e.kCGWindowStoreType==1)
                .flatMap(x => [ { id     : x.kCGWindowNumber,
                                  name   : x.kCGWindowName,
                                  bounds : [ 'X', 'Y',
                                             'Width',
                                             'Height' ].map(z => 
                                           x.kCGWindowBounds[z]) } ]);
}

Application('Script Editor').getWindowList();

That will return the list of windows that are on desktops other than the currently active one. To return only those windows on the currently active desktop, remove the ! from the beginning of !e.kCGWindowIsOnscreen.