Is it possible to find an image on the screen that matches the image in a clipboard?
Thanks.
Not really, no. It might be possible by creating the XML for the action, but I’m not sure how difficult that would be.
Well, thanks anyway.
If you ever get a chance to implement this, it would be nice. While you’re at it, it would be nice if the “fuzziness” parameter could be supplied by a variable. I’d like to be able to “prompt” for this.
In case you’re wondering, I was hoping to be able to automate a workflow for removing duplicate frames from short video clips, in Final Cut Pro X. I could go into details if it matters, but I doubt it does.
Anyway, thanks for the reply.
The tricky part is, as it often is, the UI. It's hard to see what UI would allow entering a variable for the fuzz or a path or named clipboard for the image well.
Adding complexity to the UI to support this has to be worthwhile doing, so that means there has to be enough users who need this facility to make it worth the cost in complexity to everyone else. Without a good UI for allowing this with minimum complexity it's hard to make it happen and hard to justify the time to do it.
Creating an XML for the action and running the action via AppleScript is not that hard except for creating the image data, which is just base64 encoded image data. So this sequence works for example:
Keyboard Maestro Actions.kmactions (3.3 KB)
This is the sort of thing "do script" accepting XML was made for - cases where your need of the facilities exceeds the ability of the UI.
4 Likes
That is unbelievably cool! I’ll give it a shot. Thanks!!!
BTW, I totally get the need for working only on things that would help a lot of people. I’m a developer, and I would be the same way. Of course, in this instance I’m a user, so I had to ask. 
With as fantastic as KM is, you wouldn’t think I’d have any frustrations with it, but I have one: There’s so many features I either haven’t come across, or haven’t realized how valuable they are. I discover new things almost daily, and often times I think “I wish I had known about that earlier!” Not your fault, of course.
Great job, as always. Thanks again!
It's always good to ask - many, many features of Keyboard Maestro come from user suggestions. Often I think “Why didn't I think of that!”.
The best solution to this is probably to simply skim the various facilities on the wiki.
Specifically:
Probably in that order. But just skim through them, don't try to dig in to each one, you'll be there forever.
That literally made me LOL. Thanks for the suggestion. I’ve been known to read manuals cover to cover, multiple times. It’s amazing what you can learn, even the third or fourth time through.
And yes, more than once I found myself wasting a day because of some new feature or idea I came across. So thanks for the safety tip!
Since this topic recently came up again, I just wanted to note that image files can be very large, and storing the image data in a KM Variable permanently may not be wise.
After you finish with a macro like the one @peternlewis provided, be sure to delete the KM Variables that contain the image data:
Actually, I would not even keep the image in a KM Named Clipboard. Store the image as a file, already coded to 'base64", and then load into a Local Variable when needed:
The 'Read file to variable' doesn't work for me. It gives me an error:
Read File action failed to read text file with error Error Domain=NSCocoaErrorDomain Code=264 "The file “Test” couldn’t be opened because the text encoding of its contents can’t be determined."
It's a png image created from base64 code.
Variables can only contain text (specifically UTF8). So you cannot read a .tiff file in to a variable because it is binary data.
What do you want to do with the Local__ImageData variable after this?
If you want the base64 of the image, use the Execute Shell Script I posted above.
Hi Peter,
That's what I thought.
I ended up using some similar to what you posted above. I have a text file of image data for 40 images. Each image base64 data is brought into the ImageData variable and ActionData is executed.
Out of curiosity, I noticed that when I run my Macro utilizing the XML of the Action and the ImageData etc. (as described above), like this:
it is much slower than when I run the traditional macro like this:
Again, this runs 40 times so any short pause adds up.
I really love this new Macro I've created utilizing the 'variable' images (from the Text file). Is there any way to speed it up so that it's on par with the traditional Action? Or any idea what specifically is adding to the total duration? If not, have you thought about adding the ability to insert a variable for "The found images:" ?
Thanks!
Without profiling, it is hard to say where the speed difference is, it could be in the switch out to AppleScript, or it could be in the creation of the action, especially in the conversion of the base64 back in to an image. Or it could be something else.
You could try a similar test with a trivial action, and see what affect the Execute AppleScript has, then you could try a similar test with the trivial action containing a disabled version of your action so that the creation of the action, including the decoding of the base64 all has to happen, but not the actual execution of the action, and then you could try a similar sized action and see how its performance compares.
But realistically, Keyboard Maestro is not designed with high speed performance as a primary goal.
The question would be, if the image was stored in an image file and read by Keyboard Maestro, or if the image was stored in a Named Clipboard, what would the performance be like, and I don't have an answer for that, and frankly it wouldn't be the design goal for implementing either of those features, it would simple be enhancing the power of the action. Executing XML is a sort of “catch all” to allow you to go beyond what Keyboard Maestro currently enables, but I am always attempting to add more power to what Keyboard Maestro enables, and that would be why I'd implement the features, not speed per se.
As far as implementing those features, the primary issue is UI. The actions containing these found image searches are already very complex, adding another selector and associated fields to enable the image to come from places like Named Clipboards or a file would add even more complexity.
An alternative approach, assuming you have a fixed set of 40 images is to just create a fixed set of 40 actions. The actions can be created via AppleScript from a folder full of images, so it is possible to set it up in such a way that you could change the images and then automatically re-create the actions.
Hi Peter,
I have actually already converted the images to base64 and the text file is simply a list of all 40 images AS base64 code. So the action is simply grabbing the base64 code and putting it in the ImageData variable. But maybe you're saying that KM is actually converting the base64 back to image in order to find it?
Yes, the 'faster' Macro / the original version of this macro actually has 40 separate actions for each image. So basically, the image above "For Active Groups" - there was 40 of these with a different image set for each one. The main reason why I wanted to change this Macro was for simplicity of upgrading it or adding to it later. Instead of having to create another Action / Sub-macro, I could just add another base64 Image to the text file etc.
In any case, I'll have to run some more tests. My guess is that the 'slow down' has to do with the need to "get the image data" for each Image, while when with a Traditional Action with a static image, it's already 'in memory' - this combined with Applescript etc.
Thanks
Correct. So Keyboard Maestro is reading the base64 and creating an image from that which will take some time.
Sure, but you could do that now, and then have an AppleScript create the actions from the text file, but run that only when the text file changes. Or probably better yet a folder of images which would be easier to maintain.
Just to reiterate what Peter is saying:
You can write something either in AS or JXA, that loops through all the images in a folder (for instance), and creates the KM actions automatically.
So when you need to change an image, or add more, or whatever, all you have to do is re-run the AS or JXA code, and it will re-create all your actions.
And actually, you could write most of it in KM. If memory serves, you can do everything except actually creating the actions.
I've attempted to use Peter's suggested actions and AppleScript but I'm getting mixed results.
Most of the time an image is not found (at least 95% of the time).
Things I've tried:
• Adjusting the Fuzz to various values from 1 to 100.
• Adjusting the Fuzz to values up to 1000 - it tends to work better but still fails often.
• Using newly created clipboards (in case overwriting a clipboard is an issue)
• Using different names for the file being written to (in case there are silent overwrite warnings)
• Deleting variables used
• Ensuring my capture area is unique
• Changing the image format to PNG
• Trying different values for ScreenAreaType
My script is basically the same as Peter's with a few modifications picked from other replies and my intent to capture the image into the clipboard:
Keyboard Maestro Actions.kmactions (5.3 KB)
What could be going on here?
Hey Greg,
Hard to say, but adding a few pauses works for me every time in 10 tries.
-Chris
Capture Image and Use Found Image Action v1.00.kmmacros (13 KB)
Thanks Chris - Does that mean you were able to reproduce my issue without your pauses?
I've now added pauses up to 1 second in the places you suggested, but it's still not finding the image at the same rate as before.
Hey Greg,
I didn't even try...
Just did.
No problems on my system without the pauses.
I reckon you should use a Click at Found Image action to visually test that your captured image is being found.
If you can get that working then you should be able to get your macro to work, or so I think.
-Chris
When I grab the macro captured image from /tmp/kmimage.png and load it into the Found Image box manually, it indeed does not work.
However, if I capture the image with Cmd-Shift-4 and load that into the Found Image box, it does work.
It seems like the screen capture is different in KM? ...or something else I'm not seeing? @peternlewis?
BTW I changed the format in the macro to png since that's what my Cmd-Shift-4 captures.