Tips on How to Fix or Help a Very Laggy and Slow Keyboard Maestro?

Thant’s right, collapse.
Otherwise, these actions will constantly check if conditions meet.

2 Likes

Cool, thanks for that. I do already do that. I thought maybe you'd meant there was some other way to hide actions even more that I didn't know about.

It depends on the macro and what it is executing.

If it is executing an action that is looking for an image on the screen, then yes, it will use essentially all the CPU (100% is 100% of one core. If you have 8 cores, then the max would be 800%).

When you stop the macro, Keyboard Maestro Engine will return to using a fraction of a percent. It may take a short while for the various processes to complete (a second or less generally). The system may lag behind in showing the usage since it is presumably averaging usage. So after Keyboard Maestro Engine has stopped using any CPU, it might decide that 750% previously and 1% now mean 300% displayed.

Most actions take significantly less CPU - generally with a macro running the CPU usage might be around 20% for example. But image searching and OCR will take a lot more, and inter-process communication and scripting can be relatively high too (though typically they would still be < 100%).

Keyboard Maestro Engine is designed to be highly efficient in CPU while it is not doing anything (since it is running all the time, it would be bad if it impacted performance generally).

But when it is running, it is designed to do its job, and in many cases, it is designed to do its job as fas as possible, and at that point CPU usage is a significantly lesser priority.

1 Like

Thanks for that explanation, Peter!

(ETA again - I have a quad core so I find that hyper-threading can allow up to 800% so that makes sense.)

The macro did have a few different find an image actions, so that makes sense.

I don't want to get too off track of the original intent of the thread, but this does make me wonder something about a macro I was just working on. Suppose I have ten macros in a group, and each needs to find a specific image on the screen, and only one of the images would ever be on the screen at any time. Otherwise they are all identical macros. Is it better for speed and lag to:

  1. Create one separate eleventh macro with a find an image action that can be expanded to include all ten images in one action using the 'any of the following are true' parameter, and then include an 'execute a macro' action in each of the other ten macros all pointing to that eleventh macro, so that there's only one find an image action altogether even though it's loaded with ten images

  2. create a separate find an image action for each of the ten macros, so that spread over the ten macros there would be ten find an image actions (one per macro) each with one distinct image

  3. it doesn't matter

?

I'm not really sure what you are asking.

If you are talking storage size in the macros, then each image is stored in the macro and each copied would be stored, and the image size would be the bulk of the storage size no matter how you arranged the actions.

If you are talking execution speed, then if your macro knows which image it wants to look for, it is better to only look for one image. But if you need to look for all ten images, then it may be faster to have a single If Then Else, with 10 image conditions in it, and use the FoundImage token to determine the location of the match - but that would not tell you which of the images matched. It would however operate simultaneously, though given it is already using all your processing power, whether it is simultaneous or sequential is not likely to make much difference. Hopefully I have coded it to limit the number of simultaneous threads to something reasonable, but this isn't exactly a case I would design for.

And, at the end of the day, for all profiling question like this there is really only one way to know, and that is to try each of them and see which is better.

1 Like

Thanks! Yes, I think you have the gist of what I was asking. In this instance, I don't need to look for all ten images at once, only the one for whichever of the macros is in turn working at any one time. Therefore, from what you're saying it would be slightly better to have ten separate actions each with its distinct image and each within its own macro, so that when any one macro is running it only needs to look for that one image. That's what I'll go with then.

However, to clarify, I usually don't have any problems running macros on the engine. I only mentioned the 750% cpu usage from the engine because I was looking at it to answer another poster who'd asked me what my usage was. The engine generally seems to run fine (aside from sometimes being a little laggy to start up a macro properly).

I was mainly asking for speed and lag while editing, so on the editor. I thought maybe making the actions one way or another, especially with found images, might somehow speed up or slow down the editor.

Right. You should never notice the engine’s usage - if you aren't using the engine, then there should be no usage, and if you are then you should care more about it doing what you've asked than what resources its using.

The editor is a different matter. There are lots of ways the editor can get lagged, and lots of ways that those lags can be resolved, depending on what specific issue you are having. The most important one is only have one editor window open.

Thanks, yes it'll be great if I can figure out what's causing my editor to lag so much. In all the time I've had Keyboard Maestro, I've never even thought about using more than one window so I have that part taken care of down pat.

Generally, the editor will get laggy if either or both:

  • You have a large number of actions in the macro you are editing
  • The actions you are editing are displaying real time information (specifically the resulting value of conditions for If Then Else or loops or such), and that real time information is expensive to determine.

A particular case will happen when you have a Menu condition that is searching the front app for a menu that does not exist in Keyboard Maestro. Because essentially Keyboard Maestro asks Keyboard Maestro Engine to determine if the condition is true or not, and then Keyboard Maestro Engine hammers the editor searching for the menu all through Keyboard Maestro’s very extensive menu structure. The easy fix for this is to use the menu path instead for your menu condition, which is always a good idea since you always (ok, almost always) know where the menu you are looking for will be.

You can also run in to trouble when your macros are displaying information that requires interacting with web browsers or AppleScript or JavaScript.

1 Like

I did export and delete about half of my 50 or so macro groups so now I'm down to about 25 groups listed (all disabled unless being used or the Global Macro Group), and restarted everything, and it seems to have improved performance a bit. Now instead of waiting say four seconds every time I do anything it's now down to maybe a two second wait each time.

Thanks, Peter. This could be one of my problems. Any particular macro I make has probably between 5 and 50 actions. I'll have to work on splitting them up more between macros.

I had noticed that in the past, for instance a spinning wheel pause until a result showed up of true/false on an if/then, or a search variable finding something. So because of that I'd always started collapsing/minimising every action once made hoping it would stop that from happening. Since they were minimised I wasn't exactly sure if it worked or not. If they are minimised but displayed in the editor (if I'm editing a macro with displayed but minimised actions on the screen) do they then not do the searching for the information?

I do use a lot of actions that do these kinds of tasks. I use a lot of if/thens, a lot of switches, and I use a lot of variable searches. The variable searches in particular have given me lots of headaches to edit over the long run that relates to the whole lagging issue. Even before every single thing I did started to lag, sometimes editing a search variable alone would become really laggy. Sometimes it would/will even freeze or crash the app. It seems every single change I make to the input of a search variable regular expression causes it to re-search right away, so if I'm rewriting or adding a lot to the input it can get really really laggy, and if I add a regular expression character that makes the input do something strange, even though a later character would fix it, that's when things like freezes, crashes or looong lags (of minutes) can happen. At one point I had started adding disabled 'input text by pasting' actions below each search variable, and then writing the input in those before copy and pasting into the search variable when I was done, but found that too cumbersome so I stopped. I had also tried quitting the engine while editing which did work but again was too cumbersome since I'd often need to try things out in between edits and therefore constantly open and quit the engine. Is there any advice on the best way to edit regular expressions in search variables to optimise the editor performance while editing?

Also, I sometimes use big or complex regular expressions. In general, could this contribute to lag in the editor?

I will keep this in mind, thanks. I rarely use menu conditions so I don't think it's what's causing the lag but I do use them sometimes and will make sure to do that.

This could be a problem as I do sometimes have the macros interact with browsers... although I usually don't use specifically-focused browser actions. I'd found in the past they might be a little slower or less consistent so I usually make things work with non-app/program/browser specific actions that still work on the browser. For instance, instead of 'next chrome tab' I might use 'press keystroke' to press the computer's hotkey combo that moves chrome to the next tab.

Could you post one of your macros so we can understand your style? I wonder if we can make suggestions that might help

1 Like

@peternlewis: enhancement request

Could you please add an option to the KM Editor to enable or disable real-time executions (evaluation) of Actions in the KM Editor.

I think this could greatly improve the performance of the KM Editor.

You can design the UI however you think best for most users, but my preference would be an item in the KM Editor Action menu, with a shortcut key, that toggles real-time execution of all Actions.

To be honest, I rarely need or use the data shown by evaluation of each Action in the Editor when I am developing or changing a Macro. Occasionally (I'd say ~10%) of the time it is useful. The ideal use, IMO, would be to have the real-time eval disabled (off), and then whenever I need it I can quickly get it by a simple shortcut keystroke.

Thanks. This could really help.

4 Likes

I second this request. My KM was unusable for days on end, freezing, taking 2 minutes to type a single character into a macro title, and similar things. Through email, @peternlewis and I discovered what this thread later revealed: my KMEditor was running condition and found image actions while they were onscreen in one or more editor windows.

I would very much like an easy temporary and permanent way (Preference, maybe?) to tell KM to stop doing that so that I can write, edit, and test macros without having KM hit 500-850% of CPU usage.

2 Likes

I (of course) third this request, if it's possible.

1 Like

Okay, it might take a little while but I'll work on it. I don't want to post actual macros I use for reasons, but I think I could make a dummy macro filled with similar actions and such to closely represent a type of macro I use. I will say that my macros can really vary which I suppose might be true of any of us, but I'll focus on trying to represent a more complex one. By the way, what is the best way to post a macro? Screenshots?

1 Like

See How to upload your macro. Instructions for uploading to an existing post are just below the main instruction.

This is an old thread, but since I found it when trying to fix a problem with severe lagging in the editor I think this is probably still relevant.

I tried a lot of things, managing to reduce the size of my macros file by a factor of 2. There was no impact on the problem.

What did work was to save the macro in question, delete it from the library, and then read it back in. The lag went away completely.

This is purely empirical and based on a small number of cases. O caveat emptor. But it has worked every time I've tried it.

1 Like

Hey Jeff,

That's interesting. Can you post a copy of that macro?

Did you read through the whole topic?

Did you notice this?

Numerous Large images in the Keyboard Maestro database can be a problem as well.

-Chris

1 Like

Hey Chris,

There was a LOT of text in that topic! I didn't read it all in detail, but did look through it.

Posting the macros I was working on wouldn't help. There was nothing special about them apart from the fact that the editor was lagging when I edited them. Although I should write a macro to save a macro, delete it, then read it back in. (There will be a BIG "O caveat emptor!" sign on that one. :wink: )

Following the previous discussion, my first thought was images in the KM database. I often give macros custom icons, typically to match the icons that I use to launch macros from TouchPortal. So the first thing that I tried was saving a copy of the database, then going through and removing most of the icon images. This reduced the size of the file substantially (a few MB). But when I again tried to edit the macro in question the problem remained.

Conclusion: At least in this single instance, the problem was not using custom icon images.

Then I reloaded the larger database containing the images. The problem persisted. (Conclusion: Saving and restoring the entire database didn't fix the problem with the editor lagging.)

But when I saved, deleted, and reloaded just that macro the problem went away.

While I don't have any particular insight into exactly how the editor code works, I'm not surprised at the outcome. The kmsync file is actually a binary property list file. Since it is being updated with every keystroke, I suspect that the internal structure of the file can get pretty convoluted as a large macro is edited again and again. Saving then rereading a macro might effectively "defragment" that part of the database by writing out a serialized version of the data then bringing it back in as an ordered, contiguous chunk. That should greatly simplify the bookkeeping needed to handle the next keystroke while editing.

Presumably copying then reloading the entire database doesn't fix the problem because the proper list file is read back in in its previous fragmented form.

Perhaps the right macro to write would:

  • Save the entire library for safety sake
  • Go through and save/delete every macro
  • Go back and reread all of those macros
  • Clean up after itself

I did notice the post about real-time evaluation in the KM Editor. That is an option that I probably would not use. (What can I say? I'm sloppy enough that I benefit from a guard rail against things like typos in variable names or calculations.) If my experience to date generalizes that might obviate the need for the change.

I hope you've had a great holiday. Happy New Year!

-Jeff

2 Likes