KM Editor Pauses Sometimes

Questions & Suggestions
1

There's probably other topics on this problem, but I can't find them. I have a vague recollection of having this issue once before, maybe, so it's possible that even I have a topic on this. But I can't find it.

When I'm in the KM editor, sometimes the editor will pause, or hang, or whatever, for a few seconds. It's been happening a lot recently.

I have a theory that it may happen when KM auto-saves something. I'm thinking I have one or more variables with lots of data, or one or more macros with a really large image or icon, or something like that.

The problem is, I don't know how to debug this. I thought I had a way to check the sizes of macros, but whatever I have doesn't work. And it's been long enough since I got into this deeply, I just don't remember enough anymore.

So I'm hoping someone can give me some ideas. Thanks!

PS: I did actually scroll through all my variables, in the KM variable preferences window, and I didn't see anything overly long. That doesn't mean it's not there, though, because as my wife will tell you, I can be quite blind sometimes.

2

There is a Variables.Plist file Inside KM (which you can find using instructions found in other posts) that gets fully rewritten each time you make even a minor edit I'm the Editor. That's what was causing my 3 second freeze ups every couple of seconds. My file was 72 MB. That's a lot to be writing every couple of seconds. Check the size of your file. By cutting my file to 10 MB (by deleting macros) I was able to get a responsive editor. I no longer notice the delays.

Peter said a few days ago in a different thread that he was possibly considering a database to replace this file. That would make it much speedier, but then the idea of file corruption has to be addressed. My guess is that most users of KM have files under 10 MB so it's not a high demand change at this time.

3

Thank you! I should have thought of that, but sometimes areas of my brain are inaccessible.

I cleaned up a problem variable, but I'm not sure if it'll make a difference or not. But here's the information, for anyone who might be lurking:

In KM v8.2.4, the variables are stored in a sqlite database named "Keyboard Maestro Variables.sqlite". I downloaded sqlitebrowser from https://sqlitebrowser.org/blog/version-3-11-2-released/ and opened up the database.

By the way, I got a warning from the program saying something about some unsupported something or other, but for what I was doing, it didn't matter.

I ran the following SQL command:

select name,length([value])
from variables
order by length([value]) desc

Note that the field "value" must be referenced using square brackets, because it's a reserved word. Here's my results:

image

Needless to say, there's one really large variable. And it normally gets cleaned up, but I remember running the macro responsible for it, and the macro crashed, so it never got cleaned up.

Anyway, I copied the name of the variable and used the KM Variables Preferences window to delete the variable.

I'm dubious whether this will help. On the other hand, it's quite possible the problem started around the time I ran that macro, so we'll see.

Thanks again, Sleepy!

4

Dan, here's another tool you and others may find helpful:

MACRO: Get List of KM Variables and Size

I'm not sure of the relationship between the KM Variables SQLite DB and the Variables plist (which I believe is the primary source that KM uses). and one has to wonder why have both? An old DB adage is that as soon as you store the same info in two different places, one of them will ultimately be wrong. :wink:

KM has two structural design points that are not really well scalable:

  1. KM has a monlithic macro file that contains ALL macros. It gets continually rewritten with every little change you make while editing a macro. And the same goes for the KM Sync file.
  2. Same is true with KM Variables -- the plist file contains all variables and gets continually rewritten.

Using a single file to store all macros/variables also makes it very difficult to backup and restore a particular macro. To workaround this issue, I wrote this macro:
MACRO: Export (Save) Selected Macro

Several Mac apps have taken a hybrid approach, which I really like:

  1. Use a DB to store metadata for all items
  2. Store the actual contents of the item in a separate file/folder for each item

Databases are very safe to use. I've been using them, and creating apps based on them, for decades. It is a very, very rare event that a database becomes corrupt and unrecoverable. Today's databases are even more robust.

Sorry for taking the discussion a bit off-topic. I hope your delay issue is resolved.

5

You are welcome. Sorry that I was conflating the Macros.Plist file with the Variables.sqlite file. How big is your Macros file? My experience is that if it's under 10 MB then the Editor is responsive, if it reaches 70 MB it's very unresponsive. My Variables file is only 4 MB. I'm not sure if that file gets completely rewritten every time an edit is made. Probably not.

6

I guess the Editor is the only thing that changes the database (i.e., if KM had a database). Therefore it's only a sequential process, not a bunch of asynchronous processes, that make changes to the database.

7

Dan, strangely enough, I too just had this happen, about 30 min ago.
It happened at least twice, and I believe both times just after I did a COPY of text in KM Action.

Unfortunately, I was in the middle of something urgent, so I could not stop to debug.

I just ran my Variable Sizes macro, and this is what it showed:

KM Variable List
Total Number: 464
Total Size: 157163

---------------------------------------------
Chars	Variable
---------------------------------------------
54667	ELP__LinkList
26195	kmfamSelectData
14679	OLBody
10389	kmfamUpdatedUsageData
4537	DND_MMF__LastMouseList
4342	DEBUG_Local__Log
3042	xLocal__AppList
2974	Result
2271	URL
1928	Variable
1667	DND_OCR__Collector
1439	HL_Word_Setup
1351	DND_MSW__WindowList
1299	DND_MMF__Log
1240	DND_MMF__MousePresetList
. . .

My Keyboard Maestro Variables.sqlite DB is 1.4 MB
My Keyboard Maestro Macros.plist file is only 12.3 MB.

==I don't see a plist file for KM Variables.==
Has it been replaced by the SQLite DB?

IAC, neither 157K bytes nor 1.4MB seems like very much storage for variables to me.
Nevertheless, I'm going to clean out my KM Variables list.

8

Remember that Monty Python sketch where four men are arguing, "You guys had it easy"? Well that's how I feel right now. You are arguing about a 3 second pause that's intermittent or every 30 minutes? I've been dealing with 3 second freezes every 3 seconds for the last year. That is, until someone told me recently to reduce my macro file from 70 MB to 10 MB. Now I don't notice any freezes. It has doubled my productivity because the spinning hourglass has dropped from 50% of my time editing to under 1%.

9

Do you have a spinning disk (or a fusion drive)? Even as a second mounted volume? On modern systems, the system can spin down the disk on occasion, which can then result in a several second lag when it spins it up, and that process is often synchronous, leading to a lock up while it happens.

10

I do have a Fusion drive, but I've had a Fusion drive for years, and it's never been an issue.

11

I'm with you - I don't see a plist file for variables. With that said, if I were you, I'd clean up those large variables and see if it makes a difference.

12

FYI, cleaning the big variable didn't help, and I didn't think it would.

My memory is coming back slightly. IIRC, the last time I had this, it was because I had some icons that were very large. I've been trying to remember to always resize them before I paste them into the macro, but who knows if I've been successful or not.

13

Of course, the user has control over this:
image

That seems highly unlikely to me. Modern Macs first make use of RAM, and only use HDD/SDD when needed. Of course, since KM uses a monolithic single file for ALL Macros, which is being continually updated as we edit, I suppose at some point the Macro file will become so large it must be written to disk.

Peter, two points that lead me to believe it has nothing to do with spinning disks:

  1. I have a SSD, and I just experienced the delay in the KM Editor
  2. In my old iMac, I have a fusion drive, and I don't recall ever seeing this delay in KM Editor

@DanThomas,
You can get even more control using Terminal:
How to customise the power settings from Terminal on the Mac

So, as a test, you could uncheck "Put hard disks to sleep", and then set the sleep delay time in the Terminal to never or something large.

14

If you can catch it in the act and get an Activity Monitor sample, that would tell me something. Otherwise, without a way to reproduce the issue I am unlikely to be able to figure it out.

15

It has nothing to do with the hard disk going to sleep. I don't have this issue with any other application, and my hard disk doesn't go to sleep every 30 or 60 seconds.

16

How would I get said sample? I have Activity Monitor running all the time.

17

In Activity Monitor, with Keyboard Maestro selected, select Sample Process from the gear menu.

If you leave Keyboard Maestro selected in the Activity Monitor, then you should be able to do:

Activate Activity Monitor
Type the ⌥⌘S Keystroke

As a macro to trigger it quickly. Note: Type the ⌥⌘S Keystroke to Activity Monitor does not work for whatever reason. Select Menu Item would work too.

18

Thanks, Peter. I've got it set up, and a hotkey assigned. Of course I can't duplicate it because of the Heisenberg Principle, but hopefully it'll happen and I'll capture it. Of course, the delay seems long when it happens, but now it'll seem short as I try to capture the moment.

Do you think the hotkey will work during the delay? Just curious how this works.

Note: Type the ⌥⌘S Keystroke to Activity Monitor does not work for whatever reason. Select Menu Item would work too.

I'm trying to figure out what you meant here - I think you left a word out or something.

Thanks again.

19

Yes, it should do, because it is the editor that is stalling, and the engine should still be operational. Tho it may not, it depends on what is going on.

You cannot use the Type a Keystroke action with the “Send to” option to send the ⌥⌘S to Activity Monitor while it is in the background. That does not work (or did not work in my testing anyway). This is not overly surprising, sending keys to applications in the background, while technically supported, is decidedly unusual and you can expect it to behave oddly in many cases.

20

Got it. Thanks.