After 40 minutes that last Search and Replace action finally completed. And the memory for the Engine, while reaching large values during those 40 minutes, is now back down to a more reasonable 87 MB.
I have spent hours trying to reduce my macro to the smallest macro that causes the memory leak, or the Engine hangups. And I made a wonderful discovery. In my code fragment (above) the variable name LocalTestOCR exists. The problem was in my code the correct variable name was LocalTextOCR. When I use the correct variable name, KM Engine is stable. When I make a typo in the variable name, to a non-existent local variable, the KM Engine crashes within a couple of minutes. I'll have to do some more testing but the Engine seems to be stable now that I'm using the correct variable name, at least for 5 minutes. But I know Peter would like to see the smallest code fragment that causes KM to crash, and I'll try to create that now.
Okay here's the shortest program I could write that causes the Engine to crash within 20 loops. I count the loops by listening to the sound of the glass sound effect. By the time it reaches 20, the Engine is nearly dead, although the amount of memory it grabs is fairly small.
The action has to be a Regex search, although if you add a non-Regex search to the loop that will slow/hang too, it just won't contribute itself to the memory loss. You need the vertical bar to make it crash.
It's entirely possible that many of my thousand macros use these Regex actions. I really never considered that as a problem before today. I supposed I could use awk and sed in place of the Regex action in KM, I just didn't know it was causing me any issues. I also use lots of "String contains" conditions in KM and I'm not sure if they are using Regex routines "down under."
I don't think I've ever had the search action or string condition fail or slow KM down myself. What are you using for the contents of the LocalZZ variable? If you share that too, @Tom and I and anyone else who's interested can do the same test themself and report our results.
The contents of any local variable are empty when you start running any macro. That's important here. When I used local variables that already existed, I didn't see the crashes, as I stated in a previous post.
So, you didn't initialize the variable before this search? Not to be like the doctor in that old joke who tells the patient it hurts when he does this to stop doing that, but how often do you need to search an empty or non-existent variable? Even if searching them does cause a crash, if there's no practical reason why you would ever need to do so aside from a test like this, I kind of fail to see the point. Am I just missing something?
I appreciate your humour. It's a good joke, along with the line "if it ain't broke, we cannot fix it."
I often use variables, even local variables, to accumulate data that I collect. So it is quite common for a macro to use a local text variable and assume it's empty at the start. Should I not be taking advantage of this language feature, because doing so will break the Engine?
I suspect that you, or someone, will be testing my code, to get a second opinion. Maybe you won't get the same results. Since we don't want average people downloading code that breaks the engine, I decided it was best to just include a screenshot, not upload the code itself.
Philosophical questions about how the engine ought to be used aside: would it make sense to test if the variable is empty before you try and search within it for a string? It's an extra action but it might prevent the issues you seem to have been having.
I fully understand what you are saying. That might be the best solution. On the other hand, this might be an actual bug in the Engine that can and should be fixed, in which case I don't have to modify hundreds of macros.
Do you think KM should allow us to use empty local variables? Because it's not just at the start of a macro where I would have to test this, it's every time I use the variable inside the macro. Every single time. Do we really want to have to write extra code to avoid using an empty variable in a common KM action?
I have roughly 1000 macros (most of which are archived at the moment) and if I have to modify all of them to check if a local text variable is empty before I use it in an action, that would be a terrible problem for me.
I'm a little surprised nobody has tested my Engine crash macro yet.
Could you please upload it?
(Export the macro, File > Export Macros, and drag the exported macro into the forum editor.)
OK, I tried to rebuild your macro, with this regex content: |(a-z), and I can confirm your result:
No increasing memory, but the engine becomes unresponsive quickly.
In this case the culprit seems to be the size of the variable. Your regex makes the variable content grow exponentially, and when it reaches a certain size, the engine becomes unresponsive. Since your local variable is inside the loop, it isn’t reset.
You can easily confirm it this way:
- Make your variable global (removing the “Local”)
- Remove the loop (either by inserting
1in the calculation field or just take out the main action from the loop) - Run the macro one time
- Now open your variable
ZZin Preferences > Variables - Run the macro manually a couple of times, until the engine starts to slow down.
- Now, in the open Variable window, delete a big part of the variable content (shift-select content and hit the Delete key).
- Run the macro again and you will see that now the engine is responsive again.
But I don’t see how this is related to your memory issue.
To illustrate it: The growing variable content in the Variable prefs window:
Okay. I had indicated that I didn't do that earlier because I didn't want people crashing their Engines. Although it's yet to be determined if I'm the only one affected by this macro or not.
Memory Test CRASH ENGINE.kmmacros (2.3 KB)
See post above, I already have rebuilt your macro.
I'm studying your response.
Okay, obviously I don't understand that simple Regex expression. My intent in that expression was to reduce two characters (a vertical bar and a letter) down to the single letter "a". What am I doing wrong? How would you achieve that requirement? What regex would you write to reduce a vertical bar and a letter down to a single letter?
Oh. "|" is a regex operator not a character. That explains it. Well it doesn't explain my problems, just why this macro crashes. I think I used "|" in some of my recent macros over the last week. I can fix those instances, but you are right this is unrelated to the memory leak. I guess it's about time for me to reinstall KM according to Peter's instructions. You were a good help, thanks.
Yes, | and also ( and ) are meta characters. | is an alternation (“or”) and the parenthesis define a group.
Actually I don’t understand your regex, but it seems you have created some kind of a “pathological regex” 
For more regex help, search the forum for “learn regex” and you’ll find lots of hints and great links, like for example…
I've had long (tens of seconds) hang-ups before by using regex. In my case, the file being searched was pretty large. I realized that I could speed it up hundreds of times (at least) and make it complete the search in a fraction of a second simply by making my regex more efficient. I pasted my file's text into https://regex101.com and played around until I increased the efficiency significantly.
Okay I uninstalled KM, renamed the KM folder in Libraries, reinstalled it.
To my surprise the new installation seems to already be "registered." So is this really a clean installation? Where does KM store its registration data? I thought I would have to load my KM license back into it.
I also examined how much memory it is using, a bare 23.5 MB, and nothing I can do can make that number change. Which is a good sign. I will take it slowly and import as little as possible and try to stress test this new baby to see if I can get it to leak. New babies tend to leak, I've been told. 