Erm... KM? 
You do need a bit of shell scripting to get the current memory usage, but the biggest problem is how to use the Engine to launch the Engine when you've just quit the Engine... You could farm that out to an external AppleScript app, but using an asynchronous AS action is working in my tests.
I won't guarantee the "multiplier" section, as I've only seen MB-usage -- but it should work if (as I suspect) top use K and G for kilo and giga. I didn't bother with T -- if KME's memory usage is that high then you've other problems to worry about!
Use the "Memory" figure from Activity Monitor to judge the highest amount you want KME to have before a restart, and put that value into the first action. Then just have the macro run every so often, for example on an "every 5 minutes" periodic trigger. If the Engine's memory usage is higher than your stated maximum the macro will pause until no other macros are executing then restart the Engine for you.
This has been hacked together, and I'm sure it could be improved on -- more effort has gone into making it work (and not blow up your Mac!) than in making it efficient/pretty. I look forward to what others suggest.
Monitor and Restart KME.kmmacros (9.0 KB)
