ISSUE: Why is KM 17.6X Slower Than FastScripts to Execute Same AppleScript?

@peternlewis,

As you know we have been having this basic discussion for several years now:
Why is KM So Much Slower Than FastScripts to Execute Same AppleScript?

However, today KM is 17.6X Slower Than FastScripts.

I wrote the below script, and put in a KM Macro, to make BBEdit behave like most other Mac apps. I have to say that I was appalled at how slow KM is for this simple task.

Results

When Run Directly from FastScripts

image

When Run from KM

Peter, I'm sorry to be so blunt, and you know I love KM, but this really is unacceptable.
Could you please take a fresh look at how KM is executing AppleScripts, and maybe do it like FastScripts does it?

Thanks.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

MACRO:   Extend Selection of Text DOWN To END of Paragraph in BBEDIT

-~~~ VER: 1.0    2020-09-18 ~~~
Requires: KM 8.2.4+   macOS 10.11 (El Capitan)+
(Macro was written & tested using KM 9.0+ on macOS 10.14.5 (Mojave))

DOWNLOAD Macro File:

Extend Selection of Text DOWN To END of Paragraph in BBEDIT .kmmacros
Note: This Macro was uploaded in a DISABLED state. You must enable before it can be triggered.


~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

AppleScript

property ptyScriptName : "Select Text in BBEdit From Current Position to END of Paragraph"
property ptyScriptVer : "1.1" --  ADD Handle Multiline Selection
property ptyScriptDate : "2020-09-18"
property ptyScriptAuthor : "JMichaelTX"

### NOTE: A macOS Paragraph (multiple sentences) is ONE LINE in BBEdit ###

(*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PURPOSE/METHOD:
  • Provide the equivalent behavior in BBEdit as the macOS Shortcut OPTION-SHIFT-DNarrow
  • If some text has already been selected, this will extend the selection:
      • If partial line selection, then extended to end of current line
      • If already selected to end of line, then extended to next full line

REQUIRED:
  1.  macOS Sierra+    Tested In: Mojave (10.14.6+)
  2.  Mac Applications
      • BBEdit 10+

TAGS:  @Auth.JMichaelTX @SW.KM @type.Example @CAT.Selection @CAT.Strings @CAT.UI @Lang.AS @SW.BBEdit

REF:  The following were used in some way in the writing of this script.
  1.  2012-08-23, Pierre Ignot, www.betalogue.com
      Betalogue » AppleScript scripts for paragraph navigation and selection in BBEdit
      https://www.betalogue.com/2012/08/23/paragraph-navigation-and-selection-in-bbedit/
  
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*)

use scripting additions
use framework "Foundation"

set gTimerStartDate to current application's NSDate's |date|()

#### YOUR CODE HERE THAT YOU WANT To TIME ####


tell application "BBEdit"
  --  activate
  
  set oDoc to document 1 of front text window
  
  tell text of front text window
    
    set docLen to its length
    
    set oText to its selection
    
    --- Get Char Offset of Current Text Cursor Position ---
    tell oText
      set textStart to its characterOffset
      set textLen to its length
      set textEnd to (textStart) + (its length) - 1
    end tell
    
    
    --- GET Char Offset of Start of Current Line ---
    set curLine to endLine of oText
    set oLine to line curLine
    set lineStart to characterOffset of oLine
    set lineLen to length of oLine
    set lineEnd to lineStart + lineLen
    
    if (lineEnd > textEnd) then
      --- Current Selection is WITHIN Current Line ---
      
      --- Select ALL Chars from Current Selection to End of Line ---
      select (characters textStart thru lineEnd)
      
    else
      --- Current Selection is One or More Lines ---
      
      set nextLine to curLine + 1
      set oLine to line nextLine
      set lineStart to characterOffset of oLine
      set lineLen to length of oLine
      set lineEnd to lineStart + lineLen
      if (lineEnd > docLen) then set lineEnd to docLen
      
      --- Select ALL Chars from Start of Current Selection to End of Next Line ---
      select (characters textStart thru lineEnd)
      
    end if
    
    
    
  end tell -- text of front text window
end tell -- "BBEdit"

set elapTime to (round (-(gTimerStartDate's timeIntervalSinceNow())) * 1000) / 1000.0

set msgStr to "Elapsed Time: " & (elapTime as text)
set msgTitleStr to ptyScriptName
display notification msgStr with title msgTitleStr sound name "Tink"
2 Likes

@peternlewis, I just want to add this:
I have now saved all of the AppleScripts to the scripts > applications > BBEdit folder, and set the standard macOS shortcut for each, and now the operations in BBEdit feels just as fast as the native macOS shortcuts (like option-shift-DN) do in my other apps.

1 Like

Could this be the same as my historic whinge that typing is slow? If so, and I’m clutching at straws here, a speed gain from solving your problem could have wider benefits.

1 Like

Pff ... just different architectures for different uses, I think.

FastScripts is fly-weight, and narrowly built around executing scripts from a cache.

KM does much more, and needs a different structure.

If you find a particular tool "unacceptable" (sic) for a given job, then no problem – you just reach across your desk for a different one.

Or if you are up it, make one : -)

1 Like

It may or may not be true that KM's ability to do a lot of things contributes to how slowly it executes AppleScript - that's pure speculation on your part. I think it's a perfectly reasonable question to ask. And personally I'd like to know the answer.

@peternlewis - any thoughts?

3 Likes

I’m a Performance guy so I’m wondering about this:

It strikes me the test is for a VERY short piece of work. I wonder if the same ratio would be observed for a more complex one.

You see, my suspicion is the time is spent setting up and tearing down the environment.

In practical use most of my macros (and I suspect most users’ macros) consist of lots of short steps. So, if I’m right:

  1. Advice to automators might be to consider consolidating into as few steps as possible, perhaps with AppleScript / JXA / shell scripts doing the heavy lifting where before you might have used individual Keyboard Maestro actions.
  2. Possible enhancements to the KM engine to reduce the start ‘ stop cost of an action.

It’s possible nay probable neither of these is feasible in many cases.

Just thinking out loud. And I might try Item 1. Or even experiment with timing short actions in a loop.

1 Like

Not what I said :slight_smile:

(hard to imagine how an 'ability' would contribute to anything)

architecture.

as in

1 Like

Don't nitpick. My point is that of course it's a valid question, even if you think you already know the answer and we should just accept things as is or else make something ourselves (give me a break).

1 Like

it's a valid question

You feel there's a question in there somewhere ? I can see a request ...

My eyebrow was raised not at any 'question' but at a claim which I found a little silly and rude.

this really is unacceptable


I disagree with you that there is something 'speculative' , about the point that KM embeds scripts in a richer and more expensive architecture.

Consider the example below, which, whether run with the text script or script file option, returns a result to us. (Display in window, or various other options)

If we run the same from FastScripts, we see, and get, nothing at all. Gornisht.

The Fastscripts embedding is faster, but it's cheaper too.

Each instrument has its different role.

image

The upcoming silence from me is an indication that I knew better than to try with you, and that I have no idea why I bothered. I have no one to blame but myself.

1 Like

I agree. You baffle me :slight_smile:

More interestingly, I think you will find that the KM vs FS difference is just an initial constant factor; the time from trigger to the onset of script evaluation.

There isn't, actually, any significant difference in the rate of script evaluation – scripts run at essentially the same speed in the two contexts, with the same amount of memory allocated, but FastScripts is noticeably quicker out of the starting gate.

FS achieves that quick response to the starter gun not only by carrying much less baggage, and discarding any outer layers like results collection, but also by pre-cacheing all of the scripts in its menu.

We're all aware that the first run of an .scpt file can involve a perceptible recompilation delay, at the start of a session, which disappears on repeated runs. FastScripts essentially does all that recompilation in advance. (Taking not inconsiderable time and spinning a widget for quite a while, on the first launch, if you have a lot of scripts and haven't made use of FastScripts for a while).

a 200ms-300ms initial delay of that kind, building up the execution environment, and possibly re-compiling the script, before it begins to run, will clearly be much more noticeable with a tiny BBEDit cursor move than with a report generation or some automatic diagramming on OmniGraffle.

In my view, the solution to that is probably to:

  • prefer FS to KM for flyweight editing moves that return no value,
  • and prefer KM to FS for everything else, especially if the script returns a value.
2 Likes

Not entirely sure that I understand the FastScripts side of the timing methodology used above (or how comparability was guaranteed, quite apart from the problem of interpreting a constant as a dramatic ratio) but,

if there is a solid pair of comparable tests, it would interesting to see whether the overhead of an Execute shell script is greater than the advantage of an FS pre-compilation cache :slight_smile:

image

(I would guess that it is, especially with a tiny script, but I haven't designed an experiment)

Does the default action delay have anything to do with it?

https://wiki.keyboardmaestro.com/action/Set_Action_Delay

The Set/Reset Action Delay action lets you set the various arbitrary delays used in Keyboard Maestro, either for this macro, or (semi)permanently for all macros.

You can set the following delays:

* Between Actions Delay (0.0 seconds)
* Copy/Paste Delay (0.2 seconds)
* Drag Delay (0.1 seconds + double click interval)
* Large Text Display (5 seconds)
* Simulate Normal Keystroke Delay (0.001 seconds)
* Simulate Command Keystroke Delay (0.15 seconds)
* Simulate Modifiers Delay (0.15 seconds)
* Simulate Dead Key Delay (0.03 seconds)

I've already answered this. There is nothing wrong with asking this question, but it has been asked before and the answer isn't likely to change.

As noted, your title is almost certainly inaccurate, there is no performance factor difference, the script will execute just as fast, there is almost certainly only an initial start lag as systems are setup.

FastScript has the scripts pre-compiled and pre-loaded and in its memory and executes the scripts within the FastScript process.

Long ago, Keyboard Maestro executed scripts within its process. However AppleScripts touch lots of systems, and some of them crash. And crashing the Keyboard Maestro Engine is unacceptable.

So Keyboard Maestro executes scripts outside of its process. This execution incurs a delay required to set up the new process and communicate between the processes. Some of this can be mitigated by pre-compiling your scripts, but some of it cannot.

In any performance trade, I will choose not crashing over more performance.

If I had a solution that would have better performance and not crash, I would implement it. I do periodically look at this issue and look for solutions, but I have never found one.

If the performance of any aspect of Keyboard Maestro is unacceptable to you, then don't use it.

8 Likes

Peter, regardless of the reasons, from the user's POV, this script does run 17.6X faster when run from FastScripts and from KM. I have provided you with the data and the script and macro, so you can test for yourself. In this particular script, it extra time that KM takes is very noticable when running this script in BBEdit.

While I agree that no one wants any software on their Mac to crash, the real world is that every software, including KM, crashes sometimes. I have observed this and it has been reported by others.

I'm not sure what that means. You make it sound as if you used the FastScript method of running AppleScripts KM would crash much more often. I don't believe this is the case. I and others have been running AppleScripts for years with FastScripts and rarely encounter a crash.

I understand that years ago you had a bad experience with AppleScript. Have you tested AppleScript lately using the FastScripts method? If so, could you please report the frequency of crashes you observe.

Finally, you have previously reported that KM uses AppleScript in many of its native Actions. If this be the case then speeding up the execution of AppleScript could have the effect of speeding of KM in many different areas, beyond the Execute AppleScript Action.

Thanks for your consideration of this request.

Hey Peter, I just had an idea that you might consider: Create a KM Beta that uses the FastScript method of executing AppleScripts, and let a few of us use it for a while. This could give you and us some real-world data on just how reliable AppleScript really is.

I for one would be willing to use this high-performance KM Beta on my production Mac.

So, what do you think?

1 Like

Keyboard Maestro has almost no reported crashes currently. You would have to be fairly unlucky at this point - Keyboard Maestro’s crash rate is down to around one crash for every ten thousand user-days. While “every software, including KM, crashes sometimes” may be true, Keyboard Maestro itself does not crash in normal use for most users over their entire usage and I have no intention of changing that.

Yes, I do, and yes I believe it would, and no, I do not believe anything in this has changed.

In any event, executing scripts within the Keyboard Maestro Engine process is not plausible, since Keyboard Maestro performs multiple tasks simultaneously, which is not possible with AppleScript which is single threaded. As well as crashes, any script that takes any length of time would lock the Keyboard Maestro Engine up and stop it from performing any other tasks.

I would say the only practical way forward would be advice on how to tune macros. Two elements already mentioned in this thread are:

  1. Pre-compile. How?
  2. Coalesce steps. Not necessarily easy. But I would think, for example, a sequence of keystroke steps could be replaced by an Applescript step.

A third might be to use “Pause Until”, rather than “Pause for n seconds”.

This sort of advice might already be prominent. If not we could make it so.

1 Like

And of course, optimisation in context is not necessarily (perhaps not even often) the same as optimisation for run-time speed ...

Fusion of steps, more researched choice of actions, special measures with script files – all of these risk taking noticeably more human time and attention. The circumstances in which that would be a sensible price to pay for shaving off sub-second quantities of machine time could turn out to be rare ...

(I don't notice any general current of complaint about the run-time seconds used by KM. More a sense of gratitude that it brings more efficiency and focus to the human time)

1 Like

You can pre-compile your scripts as a text file.

However, in my tests this makes negligible difference.

The issue being described applies only to the Execute an AppleScript action (and to a degree, other Execute actions).

It does not apply to things like the Type a Keystroke action, so the last thing you want to do is convert them to AppleScript.

As with everything related to any form of optimisation - only optimise something that actively is causing you an issue. In the case described in this thread, presumably the 0.25s extra delay is an issue. In other cases (as part of a longer script for example), it would not be noticeable and indeed then, the script itself might then offer different opportunities for speeding it up - if the time taken was an issue.

2 Likes