That's probably at least in part because Keyboard Maestro is slow to run AppleScripts.
Script files run a bit faster than text scripts.
You could also emplace your user-prompt within the AppleScript as XML to run with do-script, so the run-time overhead doesn't cause extra latency.
I usually run AppleScripted stuff from FastScripts for extra speed, and when I do that I put any Keyboard Maestro actions in the script if possible.
-Chris