Even a normal double-click is NOT a trigger for KM, so I really doubt adding a SHIFT to that will help.
But, as @gglick has suggested, you could open a file selected in Finder, PathFinder (or really anywhere) by using a KM Macro to do, but use a hotkey trigger. I have just such a Macro I use to open files in what I call their "alternate" app. If this is of interest to you, let me know and I'll clean it up and post.
I like your idea, it works. I have another idea to use launchbar. The actions are:
trigger “instant send”
get the word with applescript and write if logic to choose the application
type the application name in launchbar
type enter
This method will have a little delay but it even works in applications beyond finder.
I also find “instant send” feature is a god send. I hope keyboard maetro will have the same feature one day.
ps: yes, double click any mouse button is not supported now. I see there is a setting “tapped twice” in device key. But mouse button can only use “pressed” setting. Maybe it is the limit.