How to Send Sysex or Obtain More Midi Values With KM?

Hello,

I send midi with KM to avoid having an external midi controller and have more flexible mapping. However, the program I use, Max for Live, has a limitation on midi channels (I can only use 1) and CC values. Also the 127 midi limit is a historical hardware limitation, I don't see why virtual messages couldn't be unlimited. I very quickly eat up all those values between playing actual midi notes and mapping midi values to buttons.

I'm asking for advice if anyone has thought about this before. Is there a way to shoot out more messages like midi that could be supported in DAWs? I don't understand how to obtain and format sysex messages, but I think that might be simpler than OSC. Couldn't someone just create arbitrary sysex messages to obtain unlimited options? Any opinions are welcome, thank you!

This website helped me understand Sysex: Eddie's Home - MIDI - SysEx Tutorial

In fact, it's much easier than I thought to create your own custom sysex messages. Here's an example of a raw packet (it's that simple!):
F0 00 F7
here's another one:
F0 00 01 F7

As long as it starts and ends properly, the message is detected in Max for Live and can be used in JS, so it solves my problem completely. Way to go KM for allowing this amazing functionality!

1 Like

Sorry!

How did you implement SysEx text formation in KM?

You can send SysEx messages in a similar format like the one I mentioned by using the "Send MIDI Packet" Action, under MIDI.

Let me know if that isn't clear!

I tried that method, but KM splits SysEx message into 3 parts:
for example: F0 00 00 66 10 12 90 64 7F F7

12:39:43.200	From Keyboard Maestro	SysEx		Loud Technologies / Mackie $6 bytes	F0 00 00 66 10 12
12:39:43.200	From Keyboard Maestro	Note On	1	$64	$7F
12:39:43.200	From Keyboard Maestro	Invalid		$1 bytes

The last invalid message byte is F7 - EOX

Thus the second message is intercepted by LPX and everything else is discarded as unreachable.

Where are you getting this midi packet from?

It seems like this packet is sending three messages correctly. I don't think the problem is with KM, but with the midi messages, which includes those three that you're seeing.

If I delete part of that message and use midi monitor I'm able to get a single SysEx, but I'm seeing that with that full message you're sending three distinct messages.

1 Like

The example sysex message you give @Vladistone is invalid according to the midi standard specification. As you can see here, every data byte must have its MSB set to 0, so a value of 90H is invalid.

2 Likes

Ah, so you're saying we're sending a "mixed message"? 0x9n is "note on for channel n", but when that's inside a SysEx it's a "Status Byte" (MSB is 1) and so considered an EOX.

Interestingly that "considered EOX" isn't dropped by MIDI Monitor but is processed:

image

image

...so treating the packet as two commands

F0 00 00 66 10 12
/implicit EOX/
90 65 7F

..which also explains why post-fixing the KM raw packet with an explicit EOX, as @Vladistone was, throws an "invalid" for the final F7.

Interesting...

1 Like

Yes I believe that’s all correct.

KM is transmitting the full set of bytes but it’s Midi Monitor that’s trying to make sense of those bytes and displaying its interpretation to us.

1 Like

Let's say that the KM processes and sends a full SysEx data packet, thus defining byte 90 as EOX. nor does the expected act achieve its purpose. then how to achieve the desired "Note on" == 90 64 7F to be sent to header address?:

F0 00 00 66 10 12

Note:
in case using another MIDI tool: Pocket MIDI with SysEx window:

  • I send exactly the same message and get the result I want without problems with the theoretic items provided by @tiffle

Keyboard Maestro knows nothing about SysEx, it only knowns about bytes and packets.

Since we live in the glorious age of AI, I asked ChatGPT the question:

In MIDI, in a System Exclusive Message, how do you include bytes with the high bit set?

and it responded with a long description of using byte stuffing. It then produces a whole bunch of well meaning but completely bogus text, including things like saying that 0x12 + 0x80 is 0x32, and that 0x32 has the high bit set. So it was not entirely helpful.

So my guess is there is a way to specify that a byte inside a sysex is a data byte even though it has the high bit set. But I could not find any documentation on this.

Further, my guess is that Pocket MIDI knows about that, knows you are sending a SysEx, and is doing that for you. So you could try sending that SysEx packet to Keyboard Maestro, have it detect it as a raw MIDI packet and tell you what bytes it actually receives. Since Keyboard Maestro does not interpret SysEx going in or out, hopefully that will get you the real raw bytes that are actually sent in the MIDI packet.

1 Like

I did it:
Sysex receive

Pocket doesn't see open KM MIDI port... I don't know how to recognize KM MIDI input...
while PocketMIDI itself is also not defined
Could you help, please
output

As far as I understand it, you use Audio MIDI Setup to control the routing of packets.

Your trigger in the macro will not match the MIDI packet if my theory is correct that there is some byte mangling going on to deal with the high bit set characters.

It would be interesting to find out - who is the culprit of this behavior?
my doubts can only be in the direction of DANTE driver or ipMDI for use the SSL Nucleus 2 controller



sendMIDI list

New user here: gacki from the logicprohelp forums. This discussion has partly going on over there as well.

Not really, no. In theory all MIDI ports should be visible in all MIDI applications.
The "Keyboard Maestro" port is indeed visible in MIDI Monitor, Mainstage and so on. Sending data to those applications works fine. It does not show up in Audio MIDI Setup which is somewhat weird.

(I think I've addressed this over at logicprohelp; this is merely some additional clarification.)

Having the high bit set within a Sysex dump is a complete no-no. This will automatically terminate the Sysex dump. There seems to be the question where the additional F7 then comes from. Here are my thoughts:
MIDI Monitor and Logic will correctly display the shortened dump (with the additional F7); Pocket MIDI will not display anything (but otherwise will show properly formatted SysEx just fine). This seems to hint (for me) at the possibility that MIDI Monitor and Logic add this F7 on their own (as they should!). The other possibility would be that CoreMIDI manipulates the data already on the system level - but then I would expect Pocket MIDI to display this (originally malformed) Sysex dump as well.

Yes, 0x90 (or any other status byte) is an illegal character within a Sysex dump which will automatically terminate the dump at this point and result in the following bytes being interpreted as belonging to that new status byte. There are various ways to encode complete data bytes into 7 bit clean SysEx bytes (nibbles, 7+1 and so on) but this is IMO not Keyboard Maestro's problem (and also was not Vladistone's problem).
Likewise it would be nice to have some kind of "sanity check" for such SysEx dumps but again I don't think this has any priority.

What indeed would be nice would be the possibility to route data specifically to specific MIDI ports from within Keyboard Maestro. Right now everything seems to go to the "Keyboard Maestro" MIDI port by its own. Getting the list of available ports from macOS should be comparatively easy (I think).

1 Like

Hi @gacki! You voiced the problem of nibbles 7 + 1
I also started looking for solutions and mentions of this in on-line
am i looking in the right direction

I've dealt with Pocket MIDI where MIDI IN/OUT destination ports are forced and so it easily manages to send the "wrong" Sysex sequestered down to 90 64 7F which leads to the expected result - LCD switch on

Nuances can arise with virtual MIDI ports only:
not visible in apple MIDI setup

  • no ipMIDI ports
  • no other virtual ports (such as KM, MIDI pipe, Pocket MIDI)
    and if PM recognizes KM MIDI in, then doesn't see KM MIDI out

Yes, of course it does - the complete Sysex part is ignored and only the "Note On" remains, that's why.