So far the only way I can get KBM to access submenus on pop up preference or command windows in complex applications like DAWs (Digital Performer in this case), is to use the image search or mouse click at position macros, both of which in an application like a DAW that can scale and change the theme of it's GUI are not ideal.
Basically DAWs have so many preferences and commands that the top menu bar for the application itself is only half the battle, every page or window of the application can have specific commands that are not assignable internally to keyboard shortcuts, yet should be. KBM is a good choice, but so far the only solutions are image search, which will break if you change the GUI Theme, and Click and Hold at... which will break if you scale the GUI to a new setting.
I've been trying with no luck to get KBM to somehow recognize the pop up window in front with the submenu that houses specific commands for that window, but so far the solutions are to use the mentioend macros.
When asking questions like this you really need to provide more specific context.
Specifically what app are you working with?
Screenshots of what you're working with.
Etcetera.
Generic questions are usually too broad to answer well.
Sometimes other users will have your app and be able to answer based on experience – if they know what the app is that is...
Seeing an actual example of the problem gives us a better probability of giving good answers.
As Peter says – AppleScript and/or AppleScript UI-Scripting is a possibility. Many apps have no AppleScript dictionary, but if they have proper accessibility design they can often be manipulated with UI-Scripting.
The only other options I can think of at the moment are found images as you mentioned and possibly type-select in menu structures.
I know this is very frustrating, but developers do what they do...
Don't be bashful about asking them for better automation capabilities though.