That's a fair and reasonable question. I'm fairly sure KM doesn't have an existing built in condition that can deactivate the macro when macOS is in some text input mode, so the solution would probably have to trigger on every text key, (which is possible) and then the solution would check to see if macOS is in text input mode, and if it is, let the keys be sent to macOS, but if it is not, perform some action.
Moreover I suspect you still want function keys (ie, anything key that isn't a text key) to continue to trigger macros whether macOS is in text input mode or not, right?
I suspect that AppleScript is the most likely way to solve this, which means I'm not able to help.
I just googled "keyboard maestro determine if text input mode" and found this thread, which seems to have a solution.
I don't think that thread directly answers your question, but it seems to indicate a way of determining whether macOS is in text input mode, and someone could combine that with my idea in my first paragraph to get you a solution.
I like your idea, and the challenge of solving this. I'll read the web page I cited above and see if I can get it to work, but it involves AppleScript and I may not be above to solve it therefore.
Also, the thread indicated that the solution may have to work differently for different apps, so it would really help if you indicate what the app(s) is/are you want to get it to work with.