Base64 String Decoding/Formatting Issues

Task: given a base64 encoded string, decode and display it via keyboard shortcut.

Given test string, set to variable b64inputString via earlier macro step:
UwB0AGEAcgB0ACAAIgBgACIAaAB0AHQAcAA6AC8ALwBsAG8AYwBhAGwAaABvAHMAdAA6ADMAMAAwADAAYAAiACIA

the decoded text should be: Start "`"http://localhost:3000`""

Methods tried:

  1. run shell script using bash, display results in a window:
    /usr/bin/base64 -D <<< $KMVAR_b64inputString
    result: S^@t^@a^@r^@t^@ etc etc

Copying this from the window results in the unprintable characters being changed, but I really need a display without all of this garbage in it. Let's try large:

  1. run shell script using bash, display results large:
    /usr/bin/base64 -D <<< $KMVAR_b64inputString
    result: one (very) large S

  2. hmmm. let's try in sh and bash directly:
    bash3.2$ /usr/bin/base64 -D <<< UwB0AGEAcgB0ACAAIgBgACIAaAB0AHQAcAA6AC8ALwBsAG8AYwBhAGwAaABvAHMAdAA6ADMAMAAwADAAYAAiACIA
    result: Start "`"http://localhost:3000`""

hrm, okay, so something is maybe happening with how KM is handing the characters. This definitely isn't "usual" base64, but base64 -D is still able to handle it... let's try python!

  1. run shell script using python3.9, display results in window:
    /usr/local/bin/python3 -c 'import base64; import os; INPUT = os.environ.get("KMVAR_b64inputString"); print (base64.b64decode(INPUT).decode("ascii"))'
    result: S^@t^@a^@r^@t^@ etc etc

  2. okay, how about shell script, python3.9, display results large:
    result: see result 2 (one very large 'S')

  3. sanity check, in command line:
    bash3.2$: /usr/local/bin/python3 -c 'import base64; import os; INPUT = os.environ.get("KMVAR_b64inputString"); print (base64.b64decode(INPUT).decode("ascii"))'
    result: Start "`"http://localhost:3000`""

  4. ????????????

  5. Tear out hair.

What am I doing wrong, what am I missing? (UTF-8 vs ASCII decode doesn't matter in python either, fwiw). Setting the system clipboard to the output just results in S. Is this just weirdness in KM's text handling? am I returning the wrong data? Not properly formatting it in the script somehow?

Thanks in advance!

Edit, macros:
Text Manipulation Macros.kmmacros (8.7 KB)

Follow up, for anyone having future trouble like this: disabling the "Trim Results" text results in a semi-working, expected outcome, but only if I "display results large", and only in python. Saving to a variable so I can also copy it to a clipboard for later use, and then trying to display large, doesn't work. (results in 'S').

I am vastly curious as to what "Trimming" is doing such that it changes this result - is it the nullchar/string strangeness that's tripping it up? is it the ` characters in the returned string? and why doesn't it work in bash? (and, even with "Trim Errors" set to no, the macros all still result in the weird S^@t^@a^@ string if displayed in a window.)

WHILE 7
GOTO 8

Hey @Aktariel,

As far as I can see your original string has NULL character every other character.

Looking at things in BBEdit with invisibles turned on:

Copying output directly from the Terminal to the pasteboard (Clipboard) gives the same result:

echo UwB0AGEAcgB0ACAAIgBgACIAaAB0AHQAcAA6AC8ALwBsAG8AYwBhAGwAaABvAHMAdAA6ADMAMAAwADAAYAAiACIA | base64 -D | pbcopy

Redirecting to a file gives the same output:

echo UwB0AGEAcgB0ACAAIgBgACIAaAB0AHQAcAA6AC8ALwBsAG8AYwBhAGwAaABvAHMAdAA6ADMAMAAwADAAYAAiACIA | base64 -D > ~/Downloads/test.txt

So, let's go back to the BBEdit image above and look at the second command. With this we take your expected output and re-encode it to Base64.

Notice how different the two encoded strings are.

Here's one way to get rid of the NULLs:

#!/usr/bin/env bash

echo UwB0AGEAcgB0ACAAIgBgACIAaAB0AHQAcAA6AC8ALwBsAG8AYwBhAGwAaABvAHMAdAA6ADMAMAAwADAAYAAiACIA | base64 -D | tr -d '\000'

-Chris

Oh yeah, what is the original source of your encoded string?

-ccs

Something 16-bit-per-character?

The original source is something that appears to be 16-bits per character, yes. Ideally I'm trying to build a macro that will take both 8- & 16-bit base64-encoded strings, and return a nicely formatted readable/usable display & clipboard, for further examination. (I don't have other examples to hand, but they do exist, and I could probably generate some if needed.)

I'll give the tr a try; any suggestions for python?

Thanks. :slight_smile:

I did a quick Google search and came up with this:

It looks promising.

-Chris

More than two years later, here was my final solution (thanks to Chris)

the code is echo $KMVAR_b64inputString | base64 -D | tr -d '\000'