How do I percent encode a JSON string on my clipboard?

Questions & Suggestions
1

I am trying to encode a JSON string on the clipboard using KM's "filter clipboard with percent encode for URL" action . However, the result obtained is different from what I expect using other tools including Popclip and URL Decoder/Encoder.

When given the following string to encode:

{"version":0.8,"agent":"Chrome2Maxel","urls":[{"href":"http://plugin.fileopen.com/installcomplete/installcomplete.pdf","title":"your browser"}]}

I expect

%7B%22version%22%3A0.8%2C%22agent%22%3A%22Chrome2Maxel%22%2C%22urls%22%3A%5B%7B%22href%22%3A%22http%3A%2F%2Fplugin.fileopen.com%2Finstallcomplete%2Finstallcomplete.pdf%22%2C%22title%22%3A%22your%20browser%22%7D%5D%7D

Instead, using KM, I get

%7B%22version%22:0.8,%22agent%22:%22Chrome2Maxel%22,%22urls%22:%5B%7B%22href%22:%22http://plugin.fileopen.com/installcomplete/installcomplete.pdf%22,%22title%22:%22your%20browser%22%7D%5D%7D

What am I doing wrong? Thanks in advance.

2

Q: Which characters to percent-encode, and which to leave as is ?

Each context and purpose needs a different pattern of encode vs stet.

To get, for example, from a unix path which includes spaces:

"/Users/houthakker/Desktop/name includes spaces.txt"

to a URL which a browser can use:

file:///Users/houthakker/Desktop/name%20includes%20spaces.txt

I do need a (space → %20) translation, but if the characters of file:/// also get percent-encoded, then my browser will choke and I have a problem.

In JavaScript, for another example, the most common cases are covered by having two different percent-encoding functions in the library:

  • encodeURI() just the characters which need to be percent-encoded for a URI, leaving, for example the protocol string at the start, and
  • encodeURIComponent() which can be a bit more ambitious, and doesn’t have to watch out for a protocol string at the start.

In other words, there are many different points on the spectrum between percent-encoding nothing, and percent-encoding every single character. Different points on that spectrum are required for different purposes.

Perhaps one could rewrite your question from:

what am I doing wrong ? (Answer: quite probably nothing)

to:

what is the context in which you need the percent-encoding, and what, if anything, is the error that you are getting ?

3

JSON, incidentally has quite a small number of special characters which need to be escaped.

Diagram at http://json.org/string.gif

The simplest approach (depending on your context) may be to create a Javascript object, and then just call the JSON.stringify() function on it.

4

What I simply want to do is to create a macro to add a clean link (devoid of referrer information) to a download manager, Maxel using this [API] (http://maxelapp.com/api.html).
Figuring out how to do an escape url for all characters before prepending “maxel://add/” is what I am finding difficult.

5

OK, so it's essentially a URL-encoding issue rather than a question of encoding/escaping for JSON itself.

KM's encode for URL is working in the same way as the standard JavaScript encodeURI(), and protecting protocol strings.

If that is proving indigestible to the Maxel server, then I would try:

  • applying JavaScript's other library function, encodeURIComponent(), to the JSON alone, and then prepend the maxel://add/

For example, perhaps with something like this:

Maxel URL.kmmacros (2.6 KB)

(function (strProtocol) {
	var a = Application.currentApplication(),
		sa = (a.includeStandardAdditions = true && a);

	return strProtocol + encodeURIComponent(
		sa.theClipboard({
			as: 'string'
		})
	);

})('maxel://add/');
6

Probably not relevant but just in case – I notice that Maxel’s example (from which I pasted) includes JavaScript quotes (beginning with //)

People often do that, quite naturally, on the very reasonable assumption that JSON and JavaScript are essentially the same thing. One of the few differences, however, is that JSON does not formally allow for using quotes in that way, and it could be, if you have left any in place from their quoted example, that they are choking a JSON parser on Maxel’s server.

( If so, might be worth pointing that out to them – surprising how often this comes up … )

7

Thanks. Was able to get it to work after some minor tweaking.

8

It would appear that Keyboard Maestro's percent-encode filter is broken in 7.0 and will be in 7.0.1.

It came up on another thread, and Peter is looking into it.

-ccs

9

No, this is not accurate. I don't believe the behaviour has changed from previous versions of Keyboard Maestro. It just does not seem to encode much of anything, but I'm not sure why. It is possible this is a system change in recent versions of OS X as the filter calls straight through to the stringByAddingPercentEscapesUsingEncoding API, but maybe that doesn't do much of anything either/

I will look in to it though.

10

There's a discussion at:

https://developer.apple.com/library/mac/documentation/CoreFoundation/Reference/CFURLRef/index.html#//apple_ref/c/func/CFURLCreateStringByAddingPercentEscapes

It may be difficult to use this function to "clean up" unescaped or partially escaped URL strings where sequences are unpredictable and you cannot specify charactersToLeaveUnescaped. Instead, you can "pre-process" a URL string using CFURLCreateStringByReplacingPercentEscapesUsingEncoding then add the escape characters using CFURLCreateStringByAddingPercentEscapes

It doesn't actually look as if anything is broken to me, and I think it's producing the same output as:

i.e. only encoding enough to make a simply-formed URI compliant.

But perhaps there is an argument for providing an additional sister function which doesn't protect protocol substrings etc, and may possibly be closer to what users expect, in some contexts.

Analogous, in other words to:

?

( First draft of a custom action at Filter system clipboard to percent-encode a URI component )

(And a set of 4 actions simply wrapping the 4 (encode/unencode) * (uri/uriComponent) JavaScript functions, and allowing for input from variables, text, tokens etc as well as from the clipboard, at: