Repairing an invalid JSON file before processing

Questions & Suggestions
1

Hello dear everyone,

one customer occasionally sends invalid JSON files for some reason, so I am trying to use KM in order to fix the missing data, then continue to calculate the geographic center of some given GPS coordinates.

An anonymised sample JSON file is here: https://www.splitmind.de/BrokenJSON.json.zip

First step in the macro to read the file, then determine the text encoding (usually it's iso-8859-1), then convert it with a shell script to UTF-8 so stuff works okay on the Mac when dealing with text and umlauts. Then, I want to fix the coordinates as follows:

There are two sets of coordinates submitted:

The first is the "Einzelpunkt" (single point) which the customer fails to provide in the sample, but a correctly formatted JSON file would expect a decimal number there, north and east coordinates. This is why the JSON is not properly formatted and can't be processed by Keyboard Maestro the way it is.

The second is "KoordinatenFlaeche", an array of GPS coordinates, north and east, to mark an area on the map. Those coordinates are always provided.

I plan to calculate the middle point of the given array coordinates and fill it into the Einzelpunkt coordinate. However, in order to read and calculate this correctly, Keyboard Maestro first has to parse it as properly formatted JSON.

At first I tried to do a simple Search & Replace. It does work with ""Nord" : ," which can only appear if the file is broken. However, ""Ost" : " has no comma at the end and appears twice in the data, so if I just want to replace it with ""Ost" : 8.0000", it will also do the same to the array, and then the JSON is just broken again!

So I thought I could use some RegEx to determine that I am changing the correct occurrence of "Ost", but that turned out harder than I thought.

Any ideas and input are very welcome!

--

JSON Repair Macro (v11.0.3)

JSON Repair.kmmacros (21 KB)

Keyboard Maestro Export
Keyboard Maestro Export2222×7102 1.1 MB

2

Have you tried setting the Search and Replace to "First Match" (in the action's settings). It may not be an entirely "proper" fix, but it should work if you just do a string match and include the leading spaces, the : and the trailing space:

			"Ost" :

You could do the same for "Nord", rather than mess with a regex.

1 Like
3

Sounds like a problem that would be better fixed upstream ...

  • They don't seem to be encoding it as UTF8
  • Looks as if they might be breaking things with some manual post editing

(The JSON linting in a free standard tool like Visual Studio Code should help them to improve their process, and spot the glitches before shipping something leaky)

1 Like
4

I'd agree, but the developer of the customer's software is of the "too much work/won't fix" type. This is a problem for years now, and I decided to fix it myself...

DANG! I knew I had read that somewhere, but couldn't find it at all!
It's probably not perfect, but for this use case it will do. I'll take it!

Thanks a ton!