Crop images: Eliminate surrounding white space

Hello everyone,

I have a workflow where we are scanning a page with a designated area where the employee has to draw a small drawing (some lines and text) which creates a white rectangle with a drawing somewhere in the middle.

Now, I am thinking if we can use KM to remove the superfluous white space around the sketch.

At the moment we are cropping each drawing manually, and depending on the amount it can become a lot of work.

Any idea how to tell KM where the drawing is, and remove the surrounding white area? Just imagine a blank page with a small sketch on it. How do you tell KM (if at all) to cut out the drawing itself and ignore the white area around it? Any ideas are welcome!

I can probably think of three different approaches.

  1. One is very complex but reliable and fast (but requires free third party software)
  2. One is possibly unreliable but simple (using simply KM macros to calculate)
  3. Don't even ask about the third one (which uses Apple's spreadsheet to find the limits)

Let's discuss the simple (#2) approach. KM has a condition called Pixel which lets you find the value (eg, brightness) of a pixel. If you (or we) write some code that scans across the page until it hits a certain brightness/darkness, that might be where the sketch starts (top, bottom, left or right). If you save these values you can crop the image using the Crop Image action. One problem here is that a single stray dot on the page may either trigger an overly large crop or, if missed, an overly small crop. Are these sketches done with a dark marker or a light pencil? The code here will be fairly simple but will consume a lot of CPU and you may have to wait for it. However waiting isn't a problem if you let this code run overnight, for example.

There's an approach called a binary search which finds results faster by dissecting the problem. I think such an approach would help out here, since this process will be CPU bound.

Basically the solution will be running this test thousands of times in order to find the boundaries of the diagram.

image

Let me know if one of these approaches interests you.

1 Like

I just thought of a fourth approach that may be fairly fast and simple. Let me run some tests.

1 Like

Here's an image to illustrate my crazy new approach. First, I draw thick gridlines on (a copy of) the original diagram. In my case the original diagram is a bunch of squiggles near the middle of the page.

Then I use the Find Image action looking for a white square with a black outline. These would represent the unused ares of the page.

However the Find Image action has always been finicky when it comes to solid colours. What I did was have it colour in black all the squares that it found that were white (so that you could see which white squares it successfully found.) And you can see that roughly half of the blank area of the page was left undetected by the Find Image action.

This was a valiant attempt. But the Find Image action has never worked well with solid colours. In this case I thought it would work because the outer edge of the item I was searching for was a black square. But that still gives a 50% failure rate.

The core of my code contained this statement:

image

Had it worked, I think I could have easily calculated the required crop area, but since it didn't work I don't have to write that code yet.

Let me ponder this approach a little more. I might be able to trick that intransigent Find Image utility into working. If I can trick this little gremlin into working, I think you will have a fast and easy solution.

1 Like

Unfortunately, Keyboard Maestro’s Trim Image action will only trim off transparent areas, not white areas (also it seems to reduce the image if its not 72dpi, which is a bug, but lets ignore that for now).

I expect you'll have to use some external software. I would be very surprised if ImageMagick can't do it, and these days it is relatively easy to install with Homebrew.

1 Like

I seem to have found a way to trick FindImage into working! I'm so happy. But I'm also so tired, and sick today, so it's time to pack it in.

ImageMagick was my solution #1, but I don't want to go there if I can do it in KM, and I've convinced myself that I can do it in KM with little more than the Find Image action. I believe you will call me clever one more time when you see it working. Basically I use FindImage to find all the blank squares, then I sort the squares by their location and use a shell utility to calculate the cropping area.

I like puzzles, and this was a good puzzle.

1 Like

Sleepy, you are crazy. I am flabbergasted!

The sketches are usually drawn with a ball pen, so contrast should not be an issue, there is plenty.

I also had the idea of searching white squares around the sketch, but I had no idea where to go next using this approach.

Thinking about it, it may be an idea to "rasterize" it (create a map of the image) using "find white square" and save the results as booleans on a checkerboard using coordinates like in the "Battleship" game - A1 true, A2, true, C4 false etc.

Then you can search for the FIRST occurrence of "false" from every direction (if the sketch is a skewed rectangle for example) in order to obtain the outer limit of the sketch.

For example:
First "false" result from the top: F3
First "false" result from the left: B5
First "false" result from the right: I7
First "false" result from the bottom: D13
Those coordinates could be used to set the crop coordinates accordingly.

Just some food for thought... and I enjoy this puzzle.

Also, I am still open to other suggestions :wink:

Hey Chris,

I'd be looking at running ImageMagick from a Execute a Shell Script action.

One or more folks on the forum use it pretty extensively, but I forget who and don't have time to look at the moment.

@Tom?
@mrpasini?

In any case – it's likely to be able to do the job in the most flexible way.

-Chris

The -trim option (optionally with -fuzz) would do it. See this discussion for the details, @Splitty .

You may want to add a white border if that crops a bit too much off for you.

1 Like

Yes of course ImageMagick has many advantages. But it's always nice to solve problems with KM. And ImageMagick isn't always fast. I've issued some simple commands that caused ImageMagick too require five minutes to resolve on a screen image. If I can't get the code uploaded here tonight I'll try my best for Saturday.

That was my idea #3 which I told you not to ask about! Once I managed to rasterize an image into the cells of a spreadsheet and then got Apple Numbers to do some image processing on the resulting spreadsheet. I haven't mastered that technique yet, but it has a lot of potential.

I did get my code working yesterday, but parts of it, like getting the crop dimensions, I was saving for later. I'll see if I can get that working now. If not now, I'll try tomorrow.

Just a minor setback, but I just noticed that the Find Image action, while now supporting "files" for the images that you want to "seek," does not support either files or clipboards for the big image you want to search within. I had assumed that in this upgrade it would support "files" for both images, but it seems to support files for only one of those. That just means I have to keep the big image on screen rather than do it all in memory. Not a serious problem, but now I have to write some extra code. The extra code is required to scale down the image into something that will fit on the screen (and to search only the necessary area.)

However while pondering that issue I have uncovered a totally new algorithm to solve this problem. It still uses the Find Image tool, but does not use it to find blank areas. Both of my algorithms seem like they should be quite efficient, but the new one I've just come up with feels much more clean and logical (and requires less code). I hope nobody beats me to the new solution, but if they do, I'll just assert that I said so much that it gave people ideas.

Since I'm so close to getting the first one finished, I'll keep on that for now. But if I have any problems I will switch to the new variant.

If anyone knows a way to use Find Image to search a clipboard or file, please let me know.

BREAKING NEWS:

I just discovered a moment ago, Peter, that there's a builtin feature of Apple's Preview app that converts external white space in an image to the transparent colour. Combine that with your Trim action, and the solution can be performed in just a couple of KM statements!

The Apple Preview app has a little magic wand icon (called Instant Alpha, but doesn't seem to appear in the Preview menu system) that converts white space (or whatever colour region you click on) to the transparent colour. That's the hard work required to solve this problem.

So a simple KM macro could be written to do this:

  1. copy the image needing trimming to the Preview app; (using probably using CMD-N)
  2. click on the magic want then click on a white pixel near the edge of the image (or just assume the upper left pixel is white), then hit the Delete key.
  3. copy the new transparent image from preview to the clipboard (CMD-C)
  4. use the Trim action in KM to trim the transparent colour
  5. Save the clipboard image to a new file

How can you get a more perfect solution that this? I deserve some sleep now. That's the germ of the solution, and it should be easy for anyone to write a five action macro to implement this.

Nevertheless all that hard work was still fun and rewarding.