Capture screen area, OCR, remove characters, insert by typing

Hi there:
I want to:

  • capture a screen area which starts with "Phone:XXX-XXX-XXX"; the location changes slightly from screen to screen
  • OCR captured area
  • Remove "Phone:" as well as any "-"
  • Type phone number with characters, spaces stripped out

I've been having success , with selecting an area, passing to Mac OS shortcuts for OCR, but struggling as the location changes of the area to capture as well as how to best strip away the other characters(non-numeric and spaces)

Any help is appreciated

Is the number always in the same place relative to the "Phone:" text? If so, instead of the first action above use a "Find Image on Screen" action and capture the results into a variable. You can then calculate the screen capture area for the second action using parts of that variable, eg theLocation.MidX + 100, theLocation.MidY - 50, etc.

That's a simple "match any character that isn't a number and replace it with nothing":

1 Like

Thank you for the fast response
Yes, the phone number is always preceded by "phone:"

I need help with this part....can't seem to find a way to do is much appreciated

The first action will place the area of the match into the variable named “Phone number”

You can then use that in the second action, simply start typing the variable name into the first field (replacing the 125). End up with something like:

Phone number.right
Phone number.height+4

These are just examples, the -2 and +4 are probably unnecessary, but they show that you can use a calculation in there if necessary.

The 200 is just a guess at the width required for the phone number, adjust as necessary.

Note that you can use the OCR Screen action directly. It's worth trying both and seeing which one gives better results as numbers are hard to OCR, and any inaccuracy will render the entire number useless.

As well as what @peternlewis said (and forgive me if I've gone too basic!). And trust me, this takes way way longer to read than actually do when creating your macro:

Screen coordinates are in {x,y} format with {0,0} being the top-left corner of your main screen -- x increases as you move right, y as you move down, and units are "nominal resolution pixels".

That means that if you know the position of two points you can calculate the distance between them -- or if you know one point and have a distance, you can calculate the second point's position.

Really poor diagram time...

So your "Find Image" action returns the position of your found image to a variable -- I'll use Local_myImage. While it actually gives you left and top coordinates plus the width and height of the image (and fuzziness), KM knows it is a "rectangle" data type and provides various built-in calculations for you (see the "Variable Dot Notation" section of the manual) so you can easily get useful values. For the diagram above:

{x,y} = {Local_myImage.Right,Local_myImage.Top}

So now you need to find example values for {x,y}, {a,b}, and {c,d} -- it doesn't matter where on your screen you do this because we are only concerned with the differences. If you do it manually it's a bit more work, but you'll have a method you can use in almost any situation:

  1. Get the thing you want to "measure" on screen
  2. Press ⌘⇧4 to activate the OS's "screen grab a selection", which will show a "target" pointer complete with coordinates
  3. Hover the pointer over the approximate position of the top-right corner of the image you are detecting and write down the coordinates shown -- that's our {x,y}
  4. Hover the pointer over the approximate position of the top-left corner of the area you want to grab and write down the coordinates shown -- that's our {a,b}
  5. Hover the pointer over the approximate position of the bottom-right corner of the area you want to grab and write down the coordinates shown -- that's our {c,d}
  6. Hit the esc key to cancel the screen grab

And now it's maths time -- I'll use my on-screen numbers for the image above:

  1. a - x is the x-offset for our grab area's top-left corner -- for me, 1038 - 1023 = 15
  2. b - y is the y-offset for our grab area's top-left corner -- for me, 841 - 841 = 0
  3. c - a is the width of our grab area -- for me, 1425 - 1038 = 387
  4. d - b is the height of our grab area -- for me, 917 - 841 = 76

..and we now have the numbers we need for the "Screen Capture Area" action. In dialog order (and again using my numbers):

  1. Right edge is Local_myImage.Right + 15
  2. Top edge is Local_myImage.Top + 0
  3. Width is 387
  4. Height is 76

Remembering that the area we want to grab is always the same size and always the same offset from our found image, it's simple to code this into our macro:

As I said at the top, it will have taken you much longer to read all this guff than it will actually take you do this when writing your macro. Try it, and let us know how you get on. And if I've confused more than helped (not unusual :frowning: ) -- just ask what the &^%$ I'm going on about!


I so appreciate the very detailed response, not at all basic for this newbie. Will work on this.