ISSUE: OCR on KM10 + Mac OS 15

Questions & Suggestions
1

I upgraded to Mac Osx 15 and OCR stopped working. I have Keyboard Maestro 10. I've enabled all settings (given all permissions) I can think of in Privacy and Security:
-Accessibility
-App Management
-Automation
-Input monitoring
-Local Network
-Screen and System Audio Recording

  1. Is there anything else I need to enable to OCR works?
  2. If not, will there be a fix for KM10 on Mac OS 15 or should I buy KM 11?
  3. Do we know if OCR works on KM11 in OSX 15?

Thanks in advance!
-Scott
(long time lover of KM, first time posting)

2

I presume Peter is starting to look into macOS 15, but there have been no announcements. There's nothing wrong with your question, and I'm pretty sure Peter will try to make OCR work on macOS 15, and I predict he will succeed. There is usually a period of time after an OS is released before KM is updated to support it, and macOS 15 has not been publicly released yet. So you're not likely to get an answer so early.

3

Same question. I tried to delete and re-enable permissions in accessibility and restart, did not seem to fix it.

4

It doesn't work in KM 11.0.3, either. A very simple test macro...

image
image1008Ă—804 89.7 KB

...does noting. The log file indicates that the screen capture failed, despite screen capture permissions being granted:

2024-09-29 06:31:55 HScreenArea::CaptureCGImage failed to capture front window
2024-09-29 06:31:55 Action 16015608 failed: OCR Image cannot get target
2024-09-29 06:31:55 OCR Image cannot get target. Macro “OCR Test” cancelled (while executing OCR Front Window).

Hopefully @peternlewis can figure out the problem ... but as it seems to be Accessibility API related, it might be a real mess :(.

-rob.

5

It is working for me on two of my Macs (MacBook 14" M1 and Mac Pro Intel)
Mac OS sequoia 15.0
Keyboard Maestro 11.0.3

image
image1040Ă—834 90.3 KB

I tried this exact one and it works for me. So, looks like it might be some sort of permissions issue.

6

How weird. My Sequoia Mac is quite basic, new only a few months ago, and I've confirmed screen recording permissions are granted. (And other apps I use that require screen recording are working.) I'll try some more troubleshooting tomorrow.

-rob.

1 Like
7

I had no issues using Apple Text Recognition in Sequoia.

However the Tesseract text recognition was failing in my tests with a memory corruption issue - whether this indicates a bug in Tesseract or a bug in Sequoia, I don't know.

The first step I would try is ensuring that Keyboard Maestro can screen capture the desired area using the Screen Capture action with the same settings as the OCR action. If it can't, then you know the issue is with Screen Capture permissions or Accessibility permissions.

If it can, and the Apple Text Recognition fails, then that would be hard to understand.

OCR Image cannot get target error implies that Keyboard Maestro could not capture the target image (ie, this happens before it tries to OCR).

1 Like
8

I got it working by manually adding the KM Engine to Screen recording permissions—the app was listed there, but not the engine.

-rob.

4 Likes
9

Works properly with Apple Text Recognition, but only in english. Spits out gibberish for other languages. Strange because if I take a screenshot of the text, it identifies the text and allows copying perfectly out of preview so Apple's OCR does recognize the language well

1 Like
10

Good to know, I'll see if I can enhance the action in the future (especially important if I can't get Tesseract working in Sequoia).

11

You can make use of Keyboard Maestro's Execute Shortcut Action to access Apple OCR and Apple Translate from within your KM Macros. I do not know what your Macro is doing or looks like but the below should give you the idea. In my testing it worked fine to translate from Polish to English (MacOS Sequoia 15.01 and Keyboard Maestro 11.0.3)

You can edit the Apple Shortcut to do what you want in terms of language to translate from and to and whether you want it to do a screenshot (as I did) or to get the text from an image.

Of course, you don't have to have it translate - it can return the recognised text in the original Polish (by simply deleting the last step in the Apple Shortcut) if that is what you are wanting to do.

OCR Translate from Polish to English.kmmacros (2.6 KB)

OCR Translate from Polish to English
OCR Translate from Polish to English992Ă—1002 133 KB

This Macro calls this Apple Shortcut:

image
image1402Ă—442 56.6 KB

Here is a link to download the actual Apple Shortcut:
https://www.icloud.com/shortcuts/ba46a2c2f9474da9bc88d7329403ed3a

5 Likes
12

Brilliant!

1 Like
13

Love it! Thanks for sharing

1 Like
14

Good idea, but the problem is that OCR does not convert the image to Polish. The OCR result does not have Polish characters.

15

Ah - I see what you mean. The interesting thing is that it does recognise the text as Polish and can save it and can translate it to English, but if it is left as Polish it does not include some of the special characters in the text output.

16

I solved this problem. Look at: OCR problems - #11 by Kinjal

1 Like
17

Thanks, this helps a lot. The old pre-Sequoia OCR macro had also stopped working for me and getting Keyboard Maestro to trigger OCR via Apple Shortcut works now.

2 Likes