ISSUE: OCR on KM10 + Mac OS 15

Scott_Conover · July 16, 2024, 3:17am

I upgraded to Mac Osx 15 and OCR stopped working. I have Keyboard Maestro 10. I've enabled all settings (given all permissions) I can think of in Privacy and Security:
-Accessibility
-App Management
-Automation
-Input monitoring
-Local Network
-Screen and System Audio Recording

Is there anything else I need to enable to OCR works?
If not, will there be a fix for KM10 on Mac OS 15 or should I buy KM 11?
Do we know if OCR works on KM11 in OSX 15?

Thanks in advance!
-Scott
(long time lover of KM, first time posting)

Airy · July 16, 2024, 5:52am

I presume Peter is starting to look into macOS 15, but there have been no announcements. There's nothing wrong with your question, and I'm pretty sure Peter will try to make OCR work on macOS 15, and I predict he will succeed. There is usually a period of time after an OS is released before KM is updated to support it, and macOS 15 has not been publicly released yet. So you're not likely to get an answer so early.

linnic · September 29, 2024, 9:44am

Same question. I tried to delete and re-enable permissions in accessibility and restart, did not seem to fix it.

griffman · September 29, 2024, 1:34pm

It doesn't work in KM 11.0.3, either. A very simple test macro...

...does noting. The log file indicates that the screen capture failed, despite screen capture permissions being granted:

2024-09-29 06:31:55 HScreenArea::CaptureCGImage failed to capture front window
2024-09-29 06:31:55 Action 16015608 failed: OCR Image cannot get target
2024-09-29 06:31:55 OCR Image cannot get target. Macro “OCR Test” cancelled (while executing OCR Front Window).

Hopefully @peternlewis can figure out the problem ... but as it seems to be Accessibility API related, it might be a real mess :(.

-rob.

Zabobon · September 29, 2024, 7:08pm

It is working for me on two of my Macs (MacBook 14" M1 and Mac Pro Intel)
Mac OS sequoia 15.0
Keyboard Maestro 11.0.3

I tried this exact one and it works for me. So, looks like it might be some sort of permissions issue.

griffman · September 30, 2024, 4:08am

How weird. My Sequoia Mac is quite basic, new only a few months ago, and I've confirmed screen recording permissions are granted. (And other apps I use that require screen recording are working.) I'll try some more troubleshooting tomorrow.

-rob.

peternlewis · September 30, 2024, 5:06am

I had no issues using Apple Text Recognition in Sequoia.

However the Tesseract text recognition was failing in my tests with a memory corruption issue - whether this indicates a bug in Tesseract or a bug in Sequoia, I don't know.

The first step I would try is ensuring that Keyboard Maestro can screen capture the desired area using the Screen Capture action with the same settings as the OCR action. If it can't, then you know the issue is with Screen Capture permissions or Accessibility permissions.

If it can, and the Apple Text Recognition fails, then that would be hard to understand.

OCR Image cannot get target error implies that Keyboard Maestro could not capture the target image (ie, this happens before it tries to OCR).

griffman · September 30, 2024, 3:37pm

I got it working by manually adding the KM Engine to Screen recording permissions—the app was listed there, but not the engine.

-rob.

linnic · October 9, 2024, 5:57am

Works properly with Apple Text Recognition, but only in english. Spits out gibberish for other languages. Strange because if I take a screenshot of the text, it identifies the text and allows copying perfectly out of preview so Apple's OCR does recognize the language well

peternlewis · October 9, 2024, 6:45am

Good to know, I'll see if I can enhance the action in the future (especially important if I can't get Tesseract working in Sequoia).

Zabobon · October 9, 2024, 12:40pm

You can make use of Keyboard Maestro's Execute Shortcut Action to access Apple OCR and Apple Translate from within your KM Macros. I do not know what your Macro is doing or looks like but the below should give you the idea. In my testing it worked fine to translate from Polish to English (MacOS Sequoia 15.01 and Keyboard Maestro 11.0.3)

You can edit the Apple Shortcut to do what you want in terms of language to translate from and to and whether you want it to do a screenshot (as I did) or to get the text from an image.

Of course, you don't have to have it translate - it can return the recognised text in the original Polish (by simply deleting the last step in the Apple Shortcut) if that is what you are wanting to do.

OCR Translate from Polish to English.kmmacros (2.6 KB)