I use OCR a lot. The new Apple Text Recognition algorithm in KM v11 rarely makes mistakes. But there is one thing that does trip it up: the text cursor. I.e., the vertical flashing "beam." It often gets interpreted as the letter "I". (Perhaps lowercase L or uppercase I.)
It's important to consider that the "mouse arrow" would also make OCR fail, but there's something special about the mouse that it is not visible to any screen snapshots including OCR, but the "vertical text cursor" is unfortunately very visible to both screenshots and OCR.
So I would like to be able to turn the cursor off prior to image capture, and then hopefully, back on again.
If that's not possible, are there any apps that use a different style of text cursor? I've noticed that Terminal uses a square block for its cursor. Are there any other macOS apps that use something different like this?
I just noticed that the flashing cursor seems to be synchronized with the second hand. If I pause my "OCR action" until the second hand is precisely on the second, I may be able to avoid the cursor!
What kinds of applications are you thinking of? (I see the flashing text cursor as I type this...) Text editors, Web browsers, anything that handles text?
I wonder why you are using OCR rather than copying the text, but I can imagine some situations, I suppose...
Would it suit your needs to make another application active before running OCR? The cursor will probably disappear for the target app when it's in the background.
Well, as I said, the Terminal app has its own block for a cursor. I didn't test what the OCR returns when it sees a block symbol.
What you don't know is that most of the time there won't be any text. In that case, there's nothing to copy and the system generates a horrible sound. There's an entire thread on this website on how to detect if a text field is empty, but I couldn't get it to work. So I decided to go the OCR route.
Good thought, but screen animation/flashing/flickering is not acceptable to me.
I'm doing detailed testing using a "fractional second timer" to see if I can perform the OCR "during the cursor off periods". I'm not getting the results I expected, which isn't a bad thing. When you run an experiment and get unexpected results, it often means your expectations were faulty in the first place, which means a discovery could be close at hand.
Firstly, without my special pause statement that waits for the fraction seconds to reach zero, the Apple OCR algorithm reads the cursor bar (incorrectly) "as a letter" only about 10% of the time. When using my human eye to see the cursor, it seems to be "on" at least 60% of the time. But through my testing I realize that a good portion of that 60% is actually a dimmer (grey) cursor. The cursor isn't just in an "on" or "off" state, it's actually many shades of grey. And the Apple OCR software can see that, which means it detects the cursor as a letter only about 10% of the time. So maybe there's no need for a correction algorithm. It's not as bad as I expected.
However I do have a correction algorithm now. The reason I hesitate to post it is that my algorithm should be able to make the OCR fail 100% of the time by allowing the OCR to occur only when the cursor bar is solid black. However no matter what I seem to try, my algorithm works 100% of the time. That makes no sense. I mean, it gives me exactly what I want: perfect success, but there must be something I don't understand, because my technique should be able to make it fail 100% of the time if I tune it improperly.
Just a thought: Might it be that the Apple OCR correctly detects that it is a text cursor, and ignores it, when the cursor is in its fully black state? When you'd expect it to definitively fail. And that it is when it is in some shades of grey it gets interpreted as a character?
So you think the 10% error rate is when the text cursor is a different colour from the text?
HEY! That gives me an idea. If I just switch the colour of the text to red, that could give me 100% success. I feel I'm so stupid now! Thanks! (Sorry about the typo, I was definitely referring to myself as stupid.)
Yes, I read that, but perhaps there's something else I missed, because without knowing which apps, or even types of apps, that you need your macro to work with the question "are there any apps that use a different style of text cursor?" seems very open-ended. But perhaps your macro is for all possible situations.
I was thinking along the lines of using the Activate a specific application action to put any app (with no windows open) in the foreground, whereupon the text cursor in your problem app would disappear, ready for OCR. There wouldn't be any animation/flashing/flickering, but the menu bar would change, of course, and I don't know if that would be acceptable for your needs here.
I will adapt to whatever app works without a flashing cursor. I can't tell you which apps meet my requirement, which is why I asked.
Having this discussion has enabled me to form a better search term for googling this topic, and I found someone who claims to have a 99% solution to this problem. We can significantly reduce the cursor's visibility by changing this: (and restarting any app for this to take effect)
I wasn't necessarily thinking of the shade of grey in terms of it's difference from the text colour, but more in terms of it making the Apple OCR's reading of the text cursor, as a text cursor, more ambiguous. That is if the Apple OCR where to have some sort of text cursor detection built in (as it ideally definitely should, in this world of text existing on screens in increasing degrees), but it was just a thought that popped up
Any type of app that can hold text would solve my problem, it could be an editor, or a calculator, or a world globe viewer, as long as it allowed for text and didn't have a cursor that interfered with OCR. Thanks for your three suggestions. I googled them and found Youtube videos showing how they work. I guess I would have to test whether their modified cursors would spoil OCR or not. I was already aware of how Terminal handles cursors differently, so nano and pico seem to inherit Terminal's cursor.
