Integrate Firefox with Keyboard Maestro

my good frien
yo are a kind men

the macro can not find the image
please can you tell me the area
where the image is on
i want to narrow the search

The image in the example macro is from the Firefox Accounts popup menu, from your screenshot. Top-right area:

27-pty-fs8

But you should make the images yourself, from your real popup menu.

As you see, the image appears only when the popup menu is open. That means, first you must make sure that the first Click action (the one with the coordinates) hits the Accounts button. Otherwise the Accounts list will not be visible and KM will never find the image.

Use the Get button of the action to get the exact coordinates. Please see also the KM Wiki article, if you don’t know how the action works.

My macro is just an example that shows you how you can probably do it. You have to adapt the macro (coordinates, images) to your screen. Also adjust the length of the Pause actions, if necessary. The Accounts list must be on screen for the Click at Found Image action (action 3).

So, once you have the correct coordinates, but the accounts list takes too long to pop up, use a pause of 1s instead of the 0.5s.

PS:

Instead of the first Pause action you could also use a Pause until screen contains image action, with the same image as in action 3. But if the pause is long enough, it doesn’t matter.

my friend
the list is long
their is a 100 of names

it do not work

see this screenshot :

i need that operation
that if the account is on "monic zrivoic"
the macro spake "monic"

and if it "Addison Wilson"


the macro spake "Addison"
and so on for 100

so i tray to detract the image
for "Addison Wilson"
but the macro do not spake "Addison"

Of course not. You didn’t even ask for that in your OP.

In your OP you asked for actions based on which account you select in the popup list. Now you are asking for a Speak Text action based on which account is displayed in the Firefox address bar —if I understand correctly (?)

There are various problems:

You could still detect the name in the address bar with a Find Image action, but for that you’d still have to create 100 little images and 100 If Found Image Then actions. Next problem: When should the macro trigger? I guess, it should trigger when the displayed name changes. For that, KM would have to scan your screen constantly for image changes.

In theory you could use the OCR action, based on screen coordinates. This would save you from creating 100 images. But then you probably have a problem with the varying text length of the names and the ⌵ symbol (as in the “monic” screenshot). And you would still have to detect changes in the displayed account name.

Maybe someone else has a better idea?

OK
i could not succeed of doing Find Image action
i not detract the name
maybe it too small ?

please i want to tell the KM
to search in area the "dan mano" image
but i can't make the coordinate corect

whet i say is
that i cannot find the "dan mano" or "Daniel jakulovic" image
with Find Image action
maybe it too small for Find Image action

It seems you are going for the OCR solution (since you are looking for coordinates).

Try this macro:

OCR%20Based%20on%20Coordinates%20of%20Found%20Image%20%3C17DF%20200223T000950%3E-pty-fs8
OCR Based on Coordinates of Found Image <17DF 200223T000950>.kmmacros (16.1 KB)

What the macro does:

  1. It finds the “dan mano” image,
  2. Saves the coordinates of “dan mano” to a variable,
  3. Uses these coordinates to define the area for the OCR and performs the OCR,
  4. From the string “dan mano” it removes the part after the space (the last name),
  5. It speakes the remaining part (“dan”).

Please try to understand how the macro works and test if it works also on your screen. Adjust slightly the fuzziness slider, if necessary.

Now you see already the problem, right? It works for “dan mano” but not for “Addison Wilson” and the others. That means, you need to do this for every name image, which means that you might as well just combine a Find Image action with the hard-coded (fixed) name in the Speak action, without any OCR.

So, what we have to find, is an image that looks the same or similar for all your account names, so that the macro works for all names. (It only has to define the area for the OCR.)

I tried it with this mix of letters that pretends to approximate the average shape of a name:

22

And, indeed, it works for “dan mano” and for “Addison Wilson”. But only if “Addison Wilson” has the same color as “dan mano”. (And with a generous setting of the fuzziness slider, of course.)


So, probably the better way:

Instead of getting the coordinates by finding the name image itself, try to find the image of the area to the right of the name (this is from your screenshot):

26

This seems sufficiently unique, and does not change with the account names.

Then you have to add a horizontal offset to the coordinates in the second action, so that the OCR is performed in the area to the left of the the found image, like for example:

OCR%20Based%20on%20Coordinates%20of%20Found%20Image%20(with%20offset)%20%3C371C%20200223T005647%3E

In my test it worked fine for “dan mano” and for “Addison Wilson” in different colors.

I cannot deliver a macro that works for you, because my Firefox tool bar looks quite different. But all you have to do, is

  • making the little screenshot of the right end of the tool bar (or maybe the image above, which I made from your screenshot, works) and
  • experiment a bit to find a good horizontal offset. (Not too much, because it must not overlap with the text of the http address.)

PS:

If KM is unable to find an image at all and gives you a ”no unique image found” error, then please see this:

thank you for everything
just for my education
whet is
Local Image[1]
in the oCR area ?

The Find Image on Screen action saves the coordinates of the found image in this form to the Local Image variable:

1704,145,41,38,3

The first is the horizontal position, the second the vertical one, etc. See the Wiki page for details.

The variable Local Image[1] in the example is 1704, Local Image[2] is 145, and so on. It separates the content of the variable, with the comma as default separator. Please see this Wiki article on how variables work, in particular the section on Variable Arrays.

You can always check the content of your variables by throwing in a Display Text action with the variable as content. Example:

___tmp02%20%3C2FC5%20200223T190658%3E-pty-fs8
___tmp02 <2FC5 200223T190658>.kmmacros (25.9 KB)

You can do this at any point in your macro if you want to see the current value of a variable. Of course, this is only for developing or debugging a macro, not for the final one. (You can also use the Log action, if you prefer to see the info in the log instead of having windows popping up.)

my friend
i try to find the "Daniel jakulovic" image
as you did in "dan mano" image
but the OCR do not recognize the "Daniel jakulovic" image

Untitled%202

You already told me 2 days ago here that you are unable to find the “Daniel jakulovic” image.

In the meantime I have proposed you a different approach that does not require to find each individual image (here, starting with “So, probably the better way:”)

What is wrong with that? Have you tried it? If yes, then please give feedback on that; if not, then let me know why not.

I also gave you a tip here that should help if your KM is unable to find any images at all. What about that? Have you checked this?


I had a look at your other threads and noticed that in virtually every thread one of two things happened:

  • people ansered or even posted macros and you did not even reply a single time
  • you did reply but then a bit later you silently abandoned the thread without any feedback, not answering the questions by the others

You can see your threads here.

This leads me to think that the problem may be language-related and you simply don’t understand what the others are writing. If this is the case, then please copy the text of the reply you receive and paste it into DeepL, an excellent online translation tool. (If your language is not supported by DeepL, then use Google Translate.)

Also the other way round: If you have difficulties to express all the necessary details in English, then write your text in your language and let DeepL or Google Translate do the translation. (If it’s a complicated text, then check the result by doing a reverse translation.)


I’m still willing to help you, but if you don’t give feedback on the things I propose, then it is going to be really difficult. What you are asking for is not exactly trivial.

So, as the next thing, please find out if your KM is able to find images in general. Take whatever image, for example the KM icon on this page…

02
…and test with a Find Image action.

1 Like

listen
can you tel me why when i screenshot selection in macos mojave
and the screenshot selection appear in the screen
why it not detect the image
i try everything you suggest
the "dan mano" is just for example
i need to find more screenshot selection (images)
so i don't want to use OCR

a lot of my threads
end Successfully
but you right
i need to give good feedback
for macro that work

The purpose of my last post was to find out exactly that. Please try to follow my suggestions, this includes checking out the linked troubleshooting post, and please also read this post by Peter.

Then write a minimal macro with a single Click at Found Image action, that clicks the KM logo at the top of this webpage, and upload the macro file.

Once we got this working, we can continue with the account names macro.

please can you tell me
whet is :
Local Image[1]
Local Image[2]
Local Image[3]
Local Image[4]

You have asked the same question 5 days ago here.

Probably you have missed my reply in the follow-up post here, including a downloadable macro that shows you the contents of these variables and how it works – and including three links with more information.

found Shutterstock acount.kmmacros (73.5 KB)

please
i try really hard to makes this macro succeed

please i don't have any experience with coding
or macro creating