Integrate Firefox with Keyboard Maestro

[ghy76df455DFR]

good evening

i have add on on Firefox
call "Firefox Multi-Account Containers"
( https://addons.mozilla.org/en-US/firefox/addon/multi-account-containers/ )
this add on is so amazing for work flow

i want help from you please
for integrate it with Keyboard Maestro
as you see in the right corner of the screen shot
the add on create multi clean Firefox with name that i create
(if you want multi account for any site like pay-pal
it attach every name in the add on with different cookie )

i just need to unique oppression in Keyboard Maestro
in Firefox depend on with name in the add on i select

please

Untitled|690x431

[ghy76df455DFR]

this is the home page of the add-on developer

mozilla/multi-account-containers

[hello]

quickest option is Click Found Image
from there, you can do either:
a) make a list with all the containers you want to use
b) create a keystroke macro for each account

[ghy76df455DFR]

i sorry my friend
I didnt understend
i need to create macro
like :

i mean in right corner of the screen shot i upload in the first post in this page
you can see account call "dan mano" in Firefox Multi-Account Containers in the right corner of the screen shot
or account call "Daniel jakulovic" in Firefox Multi-Account Containers in the right corner of the screen shot

i need macro that do if "dan mano" account is active do something ( some operation that i will create )
and if if "Daniel jakulovic" account is active do something else ( anther operation that i will create )

thanks for everything

[hello]

tbh don't know if xPath can be used in there, but you can do a if-then-else depending on found image (dan, Daniel, etc) condition.

what do you have so far?

[Tom]

In order to make decisions based on which Firefox account you activate, KM first has to know which account you activate.

The problem is that Firefox is not scriptable (not to my knowledge), so you’ll have to resort to UI scripting, and probably best to use a Click Found Image action, as @hello has proposed.

---

Basically I would try this:

Make an area screenshot of an account list entry, like this:

Make a new macro group – active when Firefox is frontmost – with a palette for all contained macros:

Then make a macro with actions along these lines:

1. Mouse click on the coordinates of the Accounts button in Firefox.
   • This will bring up the popup list with the accounts.
2. Mouse click at the Found Image, where the image is the screenshot of the account list entry you have made.
   • This will activate the specified account.
3. Now add any actions you want to perform, specific to the activated account.

Make a macro like this for every account list entry.

When finished, KM should display a palette like this:

When you click a name on the palette, KM will click the Accounts button in FF, click on the account name that corresponds to the image, and then execute the actions you have inserted at the end of the macro.

Here is a sample macro (you’ll probably have to adjust the screen coordinates in the first Click action and the image in the Click Found Image action):

Daniel jakulovic <525C 200221T143428>.kmmacros (62.1 KB)

[ghy76df455DFR]

my good frien
yo are a kind men

the macro can not find the image
please can you tell me the area
where the image is on
i want to narrow the search

[Tom]

The image in the example macro is from the Firefox Accounts popup menu, from your screenshot. Top-right area:

But you should make the images yourself, from your real popup menu.

As you see, the image appears only when the popup menu is open. That means, first you must make sure that the first Click action (the one with the coordinates) hits the Accounts button. Otherwise the Accounts list will not be visible and KM will never find the image.

Use the Get button of the action to get the exact coordinates. Please see also the KM Wiki article, if you don’t know how the action works.

My macro is just an example that shows you how you can probably do it. You have to adapt the macro (coordinates, images) to your screen. Also adjust the length of the Pause actions, if necessary. The Accounts list must be on screen for the Click at Found Image action (action 3).

So, once you have the correct coordinates, but the accounts list takes too long to pop up, use a pause of 1s instead of the 0.5s.

PS:

Instead of the first Pause action you could also use a Pause until screen contains image action, with the same image as in action 3. But if the pause is long enough, it doesn’t matter.

[ghy76df455DFR]

my friend
the list is long
their is a 100 of names

it do not work

see this screenshot :

i need that operation
that if the account is on "monic zrivoic"
the macro spake "monic"

and if it "Addison Wilson"

the macro spake "Addison"
and so on for 100

so i tray to detract the image
for "Addison Wilson"
but the macro do not spake "Addison"

[Tom]

@ghy76df455DFR

but the macro do not spake "Addison"

Of course not. You didn’t even ask for that in your OP.

In your OP you asked for actions based on which account you select in the popup list. Now you are asking for a Speak Text action based on which account is displayed in the Firefox address bar —if I understand correctly (?)

There are various problems:

You could still detect the name in the address bar with a Find Image action, but for that you’d still have to create 100 little images and 100 If Found Image Then actions. Next problem: When should the macro trigger? I guess, it should trigger when the displayed name changes. For that, KM would have to scan your screen constantly for image changes.

In theory you could use the OCR action, based on screen coordinates. This would save you from creating 100 images. But then you probably have a problem with the varying text length of the names and the ⌵ symbol (as in the “monic” screenshot). And you would still have to detect changes in the displayed account name.

Maybe someone else has a better idea?

[ghy76df455DFR]

@Tom

Find Image action

OK
i could not succeed of doing Find Image action
i not detract the name
maybe it too small ?

[ghy76df455DFR]

please i want to tell the KM
to search in area the "dan mano" image
but i can't make the coordinate corect

[ghy76df455DFR]

whet i say is
that i cannot find the "dan mano" or "Daniel jakulovic" image
with Find Image action
maybe it too small for Find Image action

[Tom]

It seems you are going for the OCR solution (since you are looking for coordinates).

Try this macro:

OCR Based on Coordinates of Found Image <17DF 200223T000950>.kmmacros (16.1 KB)

What the macro does:

1. It finds the “dan mano” image,
2. Saves the coordinates of “dan mano” to a variable,
3. Uses these coordinates to define the area for the OCR and performs the OCR,
4. From the string “dan mano” it removes the part after the space (the last name),
5. It speakes the remaining part (“dan”).

Please try to understand how the macro works and test if it works also on your screen. Adjust slightly the fuzziness slider, if necessary.

Now you see already the problem, right? It works for “dan mano” but not for “Addison Wilson” and the others. That means, you need to do this for every name image, which means that you might as well just combine a Find Image action with the hard-coded (fixed) name in the Speak action, without any OCR.

So, what we have to find, is an image that looks the same or similar for all your account names, so that the macro works for all names. (It only has to define the area for the OCR.)

I tried it with this mix of letters that pretends to approximate the average shape of a name:

And, indeed, it works for “dan mano” and for “Addison Wilson”. But only if “Addison Wilson” has the same color as “dan mano”. (And with a generous setting of the fuzziness slider, of course.)

---

So, probably the better way:

Instead of getting the coordinates by finding the name image itself, try to find the image of the area to the right of the name (this is from your screenshot):

This seems sufficiently unique, and does not change with the account names.

Then you have to add a horizontal offset to the coordinates in the second action, so that the OCR is performed in the area to the left of the the found image, like for example:

In my test it worked fine for “dan mano” and for “Addison Wilson” in different colors.

I cannot deliver a macro that works for you, because my Firefox tool bar looks quite different. But all you have to do, is

• making the little screenshot of the right end of the tool bar (or maybe the image above, which I made from your screenshot, works) and
• experiment a bit to find a good horizontal offset. (Not too much, because it must not overlap with the text of the http address.)

[Tom]

PS:

If KM is unable to find an image at all and gives you a ”no unique image found” error, then please see this:

Get button in found image greyed out?

Yep, I think you found it! If I uncheck Keyboard Maestro in the Screen Recording permissions, I get the “no unique image found” error: [17-pty-fs8] I’m sure it is checked on your MacBook, but not on your desktop.

[ghy76df455DFR]

thank you for everything
just for my education
whet is
Local Image[1]
in the oCR area ?

[Tom]

The Find Image on Screen action saves the coordinates of the found image in this form to the Local Image variable:

1704,145,41,38,3

The first is the horizontal position, the second the vertical one, etc. See the Wiki page for details.

The variable Local Image[1] in the example is 1704, Local Image[2] is 145, and so on. It separates the content of the variable, with the comma as default separator. Please see this Wiki article on how variables work, in particular the section on Variable Arrays.

You can always check the content of your variables by throwing in a Display Text action with the variable as content. Example:

___tmp02 <2FC5 200223T190658>.kmmacros (25.9 KB)

You can do this at any point in your macro if you want to see the current value of a variable. Of course, this is only for developing or debugging a macro, not for the final one. (You can also use the Log action, if you prefer to see the info in the log instead of having windows popping up.)

[ghy76df455DFR]

my friend
i try to find the "Daniel jakulovic" image
as you did in "dan mano" image
but the OCR do not recognize the "Daniel jakulovic" image

[Tom]

You already told me 2 days ago here that you are unable to find the “Daniel jakulovic” image.

In the meantime I have proposed you a different approach that does not require to find each individual image (here, starting with “So, probably the better way:”)

What is wrong with that? Have you tried it? If yes, then please give feedback on that; if not, then let me know why not.

I also gave you a tip here that should help if your KM is unable to find any images at all. What about that? Have you checked this?

---

I had a look at your other threads and noticed that in virtually every thread one of two things happened:

• people ansered or even posted macros and you did not even reply a single time
• you did reply but then a bit later you silently abandoned the thread without any feedback, not answering the questions by the others

You can see your threads here.

This leads me to think that the problem may be language-related and you simply don’t understand what the others are writing. If this is the case, then please copy the text of the reply you receive and paste it into DeepL, an excellent online translation tool. (If your language is not supported by DeepL, then use Google Translate.)

Also the other way round: If you have difficulties to express all the necessary details in English, then write your text in your language and let DeepL or Google Translate do the translation. (If it’s a complicated text, then check the result by doing a reverse translation.)

---

I’m still willing to help you, but if you don’t give feedback on the things I propose, then it is going to be really difficult. What you are asking for is not exactly trivial.

So, as the next thing, please find out if your KM is able to find images in general. Take whatever image, for example the KM icon on this page…

…and test with a Find Image action.

[ghy76df455DFR]

listen
can you tel me why when i screenshot selection in macos mojave
and the screenshot selection appear in the screen
why it not detect the image
i try everything you suggest
the "dan mano" is just for example
i need to find more screenshot selection (images)
so i don't want to use OCR