Add Black-And-White Option to Found Image Function

Needing to use "Found Image" to locate a button on a webpage, but it's a translucent button that changes colors based on whatever random image is behind it, so the image-find function isn't finding it because it seems to care more about color than shape.

So I was hoping for the capability to grayscale the screen and the search-image and then do the search.

Can you post screenshots of some examples of the button? You could probably do the grayscale with imagemagick--but maybe it's not necessary and some clever person here could come up with a different way other than found image.


Here's an example.

Assuming you want to click the button, you could try clicking via XPath instead of by found image:

  • Install SelectorsHub (link below).
  • Right click the button and select SelectorsHub → Copy Relative XPath.
  • Paste the copied XPath into a Click Front Browser Link action.

SelectorsHub for Chrome/Brave etc.
SelectorsHub for Safari

You can't.

The best you can do is play with the cropped portion of the image involved and adjust the fuzz-factor.

@noisneil's suggestion to use JavaScript is generally the way to go with this sort of thing, although I usually prefer querySelector to XPath.

1 Like

I've seen you mention it before, and I trust your recommendations, so could I ask what the steps would be to get/use a querySelector?

  • Right-click the element you want in your web browser (I use Google Chrome, but Brave should have the same developer tools).
     
  • Right-Click the code element you want to copy.
    • Copy JS Path.
    • Often requires a bit of editing to get just right.
1 Like

Going to check this out in the morning. Thanks!

1 Like

I'm pretty good with python for data analysis, but I'm definitely not smart enough for this answer. The DOM is still pretty confusing for me. I think the element I'm trying to click is in a "Shadow DOM" (?).

These don't deal directly with querySelector, but the principles are the same.

They're pretty basic, but that'll be very helpful for newbies.

2 Likes

Just my level -- thanks, Chris!

1 Like

I'm new to all this, so this is very basic, but I think I've found an example where querySelector is clearly more useful than XPath.

Let's say I want to get the URL of an account name under a YouTube video.

The relative XPath is:
//ytd-video-owner-renderer[@class='style-scope ytd-watch-metadata']//a[@class='yt-simple-endpoint style-scope yt-formatted-string'][normalize-space()='Guardian News']

It contains the username, so it won't work for every youtube video, whereas the JS Path is:

document.querySelector("ytd-video-owner-renderer[class='style-scope ytd-watch-metadata'] a[class='yt-simple-endpoint style-scope yt-formatted-string']")

This is more generic, so I should be able to use this to get the URL of that item.

I don't know how to do that, but am I right in seeing that as a benefit of using JS Path over XPath?

Hey Neil,

Remember to provide the URL when you can.

document.querySelector('#text > a').href

-Chris

The idea is that it would work for any link of that type (username under a video).

Anyway thanks Chris, I'll check it out when I get in. :pray:t3:

EDIT: works perfectly. Thanks as always, Chris!

That's not the point – the point is being able to test...

I was able to come up with the code, because I sussed out the page on YouTube and figured out the syntax.

All I needed was the way to return a URL. A YouTube video was just an example of a page type that features an element whose JSPath is consistent, but the URL changes.

The only bit I needed to know was to add .href at the end. The rest of it I can get from SelectorsHub.

document.querySelector("ytd-video-owner-renderer[class='style-scope ytd-watch-metadata'] a[class='yt-simple-endpoint style-scope yt-formatted-string']").href

Your plug-in built a more complex query than necessary.

This works reliably for the given element on the YouTube page, although it's pretty general and therefore fragile (being that it's the first element with ID = "text", and that could easily change).

document.querySelector('#text > a').href

This is much more robust:

document.querySelector('a.yt-simple-endpoint.style-scope.yt-formatted-string').href

It's MUCH easier to build these when you can tinker with them live...

1 Like