I have been advised to use Pause Until actions in place of hard pauses, and I appreciate the advisability of doing so. I've had some success in doing this, but am confronting some difficulty in doing it in a macro I use every day that contains the following pauses in a loop that runs between three and eight times.
Some of these may even be unnecessary – you know, when you plug in a couple of pauses to get a macro to work and then don't figure out which ones you didn't need and delete those. I probably did that with this one, and it's going to be a job to streamline it now. I could post the whole macro and will if anybody wants to see it (I think I already did before, as I got help on it), but it contains clicks that wouldn't work on anything other than a screen like mine and so for the time being I'll just sequentially post the points where I have a question. I invite others who may want to do the same thing to post cases of their own and maybe we can have like a seminar here.
My macro inserts text in the "Enter any topic..." field at https://relatedwords.io/ and types the return keystroke. I then want to click and drag to select the top line of the resulting text, but I have to wait until the page loads. The current pause is 2.0 seconds, which is sufficient but it may not take that long. What kind of Pause Until might I be able to use here?
No and it works, thanks. I've seen this action in the list many times but somehow didn't take sufficient notice of it. I suppose if its name had been "Wait for [Front] Browser to Finish Loading PAGE", it might have registered.
The pause following this one was superfluous and is now deleted. I now have a line of text selected at the top of the page and jump down to the bottom. I can't simply delete the pause following this, before I shift-click near the bottom of the page to select the rest of the text, but 3 seconds seems excessive. I can probably trim the pause length down (risking that I'll trim it down too much), but is there maybe a way to replace it with a Pause Until?
Do you need to Shift-Click, or will ⇧⌘↓ select to the end of the text? If not -- is there a scroll bar? You might be able to use image detection to pause until there's no gap between the bottom of the thumb and the end of the bar.
Both. ⇧⌘↓ selects to the end of the text, but there's text at the bottom that I don't want copied.
If not -- is there a scroll bar? You might be able to use image detection to pause until there's no gap between the bottom of the thumb and the end of the bar.
This sounds interesting, but I have a lingering memory that image detection noticeably slows things down. If this is no longer an issue, great, I want to do it. I've trimmed this particular pause down from 3 seconds to 1 and it's working with that at least for the time being, so this is not going to result in much of a time savings if any, even discounting the time put into getting the new pause to work, but I still want to try it if only just to learn something that might come in handy some other time. Since there's no "Pause Until Found Image", I suppose I have to use "Find Image on Screen" and have an "If Then Else" with the new variable? If so, what's the Else? Sorry if this is dumb. Thanks.
Just click on the "Add Condition" button and select "Found Image Condition" -- that'll get you all the usual image action options.
OK -- I've just had a look at the page I think you are trying to scrape. If you are just trying to get "the words in the boxes" you are probably better off trapping all the data-term entities from the page, either with JavaScript or by searching the HTML.
If that is what you are aiming for then let us know and, particularly, say what browser you are using -- that may limit options -- and someone might be able to come up with something.
I'm sure @Nige_S will come up with something better than having to shift-click the mouse, but if that is literally what you want the pause to wait for, you can do this:
But I can't get it to work. Or, maybe I can, but it seems to mess something up so that the macro doesn't work after this step. It must find the image because the macro continues, but the following shift-click doesn't work and so the desired text isn't selected. A timed pause might take care of this, but that's what I'm trying to replace and if I have to do that I should probably just leave the original pause (which I had to take back up to 2 seconds from 1).
Thanks, but the shift-click is part of the macro, so the key isn't pressed and I don't think this would work. Also, I'm not waiting for a shift-click, but to be at the bottom of the page so I can then shift-click and select the text.
[Edit: I'm deleting a last paragraph here saying that if anybody can make a macro that selects the first line of finds for a word there, jumps to the bottom with an image-find pause, and then successfully shift-clicks selecting all of the related words, I'd be interested in seeing it. Well, that's not hard, I did it, and anybody else could do the same. The problem is that the image-find, though it works in itself, seems to mess up the execution of the rest of the macro, in consequence of which I've provisionally given up on the image pause.)
You could also copy all content of the result page with ⌘A - ⌘C to system clipboard in a WHILE-action and test if for example the system clipboard contains "Have a nice day!". And as you have all the pagecontent in the system clipboard you don't have to Shift Click to copy. You could trim unwanted parts afterwards.
I still think this is the wrong approach -- much better to get the all text then process down to what you want, or use JavaScript to get just what you want (if your browser supports KM's JavaScript actions).
The first is easy. You'll still probably need a "Click" action to deactivate the form field, but then you can rely on an auto-generated web page having the same format -- in this case, 16 unwanted lines before your terms, 1 text line plus some blank lines after -- which you can use in processing.
You'll still need some pauses to give you browser time to react to your "UI activity".
If your browser supports "Execute JavaScript" action then it's even easier. Note that I am a complete JS-newb and the gurus here could easily improve on this. But it works!
Replicating what you do manually is a good way to plan and start a macro's design -- but if you always look for ways you can do things more "programmatically" you'll end up with a more robust and efficient workflow.
As @Nige_S and @Neo are suggesting, copying all the text and then processing it is probably a more robust way to go rather than replicating manual clicks etc.
One of my main jobs at the place I worked before retiring was producing the organization's annual yearbook, which consisted of thousands of different elements. This was considered more or less a full-time year-round job and there had always been a separate person with that particular job title, but I got a huge part of it automated with a complicated combination of macros and scripts, to the point where it took far less time and I could generally take it pretty easy. I imagine there are people in this forum who are, have been, or may want to be in the same boat, lol. Anyway, looking back on this I remember that I did have a number of manual interventions, with alerts telling me what to do so I didn't have to re-think about anything; so I shouldn't be fanatically averse to manual interventions now, although part of the retirement-hobby fun is avoiding them and getting everything to run automatically from start to finish. So when you talk about trimming unwanted parts of text or processing it, it makes a difference to me whether that is scripted or manual.
This whole project actually seems rather dubious, as the only thing really wrong with my macro was that I originally inserted various overly long pauses in it and didn't go back to delete or shave them down after I got it working. The shift-click after a 3-second pause worked fine, I was just trying to get a "non-hard" pause like people were saying I should. The found-image thing apparently turned out to be impractical, since though the step itself works the macro doesn't work properly afterwards with this step inserted.
So here I am with more or less the macro I started from, now having trimmed or deleted the superfluous pauses that were making it run too slow before. I'm open to trying a new approach on the text selection, @Nige_S, and I'm pretty sure Brave Browser, which I understand is based on the Chrome engine, can handle JavaScript actions as well as Chrome. But what I'd like to do now is keep "sort by:" at relatedwords.io set to "alphabetical" after a word is entered. I can do this with a click and drag, but it keeps reverting back to "relevance" in what seems like an erratic manner. Is anyone aware of a non-click-and-drag way to set/reset this popdown to "alphabetical"? Thanks.
There's a risk it won't work in your browser, but in Firefox, if I press the TAB key twice it moves the focus to that menu item, and then the "A" key sets it to "alphabetical". I suppose another risk is that KM will enter the keystokes too quickly and you'll have to put pauses between them to make it work...
I think the way you are doing this is exactly right. Get the macro working and then, if you find you're using it regularly, fine-tune it to make it more robust.
With the "fixed time pause" or "pause until" thing, I've found that sometimes a "fixed time" pause can actually be more robust than a convoluted method of getting a "pause until". In the end, the great thing about Keyboard Maestro is that it is so modular that are many ways of building macros that work and are genuinely useful for daily tasks.
"Processing the page" is less about the pause, more about avoiding the Shift-Click. UI interaction is generally more fragile than the programmatic approach.
I'm not saying you must avoid UI interaction because of that -- often it is less painful to accept the occasional failure than it is re-write the macro to avoid the UI (if that can even be done). But here the fix is so simple you might as well do it. Especially given:
Just use the JavaScript action from above to get the text, but add a line to set the sort at the beginning:
Well, this was the paragraph I deleted; it wasn't entirely my approach. I do have the shift-click reliably working with a three-second pause, however, plus I'm also getting the lists in alphabetical order thanks to @RogerB's suggestion. So I've saved the one I have and am now getting back to your approach, which is certainly interesting and likely educational for me if I can pick up on at least some of it.
The first is easy. You'll still probably need a "Click" action to deactivate the form field, but then you can rely on an auto-generated web page having the same format -- in this case, 16 unwanted lines before your terms, 1 text line plus some blank lines after -- which you can use in processing.
You'll still need some pauses to give you browser time to react to your "UI activity".
If your browser supports "Execute JavaScript" action then it's even easier. Note that I am a complete JS-newb and the gurus here could easily improve on this. But it works!
Yes, but I had to change two things:
(1) I replaced the Found Image click, which didn't work for me, with a normal click at an arbitrary point outside the form field.
(2) I had to change ^(.\R){16} to ^(.\R){8} to get all the words. I didn't/don't see what you saw at the top of the page, so don't know why it was 16 for you but 8 for me.
The JavaScript is Greek to me and will presumably stay so at least for the time being. This macro works in Brave Browser, however, and is impressively fast without pauses, so I'll use it if I can get it to work within the existing macro, which I suppose I can. [...] Yes, it's in there.
Replicating what you do manually is a good way to plan and start a macro's design -- but if you always look for ways you can do things more "programmatically" you'll end up with a more robust and efficient workflow.
I don't doubt that in the slightest, though I may be too old and lazy to do anything like learning JavaScript.
This unfortunately didn't work: when I added the line I got a blank window. I had RogerB's method for getting alphabetical order, though, so I used that instead. I had a problem getting the new macro to work, as it kept giving me the same list on later iterations. I finally discovered that while cmd-up arrow and tab somehow put me in the form field in the original macro, it didn't when I used the JavaScript and then I didn't know how to get the cursor there at all. So I wound up using a normal click, and now I have two macros that do what I want. Such wealth. Thanks!
You'll usually have to replace someone else's images with your own -- different browsers/preferences/OSes/etc can cause mean same thing renders differently for you than for me.
Again, different browsers can present the text of a web page in different ways. It appears that Safari is delimiting paragraphs with two linefeeds, Brave with only one.
Me too, which is why I won't try and troubleshoot the not-working sorted version -- although, if you added your own "Execute JavaScript" action, check that it isn't set to the default of "Use Modern Syntax":
More importantly, there's no need to do that (or @RogerB's UI method) in this case. Changing the sort order only changes the order in which terms are returned -- just use whatever the page has defaulted to, split the returned text to one term per line, and then sort the text with a KM "Filter: Sort Lines" action.
Another example of the fragility of UI interactions, I'm afraid. But since Brave supports KM's browser actions you should be able to solve this one. It isn't as easy as if could be because there is no form defined on the page, so you can't use the "Focus Front Browser Field" action. But you can, again, turn to javaScript:
document.getElementById('searchBox').focus();
Again, I'm behind the times so make sure you uncheck "Modern Syntax".
As before, there's no need to use these tweaks if your macro is working as you want. But bear them in mind the next time you want to macro some browser interactions -- there are so many options available in KM it can be difficult to keep track of them all, but just knowing they are there is a good start!
I'm not sure whether I recorded a new image or not, but in any event "your" version failed after having initially worked, so I went back to "my" version. Then it failed too, but I could correct that by lengthening a pause. I went back and had another go at "your" version, trying both to get an image to work and to fix whatever the other problem was, but I was too confused by the various JS proposals and so I finally went back to my old slightly clunky macro.
As before, there's no need to use these tweaks if your macro is working as you want. But bear them in mind the next time you want to macro some browser interactions -- there are so many options available in KM it can be difficult to keep track of them all, but just knowing they are there is a good start!
