Thank you both a bunch, very nice work!
So I had been using this almost daily until I upgraded my Mac and it broke any ideas?
Keyboard Maestro 8.2.2 “Harvest Google result links, opening each in own tab” Macro
Harvest Google result links- opening each in own tab.kmmacros (30 KB)
Does the Safari on your new system have this setting enabled in the main menu:
Safari > Develop > Allow Javascript from Apple Events ?
Harvest Google result links (from JXA action) opening each in own tab.kmmacros (21.1 KB)
This variant (running the code from an Execute JavaScript for Automation action – rather than the shell) seems to be working here.
Any good where you are ?
1 Like
And here is an updated JXA script, which seems to be working on Sierra, with KM 8.2.2 and Safari 11.1.2 (Develop > Allow JavaScript from Apple Events enabled)
(() => {
'use strict';
const main = () => {
const
strXPath = "//*[@class='r']/a",
saf = Application("Safari"),
ws = saf.windows,
lrHarvest = bindLR(
0 < ws.length ? (
Right(ws.at(0))
) : Left('No window open in Safari'),
w => {
const
xs = pageXPathHarvest(
saf, w, strXPath
);
return 0 < xs.length ? (
// Safari effect
tabsOpened(saf, w, xs),
// Keyboard Maestro value
Right(
xs.reduce(
(a, link) =>
`${a}[${link[0]}](${link[1]})\n`,
''
)
)
) : Left(
'Perhaps not a Google search page ?\n' +
'(No links matching "' + strXPath + '")'
);
}
);
return lrHarvest.Left || lrHarvest.Right;
};
// SAFARI ---------------------------------------------
// Harvest elements from Safari by XPath pattern
const pageXPathHarvest = (browser, oWin, strXPath) =>
browser.doJavaScript(
`(${xpathHarvest})("${strXPath}")`, { in
: oWin.currentTab
}
);
// tabsOpened :: Application -> Window -> (String, String) -> IO()
const tabsOpened = (safari, oWin, links) => {
const winTabs = oWin.tabs;
links.map(link => winTabs.push(safari.Tab({
url: link[1]
})))
}
// Harvesting function to run in the browser context
const xpathHarvest = strPath => {
const
r = document.evaluate(strPath, document, null, 0, null),
xs = [];
var oNode;
while (oNode = r.iterateNext()) {
xs.push([oNode.text, oNode.href]);
}
return xs;
};
// GENERIC FUNCTIONS ----------------------------------
// https://github.com/RobTrew/prelude-jxa
// Left :: a -> Either a b
const Left = x => ({
type: 'Either',
Left: x
});
// Right :: b -> Either a b
const Right = x => ({
type: 'Either',
Right: x
});
// bindLR (>>=) :: Either a -> (a -> Either b) -> Either b
const bindLR = (m, mf) =>
m.Right !== undefined ? (
mf(m.Right)
) : m;
// MAIN
return main();
})();
1 Like
Thank you I forgot about checking that and thought it was the upgrade that I did.
Very appreciated.
1 Like
Hey @ComplexPoint
I tried to use this script to open first Google result from the keyboard and it fails on me.
I was running it like so:
I did activate: Develop > Allow JavaScript from Apple Events enabled
Perhaps I am missing something and this is not what the script does. It's just the AppleScript I was using made by @ccstone is failing in Sierra and above now.
Here is the error I get when I run it:
/var/folders/5l/5s7qw5ld1xxglm0094_hyh_c0000gn/T/Keyboard-Maestro-Script-81ECA7A6-72E3-4B9E-B603-D65552A765FE:5:6: script error: Expected expression but found “>”. (-2741)
Something to do with the triggering ? (this version is Safari-only, of course).
The following variant is working here with:
- Safari Version 11.1.2 (13605.3.8)
- MacOS 10.13.6
(Opens tabs, copies links as MD)
Harvest Google result links as Markdown.kmmacros (23.7 KB)
JS Source
(() => {
'use strict';
const main = () => {
const
strXPath = "//*[@class='r']/a",
saf = Application("Safari"),
ws = saf.windows;
return bindLR(
0 < ws.length ? (
Right(ws.at(0))
) : Left('No window open in Safari'),
w => {
const
xs = pageXPathHarvest(
saf, w, strXPath
);
return 0 < xs.length ? (
// Safari effect
tabsOpened(saf, w, xs),
// Keyboard Maestro value
Right(
xs.reduce(
(a, link) =>
`${a}[${link[0]}](${link[1]})\n`,
''
)
)
) : Left(
'Perhaps not a Google search page ?\n' +
'(No links matching "' + strXPath + '")'
);
}
);
};
// SAFARI ---------------------------------------------
// Harvest elements from Safari by XPath pattern
const pageXPathHarvest = (browser, oWin, strXPath) =>
browser.doJavaScript(
`(${xpathHarvest})("${strXPath}")`, { in
: oWin.currentTab
}
);
// tabsOpened :: Application -> Window -> (String, String) -> IO()
const tabsOpened = (safari, oWin, links) => {
const winTabs = oWin.tabs;
links.map(link => winTabs.push(safari.Tab({
url: link[1]
})))
}
// Harvesting function to run in the browser context
const xpathHarvest = strPath => {
const
r = document.evaluate(strPath, document, null, 0, null),
xs = [];
var oNode;
while (oNode = r.iterateNext()) {
xs.push([oNode.text, oNode.href]);
}
return xs;
};
// GENERIC FUNCTIONS ----------------------------------
// https://github.com/RobTrew/prelude-jxa
// Left :: a -> Either a b
const Left = x => ({
type: 'Either',
Left: x
});
// Right :: b -> Either a b
const Right = x => ({
type: 'Either',
Right: x
});
// bindLR (>>=) :: Either a -> (a -> Either b) -> Either b
const bindLR = (m, mf) =>
m.Right !== undefined ? (
mf(m.Right)
) : m;
// MAIN
return main();
})();
This one works in that it opens all the links in new tabs. I just wanted a script that will open first link only in current tab.
And a way to specify which link to open in the script so I can have 4 different hotkeys for opening either 1st or 2nd or 3rd or 4th link with a macro for each one.
Got it – so what you need, in place of the tabsOpened function, is something like:
// nthLinkOpened :: Application -> (String, String) -> Int -> IO()
const nthLinkOpened = (safari, links, i) => {
const
ds = safari.documents,
d = ( // Ensuring that there is an open document
ds.length < 1 && ds.push(safari.Document()),
ds.at(0)
);
// i is a zero-based index into the link list.
return d.url = links[i][1];
};
or more directly:
// nthLinkOpened :: Application -> (String, String) -> Int -> IO()
const nthLinkOpened = (safari, links, i) =>
(ds =>
(
ds.length < 1 && ds.push(safari.Document()),
ds.at(0)
)
.url = links[i][1]
)(safari.documents);
Where i is a zero-based index to the link that interests you.
Does that give you enough ?
(Let me know if you'd like me to flesh it out a bit more)
What do I pass in as links to nthLinkOpened?
You just need the same xs (derived by pageXPathHarvest) that were being passed to tabsOpened.
So, for example, if you are opening the first link (linkIndex = 0)
Something like:
(() => {
'use strict';
const main = () => {
const
linkIndex = 0,
strXPath = "//*[@class='r']/a",
saf = Application("Safari"),
ws = saf.windows;
return bindLR(
0 < ws.length ? (
Right(ws.at(0))
) : Left('No window open in Safari'),
w => {
const
xs = pageXPathHarvest(
saf, w, strXPath
);
return 0 < xs.length ? (
// Safari effect
//tabsOpened(saf, w, xs),
nthLinkOpened(saf, xs, linkIndex),
// Keyboard Maestro value
Right(
xs.reduce(
(a, link) =>
`${a}[${link[0]}](${link[1]})\n`,
''
)
)
) : Left(
'Perhaps not a Google search page ?\n' +
'(No links matching "' + strXPath + '")'
);
}
);
};
// SAFARI ---------------------------------------------
// Harvest elements from Safari by XPath pattern
const pageXPathHarvest = (browser, oWin, strXPath) =>
browser.doJavaScript(
`(${xpathHarvest})("${strXPath}")`, { in
: oWin.currentTab
}
);
// tabsOpened :: Application -> Window -> (String, String) -> IO()
const tabsOpened = (safari, oWin, links) => {
const winTabs = oWin.tabs;
links.map(link => winTabs.push(safari.Tab({
url: link[1]
})))
};
// nthLinkOpened :: Application -> (String, String) -> Int -> IO()
const nthLinkOpened = (safari, links, i) =>
(ds =>
(
ds.length < 1 && ds.push(safari.Document()),
ds.at(0)
)
.url = links[i][1]
)(safari.documents);
// Harvesting function to run in the browser context
const xpathHarvest = strPath => {
const
r = document.evaluate(strPath, document, null, 0, null),
xs = [];
let oNode;
while (oNode = r.iterateNext()) {
xs.push([oNode.text, oNode.href]);
}
return xs;
};
// GENERIC FUNCTIONS ----------------------------------
// https://github.com/RobTrew/prelude-jxa
// Left :: a -> Either a b
const Left = x => ({
type: 'Either',
Left: x
});
// Right :: b -> Either a b
const Right = x => ({
type: 'Either',
Right: x
});
// bindLR (>>=) :: Either a -> (a -> Either b) -> Either b
const bindLR = (m, mf) =>
m.Right !== undefined ? (
mf(m.Right)
) : m;
// MAIN
return main();
})();
1 Like
Works wonderfully. Thank you.
1 Like
Does anyone know by chance if it's possible to make @ComplexPoint JXA script work for Google Chrome?
Although this extension works quite nicely too, to solve this. I just wanted to reduce amount of extensions I use in my browsers.
For some reason none of these are working for me anymore. I am on Safari 14 with "Allow JavaScript from Apple Events" checked in the Develop tab. Any idea? It use to open in several tabs and then for a while it would go on endlessly and I would have to stop the script from running and now it just doesn't work.
I tried the code from the later posts to no avail. Web Browser - Safari Macros.kmmacros (129.3 KB)
I'll try to take a look over the weekend. I seems to remember a shift in the API for tabs, but in any case, I'll report on what I find.
Thank you!
Mmm ... I may have to defer to @peternlewis for up-to-date advice on sources of delay in Execute JavaScript in Safari actions,
but this (simple renovated test below) seems to execute:
- instantly in the JS Console of Safari (14.0.2) on macOS (10.15.7)
- pretty quickly in the rough KM macro below
unless under Safari > Develop > [Device name] I have checked:
Automatically Show Web Inspector for JS Contexts
in which case background activation of the debugger causes a delay of several seconds.
(Is it possible that that option is checked on your system ?)
In the draft below it just displays the MD links in a window, but you could, of course, redirect them to a clipboard or KM variable.
If we can sort out the delay issues first, then perhaps we can see if anything else needs renovating ?
First 10 links on Safari page as Markdown.kmmacros (2.7 KB)
JS Source
(() => {
'use strict';
const main = () => {
const
intLinks = 10,
r = document.evaluate(
'//a', document,
null, 0, null
);
return until(
pair => intLinks <= pair[0].length || (
!pair[1]
)
)(
pair => {
const node = pair[1];
return [
Boolean(node.href) ? (
pair[0].concat([
[
node.text.trim() || node.href,
node.href
]
])
) : pair[0],
r.iterateNext()
];
}
)([
[], r.iterateNext()
])[0]
.map(([label, link]) => `[${label}](${link})`)
.join('\n');
};
// ------------------- GENERIC -------------------
// until :: (a -> Bool) -> (a -> a) -> a -> a
const until = p =>
f => x => {
let v = x;
while (!p(v)) v = f(v);
return v;
};
return main();
})();
Thank you for looking into this. Yeah I don't have "Automatically Show Web Inspector for JS Contexts" checked. Hopefully @peterlewis has an idea of what is stopping this from working. This has been such a great command that I use dozens of times a day.
Was the test macro above also slow ?
It turns out that Google have changed the HTML structure of their results pages.
We used to be able to distinguish between the actual search result links and the less interesting menu links by looking for a node class labelled 'r', on the path to 'a' nodes (hyperlinks)
//*[@class='r']/a
but looking at their search result pages, I see that they are no longer using that class name.
If I search for something like "Category Theory", and then inspect the link of the result labelled What is Category Theory Anyway ?, its path on the HTML of that Google search results page turns out, these days, to be:
//*[@id="kp-wp-tab-overview"]/div[4]/div[2]/div[2]/div/div[1]/a
which we could abbreviate to:
//*[@id="kp-wp-tab-overview"]/*/a
but I'm not sure how general or reliable that id value will prove to be. I'll experiment a little.
I think, however, that we may be up against a concern on their part (probably quite understandable) to make page scraping much harder. The paths now contain quite a significant number of unpredictable identifying strings, and they seem to be evading the usual pattern of .href etc node fields.