Howdy,
Looking for a way to extract text from web elements based on xpath. I’d like to do this by providing the macro an xpath, and any text within the element will be saved to a variable.
This is for work, so I can’t provide a specific use case. Also the web page is behind a password, so some of the solutions I’ve found by searching these forums haven’t worked thus yet. Does anyone have any ideas on how this might be accomplished?
Any guidance would be greatly appreciated.
Hello and thanks! That's very helpful. I'm a total noob though - could you perhaps explain how I get that to work? Which text do i replace? How do i fit it into a km macro?
Thanks again!
For instance, if i get the xpath of my last post, it's:
/html/body/section/div/div[2]/div[2]/div[4]/div[3]/section/div[1]/div[2]/div/div[3]/article/div/div[2]/div[2]/div/p[1]
Where do I put that in your screenshot? Also, how does that script know what web page to look at? Seems like the xpath doesn't have any sort of URL or anything.
Sorry for the newbie questions! Thanks for any answers you can provide!
Ok, I took a stab at this, using the xpath from my first reply. I've assigned the script to write to a var. However, when I run the code, nothing happens. Any idea what I've done wrong?
Make sure that you go to View > Developer > Allow Javascript from Apple Events is enabled.
I recommend using the JavaScript function querySelector rather an Xpath.
If you can provide the URL or the HTML code that contains the text you want, we can make a better suggestion.
For example, given the HTML code of this forum web page:
<div class="regular contents">
<div class="cooked">
<p>
Howdy,
</p>
<p>
Looking for a way to extract text from web elements based on xpath. I’d like to do this by providing the macro an xpath, and any text within the element will be saved to a variable.
</p>
<p>
This is for work, so I can’t provide a specific use case. Also the web page is behind a password, so some of the solutions I’ve found by searching these forums haven’t worked thus yet. Does anyone have any ideas on how this might be accomplished?
</p>
<p>
Any guidance would be greatly appreciated.
</p>
</div>
<section class="post-menu-area clearfix">
<nav class="post-controls clearfix collapsed">
<div class="actions">
<div class="double-button">
<button class="widget-button btn-flat toggle-like like no-text btn-icon" aria-label="like this post" title="like this post"><svg class="fa d-icon d-icon-d-unliked svg-icon svg-node" aria-hidden="true"><use xlink:href="#far-heart"></use></svg></button>
</div>
<button class="widget-button btn-flat share no-text btn-icon" aria-label="share a link to this post" title="share a link to this post" data-share-url="/t/looking-for-a-way-to-select-a-web-element-by-its-xpath-and-save-the-text-within-it-to-a-variable/21121?u=jmichaeltx" data-post-number="1"><svg class="fa d-icon d-icon-link svg-icon svg-node" aria-hidden="true"><use xlink:href="#link"></use></svg></button><button class="widget-button btn-flat show-more-actions no-text btn-icon" aria-label="show more" title="show more"><svg class="fa d-icon d-icon-ellipsis-h svg-icon svg-node" aria-hidden="true"><use xlink:href="#ellipsis-h"></use></svg></button><button class="widget-button btn-flat reply create fade-out btn-icon-text" aria-label="begin composing a reply to this post" title="begin composing a reply to this post"><svg class="fa d-icon d-icon-reply svg-icon svg-node" aria-hidden="true"><use xlink:href="#reply"></use></svg><span class="d-button-label">Reply</span></button>
</div>
</nav>
</section>
</div>
We can identify the HTML tag of interest as
<div class="cooked">
So, we can use this JavaScript:
'use strict';
(function myMain() { // this will auto-run when script is executed
/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PURPOSE: Extract Text from HTML Element
VER: 1.0 2021-01-28
AUTHOR: @JMichaelTX
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*/
//var kmVar = document.kmvar.VARNAME;
//var scriptResults = "TBD"
var targetElem = document.querySelector('div.cooked');
if (targetElem) {
var elemText = targetElem.innerText
} else { var elemText = '[ERROR] HTML Element NOT Found'; }
return elemText;
} // END of function myMain()
)();
Here's an example macro:
Below is just an example written in response to your request. You will need to use as an example and/or change to meet your workflow automation needs.
Please let us know if it meets your needs.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-~~~ VER: 1.0 2021-01-28 ~~~
Requires: KM 8.2.4+ macOS 10.11 (El Capitan)+
(Macro was written & tested using KM 9.0+ on macOS 10.14.5 (Mojave))
Extract Text From HTML Element [Example].kmmacros
Note: This Macro was uploaded in a DISABLED state. You must enable before it can be triggered.
@Matt_Lewis - that did the trick, thanks!
@JMichaelTX - this script works amazing. Thank you so so much.
One more question if it's alright - is there a way to do this action while the desired webpage is not the active window? Can it be assigned to a minimized Chrome window, while I'm working in other Chrome windows? Or is it always going to be the active window only?
Thanks again!
Nope. It must always be the active TAB in the active Window, if you want to use the KM Execute a JavaScript in Browser actions.
If you can use AppleScript or JXA, then you can direct the JavaScript action to a specific TAB in a specific Window.
Whoa that applescript idea sounds pretty powerful. I hate to be a bother, but do you happen to know how to do that? XD
edit: Assuming that applescript piece you mentioned can be achieved with KM? Maybe I misunderstood.
Executing a JavaScript in a specific TAB in a specific Window can get complicated.
I don't have a good example to share, but maybe my friend Chris @ccstone does.
You can execute an AppleScript or JXA script from KM using Execute an AppleScript action or JavaScript for Automation action.
Cool, that gives me a good direction to look in, I'll do some research. Thanks.
If Chris chimes in with an example even better! Thanks for all the help 
Hey @cmizle,
I've made this reasonably user-friendly.
-Chris
Execute a JavaScript in the Designated Tab of Google Chrome v1.00.kmmacros (10 KB)
Hey Chris, thanks for the macro! This is awesome! I'm having a little trouble inputting my own user settings. I tried incorporating a line of code that JMichaelTX provided, attempting to make it read the text contents of the first element that matches the name 'div.cooked'.
Doing this did not return any results. Here is how my changes look:
If you have a moment I'd love to know what I'm doing wrong! Thanks again!
Hey @cmizle,
What I'm doing in the macro is allowing you to use your own JavaScript.
Did you try the macro, before you started trying to edit it?
(Real question, not being sarcastic.)
It returns the href for each link on the given page as a text list:
(Array.from(document.querySelectorAll('a'), anchorRef => anchorRef.href)).join('\n')
This was intended to provide a somewhat dramatic example of what's possible.
Since @JMichaelTX's JavaScript is working for you the thing to do would be to use it in its entirety, rather than try to rewrite my JavaScript (which is for quite a different purpose).
JM's JavaScript is good, because it's simple, readable, and provides basic error-checking.
If you were to write it in similar fashion to mine, it would be something like:
document.querySelector('div.cooked').innerText
This is absolutely fine for one-off tasks, but for routine tasks code more like JM's is better.
-Chris
Oh doh! I get it now. Sorry, I'm quite new at this! But thank you for the demo, and for this awesome script. This, combined with JMichaelTX's script is exactly what I need. You guys rock!!
Thanks again!
@Matt_Lewis @ccstone : I wanted to reply to JMichaelTX but his account seems to be no longer active. Hopefully my question is easy enough for being my first 
Question: Using the solution below as the foundation:
Could one set up a Keyboard Maestro Variable to use as malleable reference point?
Currently, this solution requires manipulation if used as a "Building Block"
I appreciate any feedback in advance
...
JMichaelTX Jan 2021
"I recommend using the JavaScript function querySelector rather an Xpath.
For example, given the HTML code of this forum web page:
We can identify the HTML tag of interest as
<div class="cooked">
So, we can use this JavaScript"
***var targetElem =*** document.querySelector(***'==div.cooked=='***);
if (targetElem) {
var elemText = targetElem.innerText
} else { var elemText = '[ERROR] HTML Element NOT Found'; }
return elemText;
Hey Jesse,
I'm sad to say that @JMichaelTX is no longer with us...
JMichaelTX – Unforgettable Generosity With His Time and Knowledge
This is a bit vague – please be more specific.
-Chris