Reading an element value from webpage onto clipboard or variable

Is it possible to take an element value, such as the distance from a google maps search, and copy that a KM variable.

For example:
A google maps search from Dublin to Cork

gives the distance element as

266 km
How can I read that value (266 km) into a variable?

Thanks

Hey Damian,

Please provide a link to an actual search, and show precisely the data you want to extract.

Grabbing data from web pages is very finicky.

-Chris

When I added a web address to a google search, it pasted a google map onto the post, so I edited it out.

This will hopefully work:

Link to google map search

The element displays as this (when right clicked - display element, in safari).

266 km

So it is the 266km value that I’m trying to grab.
Possibly the best way is through google sheets and a script, and not KM.

Two possible methods, neither of which give me a feeling of absolute reliability or robustness, but you'll have to experiment to see what works best for you.

1. Use JavaScript DOM methods to get the HTML tags of the elements in question

Doing a quick inspection of the webpage to which you linked, I isolated this element here:

It can be referenced through JavaScript like so:

    document.getElementsByClassName('section-directions-trip-distance')[0].getElementsByTagName('div')[0].textContent

Breaking this down:

document.getElementsByClassName('section-directions-trip-distance')[0]

This is probably quite a reliable class name that references the containing <div> elements that hold distance measures. It's an array, where each item represents the different routes available, by distance. As the fastest route is always listed first, I'm going to go out on a limb and say that the first item in the array (with index 0) will reliably correspond to the fastest route.

.getElementsByTagName('div')[0]

The parent div contains two <span> elements (which are not visible, by way of CSS-styling display: none), and one <div> element. Therefore, I imagine it's another fairly dependable and predictable array that of div elements, containing precisely one item (whose index is 0).

.textContent

Finally, the actual text written inside the child div element, which is our distance, "266 km".

2. GUI Scripting via AppleScript

AppleScript is usually my favourite method of doing many things Keyboard Maestro-related, but there's a very deep object hierarchy to traverse in the GUI. Still, depending on your needs and how you plan to implement your solution, it may be valuable.

First, I loaded up the map in Safari, then I ran this AppleScript in Script Editor:

	tell application "System Events" to tell process "Safari"
		set U to front window
		set T to {}
		
		repeat until (count U) is 0
			set U to a reference to every UI element of U
			set end of T to static texts in U ¬
				whose title contains "266 km"
		end repeat
	end tell
	
	return flatten(T)

which returns the following GUI static text object:

static text 1 of list 1 of group 1 of group 1 of group 14 of group 2 of  ¬
    UI element 1 of scroll area 1 of group 1 of group 1 of tab group 1 of ¬
    splitter group 1 of window "Dublin to Cork - Google Maps" of ¬
    application process "Safari" of application "System Events"

Like I said... A deep hierarchy. You can ignore the function flatten(); the array T is a nested list of lists of lists are mostly empty (they represent elements that are not static texts or do not contain "266 km"), except for the one that matches. I then just wrote a function called flatten() to flatten the array and retrieve that single item.

This single item is all I'm interested in. The code block I've written that retrieved it is inconsequential, and I included it just as part of my method. All being well, in similar Google Maps pages, that static text 1 object buried deep within the GUI hierarchy should be there containing the distance of the fastest route.

You then simply set a variable to reference that static text object, and then read its title:

    set T to title of static text 1 of list 1 of ...

which returns:

    "Driving 2 h 50 min 266 km via M7 and M8 Fastest route, the usual traffic Warning This route has tolls."

(Actually, it returns it twice—don't know why.)

It shouldn't be too difficult to predict what static text 2 will contain: a similar blurb for the second route in that list:

    "Driving 3 h 17 min 281 km via M9 and M8"

It's reasonable to assume other similar map pages will contain static text of similar syntax: "driving x h y min D km via etc..."

Therefore, you can isolate the distance in question by doing a regex match via shell script (using the variable T from above):

	do shell script "echo " & quoted form of T & " | egrep -o -e '\\d+ km'"
	return first paragraph of result
		--> "266 km"

The first paragraph is needed here because, for some reason as I mentioned earlier, static text 1 contains the same sentence twice over (that's Google's fault, not ours).


Getting the data back to KM

I'm going to assume that, even if you choose to use method 1, it will also be inside an AppleScript container, in which you can tell app "Safari" to do JavaScript "..." in front document or tell app "Chrome" to tell front tab of front window to execute JavaScript "...". Therefore, you'll end up with an AppleScript variable containing text that represents the distance of the fastest route.

In case you don't already know, to get communicated back through to and stored by Keyboard Maestro, use the following command:

    tell app "Keyboard Maestro Engine" to setvariable "km_var" to the AppleScriptVariable
3 Likes

This JavaScript in Front Browser should be reliable:

Example Results

JavaScript

var divElem = document.querySelector('div.section-directions-trip-distance div')
divElem.innerText

##Macro Library   Extract Distance from Google Maps Directions


####DOWNLOAD:
<a class="attachment" href="/uploads/default/original/3X/4/a/4aaf9c16575649a12e732dff0c7377fc252e7aa9.kmmacros">Extract Distance from Google Maps Directions.kmmacros</a> (2.8 KB)
**Note: This Macro was uploaded in a DISABLED state. You must enable before it can be triggered.**

---



<img src="/uploads/default/original/3X/1/a/1a0b8fbd71b3e08658e609116fa60dda8a3a9c0a.png" width="560" height="508">
3 Likes

Thanks @CJK for such a detailed and informative reply. I was waiting till I had time to dig into it. Is the section quoted above missing the end of the line? I presume that is getElementby but it could be a number of methods.
I'll enjoy looking into it. I've just started an online JS course, and a CSS course, so I'll enjoy trying to understand all of that.

@JMichaelTX That works great thanks. If I manage to turn it all into a neat macro, I'll post it up.

Once again the KM community is incredible.

I think you must not have scrolled all the way to the right when copying the JavaScript snippet. If you scroll back up where it appears originally in my first reply, it’s completely intact (at least, it is on my machine). But in case there’s something weird going on at your end that leaves you befuddled, here it is in plain text:

document.getElementsByClassName(‘section-directions-trip-distance’)[0].getElementsByTagName(‘div’)[0].textContent

That’s all one long string, with no spaces.

1 Like

Doh. What a rookie mistake!