Extract data from website

Questions & Suggestions
#1

Greetings,

I'm trying to extract a FEN (Forsyth-Edwards Notation)–which is a notation for describing the position of a chess board–from a website so I can then try to input them into another website (that's another step).

The long-term plan here is, I play a game, look at the analysis, see where I made a blunder or missed a tactic and run a macro that takes the position and puts it into another page on the website www.chessable.com which will allow me to drill the positions at a later date. It can't all be done with automation I don't suspect, but a lot can.

Here's a sample page: https://lichess.org/mVU0Rqj5#23 (that would be move 23).

If I 'inspect element' on the FEN (which is displayed on screen) using Safari I get this result:
<input readonly="readonly" spellcheck="false" class="copyable autoselect analyse__underboard__fen">
(Actually there is a '= $0' in light grey following in the inspector that doesn't copy and paste.

The actual FEN in this position looks like r1bqr1k1/ppn1bppp/2p2p2/8/2BP1B2/5N1P/PPP1QPP1/R3R1K1 b - - 0 12

If I search for the FEN in the inspector, I can see that each move is listed in FEN notation like so:

{
            "ply": 22,
            "fen": "r1bqr1k1/ppn1bppp/2p2p2/8/2BP1B2/5N2/PPP1QPPP/R3R1K1 w - - 12 12",
            "id": "KU",
            "uci": "a6c7",
            "san": "Nc7",
            "eval": {
                "cp": 23
            },
            "clock": 5547
        }, {
            "ply": 23,
            "fen": "r1bqr1k1/ppn1bppp/2p2p2/8/2BP1B2/5N1P/PPP1QPP1/R3R1K1 b - - 0 12",
            "id": "2:",
            "uci": "h2h3",
            "san": "h3",
            "eval": {
                "cp": 4
            },
            "clock": 5115,
            "children": [{
                "ply": 24,
                "fen": "r1bqr1k1/ppn2ppp/2pb1p2/8/2BP1B2/5N1P/PPP1QPP1/R3R1K1 w - - 1 13",
                "id": "WN",
                "uci": "e7d6",
                "san": "Bd6",
                "comp": true,
                "children": []
            }]

That's move 22, 23 and 24.
It also exists in two other places on the webpage.

Is there a way of extracting the FEN of the move I am looking at (like when you go to the first url I gave)?

Thanks!

#2

I've actually made progress here, but can someone explain why this macro works while saving the results to the clipboard, but not when to a variable?
Works:

Keyboard Maestro Export.png807×521 77.3 KB

Doesn't Work:

Keyboard Maestro Export.png807×521 69.3 KB

#3

This is my guess,

The variable is a text string
The clipboard handles RichText, Images etc

So the web page has RichText and therefore works with the Clipboard version.

1 Like
#4

Interesting theory and makes sense, but getURL action seems to capture the source code if I'm not mistaken, in plain text. It works now, so I'm presuming it was a small user error basically.
Also, I've changed some of the reg ex, so it could well be something to do with UTF codes or something when pasted to variable vs clipboard, like you say. Certain regex's were working on https://regex101.com but not in the macro, usually a character needed an extra escape key () to be recognised. Anyway I got there in the end! Many thanks!

Keyboard Maestro Actions.kmactions (2.7 KB)

Keyboard Maestro Export.png807×1420 173 KB