Grabbing text from a website and saving to KM as a variable for further actions

Questions & Suggestions
#1

Hi,

Using KM I want to grab details from a webpage and then save the details as KM variables, time is of the essence as a result is it possible to asynchronies savings the details as KM variables? https://bittrex.com/api/v1.1/public/getmarketsummaries
To speed things up can this be done without loading safari by using some sort of ping in the background?

Raw version of the DATA:

{“MarketName”:“BTC-1ST”,“High”:0.00005300,“Low”:0.00004240,“Volume”:626763.25047179,“Last”:0.00005063,“BaseVolume”:29.70606936,“TimeStamp”:“2017-10-24T09:55:36.697”,“Bid”:0.00005042,“Ask”:0.00005149,“OpenBuyOrders”:169,“OpenSellOrders”:5855,“PrevDay”:0.00004542,“Created”:“2017-06-06T01:22:35.727”}

KM Variables version of the Data:
Variable name: String

Market Name BTC 1ST: BTC-1ST
BTC 1ST High: 0.00005300
BTC 1ST Low: 0.00004240
BTC 1ST Volume: 626763.25047179
BTC 1ST Bid: 0.00005042
BTC 1ST Ask: 0.00005149

Is this possible if so how, I am not a programmer by the way :slight_smile:

#2

Hey Ali,

It's simple to get the data using an Execute a Shell Script action.

curl -L --user-agent 'Opera/9.70 (Linux ppc64 ; U; en) Presto/2.2.1' https://bittrex.com/api/v1.1/public/getmarketsummaries

You can also use the new Get URL action in Keyboard Maestro 8.x:

From there it's not too difficult to parse the data.

-Chris

1 Like
#3

Many thanks for this Chris.

How would I start saving the results to KM?

#4

Hey Ali,

Both the Execute a Shell Script action and the Get URL action can save to a variable.

-Chris

#5

Sorry I was referring to sorting the results to KM Variables after saving the result of this action to a KM variable

#6

I’m not completely sure if I understood correctly what you are looking for.

This will export each of the 6 keys as indicated in your post #1 to individual variables. Afterwards your variable inventory will be well populated like this:

Text from Website to Individual Variables.kmmacros (3.4 KB)

<img src="/uploads/default/original/3X/a/c/accd2a887bd849da485e8bbb9acc319e5bf5c686.png" width="597" height=“709">

PS:

Here is a “Pretty Print” version of the macro:

Text from Website (Pretty Print).kmmacros (3.3 KB)

It will output the data like this:

3 Likes
Delete All Variables Except…
#7

Hi Tom, many thanks for your assistance. Which language is this? am I right in assuming that this is JavaScript?

#8

Yes. But you can do it in any language, including KM itself. However, since your data is already a JSON string the built-in JSON parser of JavaScript is very convenient.

#9

Many thanks for this Tom. Evidently what I am trying to do is not available as simple KM actions.

#10

Of course you can do to with KM actions only, too:

Untitled-pty-fs8.png1270×7342 347 KB

Text from Website to Individual Variables (KM pure).kmmacros (10.6 KB)

It’s a bit clumsy, but it works. (Only lightly tested!)

The idea is:

  1. Split up the data to lines (for looping)
  2. Parse the desired values with a regex

The titles of the groups explain what the actions are for.

#11

@demirtas1

Just a general thought:

With the input from your source URL the macro is creating about 1600 variables. So, I think it’s not exaggerated to say that you are misusing KM as a database :wink:

Considering your follow-up post where you are trying to query the data, wouldn’t it be better to export the data to a CSV text file which can be read by Excel for example? Or to a proper database which you can properly query for whatever you need?

#12

I am supposed to match then make the action :), no intention of creating a database. It should actually be about 10% of the current variables created.

#13

@demirtas1, and any readers: I agree with @Tom.

It is inadvisible to create a lot of KM global variables, unless you really need them. And it is unlikely that you need 100’s of variables. As Tom says, if you find that you think you need 100s of variables, for just one macro (or a set of related macros), then you may want to reconsider your design.

If you need to store data for 100s of variables, then some type of database is indicated, or at least one file with all the data in something like a plist of JSON format.

If anyone has questions about this, feel free to ask.

#14

The irony here is that the data grabbed from the web page already is a JSON string :wink:

In case you are going to try out the macro: I have posted another macro that deletes all the hundreds of created variables pretty fast.

1 Like
#15

Good point. Thanks for pointing this out. I had forgot where the source came from.

BTW, if it were me, I’d put this data in a SQLite database. @ShaneStanley has recently released SQLite Lib, an AppleScript Script Library that makes accessing a SQLite database very easy and fast.

1 Like
#16

Using KM and the JSON data how can I match the coins trading in more than one market and do an action for each matching MarketName? e.g. ‘1ST’ exists in ‘BTC-1ST’ match to see if it also exists in ‘ETH-1ST’ or ‘USDT-1ST’.
How can I do a match in KM to retrieve all coins (1ST and more trading in more than one exchange (BTC, ETH or USDT).
Preferably I would like the MarketName to be returned to do an action for each.

{“MarketName”:“BTC-1ST”,“High”:0.00005300,“Low”:0.00004240,“Volume”:626763.25047179,“Last”:0.00005063,“BaseVolume”:29.70606936,“TimeStamp”:“2017-10-24T09:55:36.697”,“Bid”:0.00005042,“Ask”:0.00005149,“OpenBuyOrders”:169,“OpenSellOrders”:5855,“PrevDay”:0.00004542,“Created”:“2017-06-06T01:22:35.727”}

#17

I’m afraid I can’t help you with the JSON data, but in case you missed it, I did show you in another thread how to assemble a list of variables with matching suffixes that you can probably use or modify to include further For Each actions for matching sets: How to compare 2 string variables? (if nameA == nameB)