How to automate the generation of lists of things? ... with ideas

[Airy]

I want to make my KM macros more dynamic, without having to hard-code a lot of things. For example, I want my KM macro to fetch from the Internet series of strings like "the names of the planets", "the NATO phonetic alphabet", "the colours of the rainbow," "the names of the 50 US states," etc.

But when you use any search engine to do a search, the results are only a very small part of the page's results, often in inconvenient formals like multi-column tables. There's no way for me to easily isolate the actual answer programmatically. I'm unaware of any web search engine that does what I want - "just return the results". I even asked ChatGPT today, and it gave incorrect answers.

When Google first created Google Sheets, it had a feature that did what I wanted. For example, if I put the first items into a Google Sheets document, say, "Mercury", "Venus", and "Earth", I could select those words and stretch them out, and it used it's search engine savvy to populate the next fields with the remaining words, whatever they would be. As another example, I could put the words, "Kermit" and "Miss Piggy" in the first two cells in a table, and then when I stretched the table it would insert more names of muppets. This is very close to what I want. Perhaps with a little AppleScript programming a person could create a Google Sheet, insert a few terms, stretch the cells, and copy the results back to a KM variable.

Unfortunately I seem to have forgotten how to get Google Sheets to do this text expansion. Perhaps they have removed this feature, or hidden it in a submenu, or moved it into their "add-ons". Does anyone know where that feature is now? I've searched Google for "how do i get google sheets to produce similar items in new cells" and couldn't find the results. I've searched Google Sheets help for the same idea. I've searched their add-on library but found nothing.

STOP THE PRESS - as I write this post, I found an amazing function in Google Sheets that almost gives me what I want. The function is:

=IMPORTHTML("https://en.wikipedia.org/wiki/NATO_phonetic_alphabet","table",1,"en_US")

If you place the URL of the page that contains the data you want, this function will go to the page, fetch and isolate the data(!!!!!!), and place it into the cells of the spreadsheet. I see two downsides: (1) if the data in the web page is multi-column, it returns multiple columns of cells. (2) instead of a general web query, you have to use the actual URL of the page that contains the data.

I can probably live with problem #2, and I think I can use some KM programming to live with problem #1.

Google Sheets has a few functions like this that I never knew about. I'm really amazed. I think I may have to reconsider my aversion to using Google to process my data. The IMPORTHTML function also was able to return results from more complicated pages that I tested, but often with errors. I can mitigate these errors by finding the URL of a simple page that contains the information I need.

Remember, this is all being done to get my KM macros to be able to work more autonomously, without having to hard-code lots of constants. For example, I would like to create a KM macro that takes a short list of items, and uses the methods above to expand that short list into a longer or complete list. This would be a pretty awesome macro.

[griffman]

This will be a very complicated problem to solve :). The problem will be finding a reliable data source that provides data in the form you need it, at a price you're willing to pay. WolframAlpha, for example, has an API with a number of interfaces available (simple, short answer, LLM, etc.). Using the LLM API, you can write a query like this:

https://www.wolframalpha.com/api/v1/llm-api?input=a+list+of+the+planets&appid=your_api_number

And that would spit back this:

Query:
"a list of the planets"

Assumption:
Assuming "planets" is referring to planets
To use as  referring to exoplanets set assumption=*C.planets-_*ExoplanetClass-
To use as  referring to minor planets set assumption=*C.planets-_*MinorPlanetClass-

Input interpretation:
planets

Members:
Mercury | Venus | Earth | Mars | Jupiter | Saturn | Uranus | Neptune (total: 8)

Orbital properties:
image: https://www6b3.wolframalpha.c

You could then easily process this text to extract the list of planets. Wolfram gives you 2,000 requests a month for free, and after that, you'd have to pay. So you wouldn't want this query in a macro that runs on a 10 second interval :).

The LLMs themselves have APIs, but I don't believe any of them are free to use. Claude's API is separate from Claude itself, so even if you pay for Claude, you don't get free access to the API.

There are probably dozens or hundreds of similar sources out there, but it's going to be very hard to find one that does exactly what you need, I think.

-rob.

[Airy]

@griffman

Using the LLM API, you can write a query like this:

https://www.wolframalpha.com/api/v1/llm-api?input=a+list+of+the+planets&appid=your_api_number

Wow, that's cool. That's easier to work with than Google Sheets. Yes, I could extract the results fairly easily with a little KM code. Yes, I can find a way to make sure I don't run this 2,000 times per month.

@griffman

This will be a very complicated problem to solve :).

Challenge accepted. Actually, the Wolfram API that you showed me should make this fairly easy. Easier than a lot of problems we've already solved using Keyboard Maestro.

[Airy]

This opens up an exciting new world for me. I applied for a "Simple Response API" from Wolfram Alpha after registering my email address, and within 5 minute I had built a KM macro that listened to my spoken words and produced a spoken result coming straight from Wolfram Alpha. I can't share my personal API key with you, so I've modified my macro to let people insert their own API and try this macro. Here is the macro. It loops, letting you ask questions, until you say the word "abort."

Speak with Wolfram Alpha, and get Spoken Results Macro (v11.0.3)

Speak with Wolfram Alpha- andget Spoken Results.kmmacros (5.9 KB)

In order for this to work, you also need to create a macOS Shortcut with the following name and content:

If you do these things, you will be able to talk to Wolfram Alpha's API, and get spoken results back.

Yes, there is an annoying beep every 5 seconds in my macro. And yes, Wolfram Alpha's domain is mostly in the math and science area, but it can also answer a lot of basic questions in geography, politics, and other non-science domains.

If I can make the spoken input more reliable, I think I might create my own virtual assistant to replace "Hey Siri."

[vincent_ardern]

I’m surprised chatGPT didn’t get it for you. Did it fail on your listed examples, or something more complex?

[Airy]

I did look at ChatGPT, but it failed for two reasons. (1) The ChatGPT app on macOS is very slow. It seems to take about a minute for ChatGPT to "type" an answer. Wolfram never takes longer than a second, and usually less. (2) ChatGPT includes a ton of superfluous information in its output. At the beginning and ending of every answer is an explanation that has nothing to do with the answer itself. For example, in this question I specifically asked for 26 words, but got about 100 to 200 words.

...etc...

By comparison, Wolfram's answers are virtually instantaneous, and don't include lots of fluff at the beginning and end of each answer.

Also, ChatGPT tends to put answers in tables, with extra information that I didn't ask for. I would need to write a lot of KM code to strip out all the garbage that I didn't ask for. That can be very complicated to do.

@vincent_ardern

Did it fail on your listed examples, or something more complex?

Actually, I need something LESS complex than ChatGPT's flowery verbal diarrhea. Wolfram's answers are simple and succinct.

[vincent_ardern]

I can't help you with the speed, but prompts are an art form. See screenshot.

[DanThomas]

I would think you could tell ChatGPT not to include the summary. I've told it to do or not do a lot of things, and it's been helpful in cleaning up the responses.

But that's the only help I can offer.

[Airy]

That's a great idea. Even so, now there's the problem of getting a KM macro to read the results. How do you recommend I approach that? What key sequence will work?

[vincent_ardern]

I use AI Actions personally

[griffman]

That requires a paid API key, right?

" The app requires you to specify your own OpenAI / Anthropic API key.
Note that ChatGPT Plus or an Anthropic paid account does not give you free API access."

-rob.

[vincent_ardern]

Correct. I find the API key route to be extremely cost effective (i.e. dirt cheap)

[DanThomas]

Wish I understood what you guys were talking about. I don't deal well with abstract ideas, so I can't wrap my head around this stuff. It's a problem I've had my whole life.

[vincent_ardern]

Not sure if you're looking for a suggestion or not, but you could copy and paste this thread into a (free) chatgpt conversation and keep on asking it questions like "explain in detail what these guys are talking about" until it makes a bit more sense Or I could have a go at explaining it, or we could just leave it and acknowledge that that's probably frustrating for you.

[Airy]

Everyone knows he's a genius. If there's one post he doesn't understand, like this one, that just means he's human.

[DanThomas]

@Airy

Everyone knows he's a genius.

Well, that's a laugh. Yes, I've created some cool things, but that doesn't make me a "genius", that's for sure! But thanks for the compliment!