[WEB] Get URLs from Google Search Results [Example]

MACRO:   Get URLs from Search Engine Results [Example]

~~~ VER: 2.0    2019-10-26 ~~~
Requires: KM 8.2.4+   macOS 10.11 (El Capitan)+
(Macro was written & tested using KM 9.0+ on macOS 10.14.5 (Mojave))

UPDATED: 2019-10-26 18:46 GMT-5
Ver 2.0 is a Major Update

  • Adds option for using Search Engines other than Google
  • Includes LinkedIn with Option for "Other"
  • Prompts for Search Engine to use and other data

DOWNLOAD Macro File:

Get URLs from Search Engine Results [Example].kmmacros
Note: This Macro was uploaded in a DISABLED state. You must enable before it can be triggered.


Example Output

image
image


ReleaseNotes

Author.@JMichaelTX based on script by @ccstone

PURPOSE:

  • Get List of URLs (or MD Links) from Search Engine Results
  • Works for both Google, LinkedIn, and Other

HOW TO USE:

  1. Do a Search in any Browser Supported by KM
  2. Trigger this macro
  3. Select the Search Engine

WHAT IT DOES:

  1. Scans the Search results, and returns one link per result
  2. IF GS__Return_Data is set to "MD", then it returns Markdown links
  • Default: URL
  • Markdown Format: [Link Text](URL)
  1. The max number of links returned is set by the Variable GS__Max_Links
  • Default: 10

MACRO SETUP:

  1. Review the Default Settings in the Prompt and change as desired
    • GS__Max_Links
    • GS__Return_Data
  2. Review the Actions in magena color, and change as desired
    • GS_XPath for "OTHER"
  3. ADD the Actions you want to process the results (list of URLs)

TAGS: @XPath @Google @Search @Links @JavaScript

USER SETTINGS:
• Any Action in magenta color is designed to be changed by end-user

REF:

ACTION COLOR CODES

• To facilitate the reading, customizing, and maintenance of this macro,
key Actions are colored as follows:
• GREEN -- Key Comments designed to highlight main sections of macro
• MAGENTA -- Actions designed to be customized by user
• YELLOW -- Primary Actions (usually the main purpose of the macro)
• ORANGE -- Actions that permanently destroy Varibles or Clipboards

REQUIRES:

(1) Keyboard Maestro Ver 7.2.1+
(2) Yosemite (10.10.5)+
(3) Any Scriptable Browser Supported by KM

1 Like

A post was split to a new topic: How would I get the line number of Selected Text in a variable?

@JMichaelTX This is amazing, how tricky would this be to alter it for a different search engine/website?

Depends on how the results page of the other search engine is designed.
The more similar it is to Google, the easier it will be.

Would you mind taking a look at the linkedin search engine if it's not to time intensive?

It's just the //--- SETUP XPATH --- and //--- GET FIRST LINK --- sections that would require alterations, correct?

Just posted a major update.

Just posted an update which should meet your needs.

Amazing, thanks so much Jmichael

1 Like

@JMichaelTX

I'm currently working on implementing this with a site that lazy loads the search results, is there anyway in keyboard maestro to force load all the results beyond implementing a scroll and pause sequence?

If by "lazy loads" you mean the page stops loading until the user scrolls to the bottom of the current page, then no, I don't know of any other method. This is exactly how this site (KM Forum / Discourse forum) works. It's a PITA. :wink:

You might try posing your question on the stackoverflow.com site.

1 Like