Convert URL contents to PDF

I’m very new to Keyboard Maestro and I’d like to get some help with setting up a macro that does the following:

  1. Select links on a webpage
  2. Open them
  3. Convert the pages into PDF
  4. Save them with a meaningful title to Dropbox

I’m currently using a workflow in Workflow.app on my iOS devices that does exactly this. I use it to take a bunch of newsletters I like and convert them to PDF. The problem is that it only works with Safari (I prefer Chrome), and the operation times out when the device goes to sleep.

If someone can start me off or point me in the right direction, I’d be super grateful.

You probably know this, but just in case . . .

If you print a web page, you have the option to save as PDF.
You should be able to do that with KM Actions.

To select the links on a web page, here is one example that @peternlewis just posted:

Yes I did know that and I’m now trying to wrap my head around the macro that Peter posted. The part that I’m getting stuck is figuring out how to select the list of links, get their URL and open up each page. I can figure out how to save them to PDF after that.

I do not know or understand JavaScript but know a little bit of HTML and I understood the document.getelementsByClassName piece and I can figure that out for my target webpage. I’m not sure how to go about iterating through each of the links found in that class. Any pointers there?

This was taken from @peternlewis' above macro. It is just a guide. You will need to adjust as needed to fit your setup/requirements. But it should get you started.

If you don't understand any of these steps/Actions, please feel free to ask for more info.

Ok here is where I'm stuck. I can get Chrome to open up the target URL, but nothing seems to be happening after that. I tried using the debugger to troubleshoot but it escapes after hitting the execute JavaScript block.
Can you help me figure out where I'm going wrong? I'd also like to know how to debug macros going forward!

Any help is much appreciated. I'm learning slowly and I appreciate the assistance from the forum. I'll share the completed workflow once I'm up and running.

Get URLS and create PDF.kmmacros (7.3 KB)

I’ll be glad to try, but I need more information.
Could you please post the contents of the “linklist” variable in a code block?
You can use this format:

 ```html
 <put your text here>

If "linklist" is a very long list, then you need only post enough to cover the first few links.

Here you go. I grabbed the block that has the links. I hope this helps. Thank you so much for helping troubleshoot this for me.

<div class="post-content">
        <h2 id="macstories-weekly">MacStories Weekly</h2>
<h3 id="2016">2016</h3>
<ul><li><a href="http://us8.campaign-archive1.com/?u=9f4b80a35728f7271fe3ea6ff&amp;id=c77b459b37">Issue 61: Friday, December 16</a></li>
<li><a href="http://us8.campaign-archive2.com/?u=9f4b80a35728f7271fe3ea6ff&amp;id=5800ed33e4">Issue 60: Friday, December 9</a></li>
<li><a href="http://us8.campaign-archive2.com/?u=9f4b80a35728f7271fe3ea6ff&amp;id=a5942c99a3">Issue 59: Friday, December 2</a></li>
<li><a href="http://us8.campaign-archive2.com/?u=9f4b80a35728f7271fe3ea6ff&amp;id=0cbb7b69c9">Issue 58: Friday, November 25</a></li>
<li><a href="http://us8.campaign-archive1.com/?u=9f4b80a35728f7271fe3ea6ff&amp;id=8628452e0e">Issue 57: Friday, November 18</a></li>
<li><a href="http://us8.campaign-archive1.com/?u=9f4b80a35728f7271fe3ea6ff&amp;id=52ea5bb0f6&amp;e=79be37a0b8">Issue 56: Friday, November 11</a></li>
<li><a href="http://us8.campaign-archive2.com/?u=9f4b80a35728f7271fe3ea6ff&amp;id=d90450578c">Issue 55: Friday, November 4</a></li>
<li><a href="http://us8.campaign-archive2.com/?u=9f4b80a35728f7271fe3ea6ff&amp;id=a8f3205f0e">Issue 54: Friday, October 28</a></li>
</li>
</ul>
    </div>

OK, thanks for posting.
I’m very busy today, so it may take me a day or so to get back to you.
But this info really helps. I should be able to give you an answer/fix.

Anyone have any thoughts on where this is getting stuck?

@ssheth, here’s a quick update:

I think I have solved the basic problem. I’m now putting the pieces together into a hopefully workable macro.

I should be able to upload the macro by end of today (hopefully much sooner).

OK, based on all of the follow-up info you have provided, I think the below macro should work. Please note that I cannot fully test it because I do NOT have access to the main web page you want to use:
https://club.macstories.net/newsletter-archive/

However, I have fully tested it on other web sites, and it works for them.
But, you may need to make some adjustments.
Also note that I have tested this ONLY on Keyboard Maestro 7.3.1 (7.3.1) on macOS 10.11.4.

If you run into any strange problems, please post the details/screenshot, and I'll try to help you debug/fix them.

Please feel free to ask any questions about the macro and/or my approach.


###Example Results
After saving each web page as a PDF, the following are displayed:

####Summary List of Links/Pages

####Finder Opened to the Output Folder


###MACRO:   @PDF Save (Print) Web Page for Each Link on Web Page as Individual PDFs [Example]

~~~ VER: 1.1    2016-12-21 ~~~

####DOWNLOAD:
@PDF Save (Print) Web Page for Each Link on Web Page as Individual PDFs [Example].kmmacros (33 KB)


###ReleaseNotes

Author.@JMichaelTX

PURPOSE:

  • Extract HTML Links from a Web Page, then Print as PDF the Web Page for Each Link

MACRO SETUP:

  1. Enter the main web page URL in the Action
    "OPEN URL of Web Page Containing List of Links"
    .
  2. Enter the JavaScript to Extract the HTML from this page in the Action
    "SCRIPT: Extract Link List from Page (JS)"
    .
  3. Enter the Folder for PDF Output in the Action
    "SET the Full Path of the Output Folder for the PDFs (must exist)"

TAGS: @PDF @Web @Links @Chrome @Example

USER SETTINGS:

  • Any Action in magenta color is designed to be changed by end-user
  • This macro uses Google Search and Google Chrome, but can be easily changed

ACTION COLOR CODES

  • To facilitate the reading, customizing, and maintenance of this macro,
    key Actions are colored as follows:
  • GREEN -- Key Comments designed to highlight main sections of macro
  • MAGENTA -- Actions designed to be customized by user
  • YELLOW -- Primary Actions (usually the main purpose of the macro)
  • ORANGE -- Actions that permanently destroy Varibles or Clipboards

REQUIRES:
(1) Keyboard Maestro Ver 7.2.1+
(2) Yosemite (10.10.5)+


Michael,
Many many thanks for taking time out to help me figure this out. It is obvious you have a great deal of expertise and mastery of CS fundamentals and concepts.

I by no means claim to be an expert in any of this so I truly appreciate the time and knowledge that experienced folks like you share.

I have a few questions about why you are doing some of the things that you are in the macro so that I can learn along the way:

  • What is a semaphore lock and why and when do you use one?

  • Is there any particular resource that you can recommend about how to debug KM macros?

  • What is getting written to the global KM variables that you have to delete them after the link block gets processed?

  • Reading through the macro it is abundantly clear that what I was trying to achieve was way out of my expertise, since this involves a fair deal of AppleScripts. Could this have been achieved without the scripting portion of the macro?

  • Are there any best practices of when to use JXA, AS, JS, Python, Shell etc for achieving a particular result, or is the choice mainly dictated by what one knows? I would like to start somewhere and would like to know if you have any recommendations for a language to use with KM to achieve some of the wonderful things it can do.

  • Are there any resources for JXA and AS in particular that you’d recommend for people starting out?

Once again, many many thanks for helping figure this out.

Happy Holidays!

No problem, glad to answer them. Most likely others with have the same questions.

It prevents the execution of another trigger of the same macro, until the current execution completes. This macro can take a while to complete, so I did not want another trigger of it to happen while it was executing.

See

The primary reason for deleting KM Variables used in a Macro is to remove clutter from your KM Variable list. If the Variables are not needed later, they just get in the way.

Actually, I think there are only 3 scripts, and all are needed.
They are the best way to accomplish the objective, and I don’t know of any KM Actions that could have replaced them.

If there is a KM Action that will accomplish my objective, I always try to use that. I only revert to scripts if it can’t be done, or done easily/effectively with non-script KM Actions.

I generally stay away from Shell Scripts, mainly because of my ignorance about them, but also because they are harder, generally, to customize, unless you are an expert. A small change in a Shell Script can lead to loss of data if you are not careful.

I think each of us uses the scripting language we are more familiar with and most comfortable with. I know AppleScript the best, but I’m trying to to learn and use JXA as often as I can. If you don’t know either AppleScript or JXA, then I’d recommend starting with JXA.

See JXA Resources

Hope this all helps!

How did you know and find the semaphore that you needed to lock?[quote=“JMichaelTX, post:13, topic:5760”]
If you don’t know either AppleScript or JXA, then I’d recommend starting with JXA.
[/quote]

Your wiki is an excellent resource. Many thanks for sharing.

Thank you so very much. All of this definitely helps.

-Sid

The semaphore name is just an arbitrary name that you assign when you add the KM Action to your macro. Just pick a name that will be unique to your macro.

Adding to @JMichaelTX’s response to this:

The name of the semaphore can be anything. It just needs to be unique - in other words, you don’t want another Semaphore action to accidentally use the same name. But even if another one did use the same name, it would only matter if both macros ran at the same time.

If you want to know more about Semaphores, just ask. I have a personal affinity for Semaphores, having done a ton of multi-threading work in my life. You don’t need them very often, but when you do need them, they’re way awesome! :slight_smile:

1 Like