Help with Downloading Hundreds of Files?

[Slugworth]

Hi y'all. I'm new to scripting and KM, but I think it would be a useful tool. If I make it through the learning curve, I think it will be a worthwhile tool. I'm currently trying to download hundreds of files from an academic database. For example, I have over 300 newspaper articles that I can download. There is no function through the database website to download PDFs in batches.

Each page contains the article with 3 different possibilities: 1) Download PDF; 2) Download Page PDF; 3) Save as PDF.

Since some news articles are scanned into the database they are only available as a PDF or Page PDF. More modern news articles only have the "save as PDF" because the database has their text available on the webpage you can simply save the page as a PDF. Confusing and probably quite outdated, I know. But such are academic databases.

In cases were both Download PDF and Download Page PDF are available, I would like to click the first. In other scenarios, either the Download Page PDF or Save PDF buttons will likely be stand alones (see photos).

That said, I would like to make a macro with the conditions that if Download Page PDF is available that I select that option and click next page to go to the next article in the search result. Where I am stuck is setting the conditions that when the Download Page PDF or Save as PDF functions are available those are clicked instead.

I've tried to create marcos using "click on found image". Then I realized that "click google chrome link" might be better. The problem is that I cannot figure out how to map the conditions to get this program to work when Download PDF is not present I only have the option "Download Page PDF" or "Save as PDF." Then I set the mouse to move over to the next button and have it repeat. It worked for a while but then would often get jammed up somewhere when I got to a page that only had the "save as pdf" option.

I've attached some pictures so that you might be able to better understand my issue. I'm sure this is an easy one, but I'm a bit stuck. Any help is much appreciated and would save me hours of my life.

[tjluoma]

I have a similar database of journal articles that I use.

FWIW - I found that using the “Email” feature was the easiest to automate, and I sent them to my Send to Dropbox address.

Of course, that relies on the actual PDF being included with the email. Otherwise, it’s not much use if they are just sending you an URL or something.

[tudor_eynon]

You might find you are up against a barrier put there exactly to stop you doing batch downloads for reasons connected with 'fair use' policies and so on. The question is, of course, why do you need so many downloads? I appreciate that is now the default a lot of academics take, often, as far as I can see, to do with citation counts and so on rather than actual research.

[Slugworth]

This has nothing to do with citation counts. I'm downloading archived newspaper articles for my research. I just want to be able to download the PDFs quickly and efficiently. These are not being reproduced in anyway, they are being analyzed for certain research projects.

[ccstone]

@Slugworth

Any help is much appreciated and would save me hours of my life.

Hey @Slugworth,

This kind of task is difficult or impossible to help with, unless we have access to the original web page.

If you can post that – so we have something to test with – we might be able to help.

-Chris