Download Bank Statement Almost Working

Hey John,

Keyboard Maestro's do script will NOT work unless the macro it references is active, so unless said macro is global you also need to tell the relevant application to come to the front.

So if your macro is app-specific you need to do something like this:

tell application "Finder"
  activate
end tell
tell application "Keyboard Maestro Engine"
  do script "A05CD4DA-0D6B-4185-B38C-3A3X65F27ECD"
end tell

That doesn't enable anything; it just provides you with the script to use.

What is the file-type and the format of the file name for your statement? It shouldn't be hard to use AppleScript or the shell to get it.

The biggest problem is waiting for the download to finish. How long does it usually take?

--
Best Regards,
Chris

Hi Chris,
Thanks for responding. Ok... for your questions. The file type is .pdf. It usually downloads within second or two. I do have a pause inserted of 1.5 seconds. (I wonder about something to check here to see if the files exists but perhaps in version 2.0 heh!)

Today I got the email from the financial institution and it fired off the rule. The rule marks the email red (colorizes it). Then fires off the AppleScript.
Here is the AppleScript:

-- ignoring application responses
tell application "Keyboard Maestro Engine"
do script "76C89240-EB61-47F0-8007-F6D68EF28C9D"
-- or: do script "Financial_Institution_Name Statement Download"
end tell
-- end ignoring

I am not sure what you mean by "unless the macro it references is active". I do have the Macro enabled. Is that what you mean? I also see the folder for the "Global Macro Group" would I need to move it to that folder to make it Global? (I think I might have this part wrong though based on some forum reading I did last night.) Currently I have it in a folder titled "Work In Progress". (per Bwn or Boon in the my easy buttons videos on Youtube here).

The macro is app-specific I would think because I am using Safari to open the link (likely it's using the default browser). I also saved the password for the website and I am wondering if there's a more secure way to do this.

The steps are:

  1. Open URL using safari
  2. Pause for 1.5 second
  3. Click in the center of the found image "Sign In" button (Works really good!)
  4. Pause for 3 seconds
  5. Open URL for the statement page
  6. Pause for 1.5 seconds
  7. Click in the center of the found image "Print/Save PDF" button.
    ----(This downloads the statement as "InstitutionNameStatement" Which I want to insert a date and a space between the institution name and 'statement'. I suppose I am being really picky wanting to add the space but isn't that what the fun is all about?)
  8. Pause for 5 seconds.

This is as far as I have gotten. I intend to add the following steps...
--locate the file in downloads and
--rename it with the statement date (note the method Stephen mentions in this thread about using Hazel, something like that would be cool but the statement date has historically been the day the email went out so If I can date it with the current date it would probably be fine. e.g. "05/05/2015 FIN Statement" Where FIN is the financial institution name.)
--Move the file to a network volume, in this case a folder on a Mac I have downstairs that gets backed up.

I stopped after that then I posted this thread and started reading the forums. I also went through Boon's tutorials once again.

I hope I have answered all your questions.

Today when the statement email came through I got all excited when I saw it start to run but it stalled before the download step. Doh!

Hey John,

ACTIVE does NOT mean ENABLED in the Keyboard Maestro editor. It means ACTIVE in the current context of OSX.

Therefore global macros will ALWAYS be ACTIVE, but application-specific macros will NOT be active UNLESS that application is FRONTMOST.

Your Mail-Rule AppleScript must tell Safari to come to the front before it runs the KM-macro.

tell application "Safari"
  if not running then run
  activate
end tell
tell application "Keyboard Maestro Engine"
  # Financial_Institution_Name Statement Download
  do script "76C89240-EB61-47F0-8007-F6D68EF28C9D"
end tell

Does the file not have a .pdf file suffix?

How many assorted files are in your Downloads folder at any one time?

Getting this running should be pretty simple, but I need a snack and a nap before fiddling with it some more.

-Chris

Hey John,

Here’s the basic AppleScript code for finding your file, renaming it, and moving it to another folder.

-Chris

------------------------------------------------------------------------

try
  
  set destinationFolder to alias ((path to desktop as text) & "Financial_Statement_Destination_Folder:")
  set downloadsFolder to path to downloads folder
  set fileNameList to list folder downloadsFolder without invisibles
  set statementFile to missing value
  
  repeat with i in fileNameList
    if i starts with "InstitutionNameStatement" then
      set statementFile to alias ((downloadsFolder as text) & i)
    end if
  end repeat
  
  if statementFile ≠ missing value then
    
    set AppleScript's text item delimiters to "T"
    
    tell application "Finder"
      set creDate to first text item of ((creation date of statementFile) as «class isot» as string)
      set newFileName to creDate & " Institution_Name " & "Statement.pdf"
      set name of statementFile to newFileName
      move statementFile to destinationFolder
    end tell
    
  else
    
    error "Statement File NOT Found!"
    
  end if
  
on error e number n
  set e to e & return & return & "Num: " & n
  if n ≠ -128 then
    try
      beep
      tell application "Finder" to button returned of ¬
        (display dialog e with title "ERROR!" buttons {"Copy Error Message", "Cancel", "OK"} ¬
          default button "OK" giving up after 30)
      if ddButton = "Copy" then set the clipboard to e
    end try
  end if
end try

------------------------------------------------------------------------

Chris,
Got sideswiped by life. So it's been a while, forgive me. Thanks for your script example. I can already tell your script is much more sophisticated than what I was attempting to do. I also now understand using apple script to bring the application forward.

I will have to chew on your script for a while so I can understand what's happening.

So I take it that it's not as simple as letting the Apple script bring the application forward with a pause in KM then continuing in KM to run the PRINT command in OSX, then my keyboard shortcut for Printing to PDF, then save it to a specific folder with a pre-defined name?

I have a keyboard shortcut setup in OSX for the PDF menu which works once the print dialogue is open. Maybe this isn't necessary with your script.

It looks like your script is taking care of some of the stuff I was plodding through. :wink:

Hey John,

That's what automation is for — to keep us from having to look at the south end of a mule.   :smile:

You realize you can run AppleScript from the 'Script Editor.app'? (Applescript Editor in pre-Yosemite versions of OSX.) Using it you can actually see how a script works, by opening the Log History window and clicking on the 'Replies' panel.

I figured out how to write my own Print-PDF-to-Destination scripts for the Print-Sheet (dialog). See this thread:

It might be possible to write one of these, so the print-to script does the post-processing for you.

-Chris

No I didn't. But I do now. :smile:
So I ran it and got an interesting reply:

tell current application
	path to desktop as text
		--> "Mac HD:Users:jon:Desktop:"
	beep
end tell
tell application "Finder"
	display dialog "File alias Mac HD:Users:jon:Desktop:Financial_Statement_Destination_Folder: wasn’t found.

Num: -43" with title "ERROR!" buttons {"Copy Error Message", "Cancel", "OK"} default button "OK" giving up after 30
		--> error number -1708
	«event ascrgdut»
		--> error number -1708
	display dialog "File alias Mac HD:Users:jon:Desktop:Financial_Statement_Destination_Folder: wasn’t found.

Num: -43" with title "ERROR!" buttons {"Copy Error Message", "Cancel", "OK"} default button "OK" giving up after 30
		--> {button returned:"Copy Error Message", gave up:false}
end tell

So I assume from this I need to insert my own folder name which is really a folder that exists on my drive somewhere, would that be correct?

I should also add that I am saving everything to another mac in the house so...

I am wondering if it would be better to save everything to my local machine then either setup a watch folder script or add a mount volume statement which checks to see if the volume is mounted and if it is, saves it to the desired folder? I mention this because I end up waiting many times when I navigate to the folder on the other machine while it polls to list the files in each folder I have drilled down into. If the folder is several levels deep, sometimes I wait for each one quite a while.

I read the thread and downloaded your bash script along with the KM macro mentioned in the thread. So from what I gather instead of my Keyboard Shortcut (^P) for printing to PDF, I can execute your Keyboard Shortcut instead and everything will print to a desired name folder in my Downloads folder instead. I am slightly confused here, it appears you have two different Print workflows; one that prints to a folder and the other that prints to the Web Receipts folder. Did I miss something?

Hey John,

In the Bash script you need to define your own location.

Keep in mind that those are examples — not specific solutions for you.

It's been a while since I wrote that stuff. I'll eyeball it again and see if I can make things more clear.

-Chris

Chris,
I don’t want to be the newbie hanging on your apron strings. :wink:
If you have time that’s great and there’s certainly no hurry but if I can sort it out I will report back in case someone has that issue. We all have to earn our own stripes one way or another. No pain, no gain. Heh!

Hey John,

Your log shows the script erroring-out because there is no Financial_Statement_Destination_Folder folder on your Desktop.

You have to provide the proper location for the script to search.

You also have to provide the unique initial name of the files whatever that is. Like:

Bank Statement 01.pdf

The unique part would be "Bank Statement".

When working with scripts in the Script Editor you can comment-out lines using -- or # to keep them from running.

You can use block comments as well:

(*
The text you want not to run.
*)

OR you can just remove text from the script and run the partial script.

Beware of commenting-out essential parts of blocks though like tell...end tell or if... end if. If you comment out one part of the block and not the other then you'll create an error.

That's debugging 101.

Of course you need to learn a bit about what you're doing, so you can read scripts with an educated eye. They tend to look like so much Greek to begin with, but once you get a little bit of understanding you can puzzle-out what a script is doing — even when you don't completely understand the specifics.

Make sure you test scripts on TEST folders — NOT important data folders.

Once you get a script working satisfactorily you can work with real data.

Use copies of the real thing, so you don't make mistakes making mockups.

-Chris

Hi Chris,
Thanks for the feedback on the script. The "file alias mac" thing was probably a paste error when I copied it or maybe something from TextWrangler.

How does this work if I change the file path and folder location? What if it's on another computer?

Hey John,

Humph. That part was supposed to be deleted from my post, because I figured out where it came from. It was just the way the Script Editor was logging. I edited it out of the post just now.

You have to do something like this:

tell application "Finder"
  make new folder at desktop with properties {name:"JunkMe!.ccs"}
end tell

set myFolder to alias ((path to desktop as text) & "JunkMe!.ccs:")

I'm using the Desktop as a known anchor, and that will work on any system.

There are many path-to specifications. Many of them are listed in the Applescript Language Guide, but all should be listed in the afdr enumeration in the AppleScript dictionary of the StandardAdditons osax.

-Chris

Hey John,

I forgot to ask — what version of OSX are you using?

-ccs

10.9… Still haven’t gone to 10.10.

So my download script for credit card statements was working on one site but this month it didn’t fire off. At some point I must have disabled some steps and got distracted and never re-enabled them.

I set it up to work again and tested it. It seems to be back online.

I am going to see if it fires off next time but that’s a month away. In the meantime I thought I would add some more actions to the workflow. After the statement downloads the file is named "Financial_Institution_Statement. I want to scrape the date inside the .PDF statement and insert it in the name of the downloaded statement.

I want to append the scraped date to the beginning of the name as in:
"[MM]-[DD]-[YYYY} Financial_Institution_Name"
Is there a way to do that with KM?

Since the file is pre-named partially correct, I might also utilize a FIND to locate it. Then in the next step rename it using the date in front.


Moderator's Note 2022/11/19 14:09 CST:

Carsten Blüm's version of pdftotext is no longer available, but you can find a working version here.

See this section:

  • Download the Xpdf command line tools:
    • Mac 64-bit: download (GPG signature)

Hey John,

If you were on 10.10.x it would be fairly easy with ASObjC.

Since you're on 10.9.x I'd install this little Unix executable:

http://www.bluem.net/files/pdftotext.dmg?rev=0

The package installs it here:

/usr/local/bin/pdftotext

Invoke it in an Execute Shell Script action like this:

/usr/local/bin/pdftotext -layout -

Note the dash at the end — this is vital, because it caused the output to to go StdOUT instead of a file.

A working example on my system:

export PATH=/opt/local/bin:/opt/local/sbin:/usr/local/bin:$PATH;
pdftotext -layout "/Users/chris/test_directory/pdf_test_files/test.pdf" -

Turning it into an AppleScript that works on the selected file:

tell application "Finder" to set finderSelectionList to selection as alias list

if finderSelectionList ≠ {} then
	set posixFilePath to POSIX path of (first item of finderSelectionList)

	set shCMD to text 2 thru -2 of ("
		export PATH=/opt/local/bin:/opt/local/sbin:/usr/local/bin:$PATH;
		pdftotext -layout " & (quoted form of posixFilePath) & " - 
	")
	
	set pdfText to do shell script shCMD
	
end if

You can see what the output looks like by running this in the Terminal:

pdftotext -layout -

Make sure the trailing dash is on the end of the command, and run it.

-Chris

2 Likes

Love this Chris!

Since you mentioned ASObjC, are you using ASObjC Explorer 4?
It looks pretty nifty for debugging.

Hey John,

I do own a copy.

I also own Script Debugger, and I use that for regular AppleScript. (SD-5 doesn't support ASObjC though.)

-Chris

1 Like

Something tells me I should look into those. The script Debugger in Filemaker Advanced is so helpful it’s a no brainer to be looking at these as well. So many toys, so little time. :wink:

Edit 01/20/2016 4:27 PM… I was just looking over this entire thread and realized this is a gold mine of information. I am going to have to print it or clip it to Evernote or something.

Some time there is an issue with downloading the files.