Let's say I have this path:
/Folder 1/Folder 2/Folder 3/File.txt
Now I want to copy File.txt to Dropbox, for example, but I want KM to automatically create the structure for me, meaning, it will create all the extra folders so I would end up with this:
/Dropbox/Folder 1/Folder 2/Folder 3/File.txt
Is this possible?
If so, would I be able to create a macro where I paste a list of paths and it will create all of those for me the same way?
I think part of the problem is solved:
rsync -R [source] [destination]
So it would be:
rsync /Folder\ 1/Folder\ 2/Folder\ 3/File.txt /Dropbox/
Now the part with a list and variables, I will try it tomorrow when my brain is back to normal, but I'm already checking it:
action:Execute a Shell Script [Keyboard Maestro Wiki]
If you know the answer, please do not share it just yet. Let me see if I can get there myself
Just a list of arbitrary paths?
Sure.
Before you go down that path, tag a file then rsync it. Does the tag copy too? Will that be important?
Then take a look at ditto, which (by default) respects macOS extended attributes, metadata, etc.
Yesterday I saw an article about ditto, but when I tested it, it didn't copy the folder structure the same way rsync - R does. Basically it just copied the file to the destination
It copied File.txt to /Dropbox/ like this /Dropbox/File.txt
Instead of /Dropbox/Folder 1/Folder 2/Folder 3/File.txt
I tried it today and yes, ditto preserves the tag, whereas rsync doesn't. That's a minor issue I will have to live with, if ditto doesn't actually allow me to copy the structure, which is the most important part for me.
I was looking at the different options I can add when using ditto, but I can't seem to find one that does what I need. Am I missing something?
Ok so after a long battle trying to understand how things work, because of my 0 knowledge of scripting, I was able to make it work using rsync -R
Test Copy Files and Structure.kmmacros (22 KB)
Now I can't seem to figure this out, even though it's not relevant to this particular situation, but just so I can learn it: if my destination path inside the script uses a tilde, it will not work. What should I add for it to accept the tilde?
One workaround would be to set the destination path as a variable and then use that in the script, but I would like to know if there's a way to do it without the variable?
I just made it! I think everything's good now. Can you check it above?
So now I'm noticing that a particular type of file is not being copied properly and that is, Logic Pro packages (maybe this will be an issue with other file types?). Even though Finder shows them as files, they are more like a folder. When I right click and choose Show Package Content:
If I try to copy the original package (.logicx) using rsync -R, it doesn't copy at all.
If I instead try to copy using ditto, those 3 folders get copied individually, so instead of having a .logicx file only, I get this:
When I should have this:
Any suggestions to fix this?
rsync copies the folder structure, but doesn't copy Logic files
ditto doesn't seem to copy the folder structure, copies Logic files, but the wrong way
UPDATE: It seems that using rsync -R and rsync -r produce different results. Using -r now copies Logic files 
Are there any other "issues" that I should be aware of when using rsync for other file types or something? I'm building this macro as a quick "backup" solution (besides my normal CCC Backup strategy) and I want to make sure all files get copied.
Is it possible to create a variable with the total amount of files KM actually copied?
Well, of course as they relate to two different options for rsync. Have a look at the rsync Man Page - macOS - SS64.com
-r means rsync will recurse into directories, hence it will process folders within folders…
Yeah, I'm still in the very beginning of even understanding what this whole thing is when it comes to scripting. Thanks for the link
I just noticed that I can use more options together
rsync -r copies Logic files, but now I noticed it doesn't copy the folder structure
rsync -R doesn't copy Logic folders, but keeps folder structure...
So I had to use rsync -rR
It's now working!
All I need now is to find a way to check how many files were actually copied.
I think I'm on my way there. Maybe using "Append Variable" after the script? And at the end count the number of lines in that variable?
Instead of that you can try “save results to a variable” in the execute action and then you can do whatever you need to that variable.
Also (you really should read that man page) rsync has a —stats (that’s two hyphens followed by stats) option to show basic file transfer statistics; I don’t know if that will help but you can try it… I’m away from my Mac so can’t test for you, sorry!
Awesome. Actually the Append Results to a Variable is what I need, because I'm using the For Each action so for each path it will copy the file and append the result to a variable. Let me try it...
Will check that as well. Thank you for that extra bit of information!
For some reason it's not working.
I tried different options such as "display results large/briefly/in a window", but nothing comes up
Why would it work when (from the man page) “By default, rsync works silently”? So by default there won’t be anything to display!
Try adding the -v option to the command line: rsync -rRv etc. (v for verbose, BTW)
Thanks for the tip.
Unfortunately that didn't help, because when I append the results to a variable, it shows me way more lines than files were copied, because the result are several lines with detailed information.
No big deal. Most of the workflow is working and if I have to check the number of files another way, I will. I appreciate the help!
In Terminal, type man rsync to get its manual page. That way you get the one for the utility included in your macOS version -- they don't often change, but it does save you from wasting time when a google leads you to the page for the linux version instead...
The same works for most Unix utilities: man ditto, man cp, etc.
r is "recursive" -- as you've found, you need that to get the sub-directories of the thing you are copying. R is "use relative paths", which is what keeps your "parent" folder structure.
To do similar folder structuring in ditto you'd usually set a sourceBase and destinationBase and create the "copy to" path from the file path by
sourceBase of the frontdestinationBase to the frontSo if you wanted to copy ~/Documents/Projects/A/B/myfile.txt to the (currently empty) ~/Dropbox/Project Backups/ to get ~/Dropbox/Project Backups/A/B/myfile.txt you'd set sourceBase to ~/Documents/Projects/, destinationBase to ~/Dropbox/Project Backups/ and then "Search thePath for sourceBase and Replace with destinationBase", using the result in ditto.
rsync is obviously less work! So yes, use that if you don't care about Mac stuff like labels and other metadata.
This will crop up a lot in your shell scripts, so is worth remembering: ~ is not expanded when your path is in double-quotes. As you found, the KM solution is to expand the path before using it in the shell script.
Use the --stats option, as @tiffle suggested above, and don't use -v. If you "Save results to variable" you'll get similar to
Number of files: 5
Number of files transferred: 1
Total file size: 426405 bytes
Total transferred file size: 426405 bytes
Literal data: 426405 bytes
Matched data: 0 bytes
File list size: 133
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 426676
Total bytes received: 66
sent 426676 bytes received 66 bytes 853484.00 bytes/sec
total size is 426405 speedup is 1.00
...and you can use KM actions to extract the values you want, like
...to get the number of files transferred. If only want one value from the stats you can even get it directly in the shell script
rsync -rR --stats sourcePath destPath | egrep -o "transferred: (\d+)" | egrep -o "\d+"
...which will return just the number from the "Number of files transferred:" line -- you could add those up as you went through your "For Each" loop.
I still haven't had the time to look into this to fully understand what's happening... there's a lot to figure out, since I'm 100% new to this. I will let you know once I do.
Thank you so much!
As mentioned elsewhere, I think we've headed down the wrong path. This is doable with KM actions, without the overhead of shelling-out, futzing with funky filenames, losing metadata, etc.
The "Create New Folder" action (deliberately) does nothing, and doesn't error and abort the macro, if the "new" folder already exists. It also has the option to create any intermediate folders that are required. So all we have to to "copy while retaining folder structure" is to:
Local_sourceBase in the macro below)Local_destBase in the macro below)parent of our target path, creating the folder structure to copy our item toHere's a demo that will copy selected items in the user's home folder or its subdirectories and copy them to ~/Desktop/Backups/%ICUDateTime%hhmm%. So if I select the file ~/Desktop/For Forum/Palette Test.png and run the macro at 1:15pm the copied file will be at ~/Desktop/Backups/1315/Desktop/For Forum/Palette Test.png.
Copy Selected Items to Backup Folder.kmmacros (6.1 KB)
It looks long because I'm being (overly?) cautious with the paths, standardising and encoding them. That's partly in case you want to turn it into a sub-routine and call it from other macros -- I haven't because I thought that parameters etc might confuse the issue.
It shouldn't take much to do similar in your "Backup" macro, but shout if you have problems.
