Copy a Folder With the Content of All Sub-Folders (On Multiple Levels) to Another Folder?

Hi everyone,

I'm new to KM and the first macro. I would like to create a macro to trigger when Obsidian quits (or periodically when the app is running) and then my vault folder which contains a non-set number of sub-folders (on multiple levels) and their files should be copied to a different folder. I would like this to silently happen in the background and not by manually selecting the folder with the finder.

Note for the Source folder: the number of folders, sub-folders and files can increase over time and can be deleted as well from any sub-folder.

Note for destination folder: it will need to be deleted and replaced by the updated copy every time the macro would run.

After testing different scenarios for few hours based on what I understood from 2 forum posts, I still struggle to copy all files of sub-folders and each sub-folder. This means that some sub-folders and files were not copied. I can't understand the logic behind the output of the macro.

Here in attachement is what I tried at last. But it fails to completely do what I want.

Hopefully I was clear enough with what I attend to do. I will be grateful if some of you cold show me the right direction.

I use KM 10.0.2 version.

You have a couple of different requirements, requiring different things to be done to solve it. In this post of mine I will address only one of your requirements. I might address your other requirements (e.g., copying files) in a subsequent post.

Your first requirement pertains to the triggers. You want two different things to trigger this macro: (a) the closing of an app; and (b) an intermittent time trigger. You didn't show your triggers in your screenshot so we can't see if you are handling this correctly. Since you have two different triggers that can trigger this macro, you need a mechanism that will ensure that the macro won't run twice at the same time. Your macro could seriously fail if you tried to run it twice at the same time. So you probably should use a semaphore to prevent your macro from running twice at the same time. For example, if your macro was triggered by a periodic time trigger, then I presume you do NOT want your macro to run when the app closes. It seems only logical. Here's how the start of your macro should look:

NOTE: You will need to replace "Preview" in the trigger above with "Obsidian." Since I don't have Obsidian I can't set that value.

As for your requirement to recursively copy folders from multiple source folders to a single destination folder, I think you're going about that wrong. You're trying to put the burden on KM to copy files individually and you are trying to implement recursion yourself. None of that is necessary. Instead, there are built in shell commands to do that.

To be honest, I am scared to write the macro to help you with this because you are asking for an entire folder structure to be deleted, and I don't want to write a macro for you that includes a recursive delete, because I don't want to be responsible for deleting more files that you intended. However I will show you the following simple macro which does the recursive copy, but omits the delete. This is good enough for testing purposes.

Notice that I placed "xxx" in part of my path. You must specify your own full path to the files here. No doubt your path will be different. Also notice that I didn't include a recursive delete command. If you can get this to work, we can talk about adding a recursive delete command later. I'm just too scared to give you a recursive delete command now until I'm sure everything else is working.

Hello @Sleepy !

Thanks a lot for the time you gave to guide me!! I appreciate it. I definitely progressed in understanding what was wrong.
I tested exactly what you suggested first and I think it is working well (all sub-folders and their content are copied).

As Obsidian stores notes in files residing in subfolders (of the main vault "Food4Thought"), so every time I delete a note or folder while running the app, that particular file or folder gets deleted.

I tested the macro after I did deletion of note or folder within Obsidian. And since we don't have the "delete" action of the destination folder before copying the updated source folder, the deleted subfolder (of the vault) is not deleted from the destination folder.
I will need your help for deleting the destination folder before copying the source folder to its destination folder. The destination folder to be deleted is "/Volumes/Obsidian_Dropbox/Food4Thought".

Since I wanted the macro to only run when Obsidian is active and one time after I close the app, I made some modifications to the macro. See the attachement.

In principle you are free to make that change, but I have a concern about that. I'm just not sure if Obsidian is capable of handling the situation where its files are deleted while it is running. Some apps (most?) will get corrupted if you delete their files while they are running. This is (perhaps) partly why I was afraid to delete an entire folder with you on the first draft of my solution. How do you know that Obsidian won't get confused or damaged when you delete some of the files it is using while it's running? Now that I think of it, my original solution for you didn't have an IF statement to abort if Obsidian is running, and I probably should have had that. But if you are absolutely certain that you will accept the risk of data corruption, I suppose I can give you the command to delete all files recursively.

Thanks for being cautious!

When deleting a note or folder while Obsidian is running and from within the Obsidian app, the deleted file or folder can be sent to .trash inside the vault. I checked that and all happens as it should.
The .trash folder also get copied with the macro. Therefore I don't think the data would get corrupted at some point.

I have some old knowledge of Linux commands and C programmation so I will be careful with testing the delete folder command.

That's probably a safe way to delete a folder/note, but you are trying to delete notes from OUTSIDE the app. That's a big difference.

It is possible to write code in KM that will open Obsidian and use the Obsidian GUI to safely delete folders/notes, but that's trickier. It might be the safe way to go.

If you want to experiment with deleting the folder from KM, I can't stop you. But you better have backups and be prepared to do some testing. You should probably watch Youtube videos on how to remove entire directories in UNIX, to make sure you are doing it safely. There are stories of people accidentally deleting their entire MacOS computer by entering wrong parameters to the "rm" command.

There are probably other people on this forum who aren't worried about giving you the rm command's parameters to delete an entire folder. But I'm a coward and I don't want to live with the responsibility of telling you unless we're over 99.99% sure you can use it safely. And we aren't that high yet.

Maybe this will help or at least give you some ideas:

Sync 2 Folders.kmmacros (14.6 KB)


This discussion has actually opened my eyes to a better solution.
The overall thing I initially wanted to do was to backup my main Obsidian vault (with no demo data at the moment) located on Dropbox to different places like my mounted decrypted cryptomator vault and/or to my local drive (which is daily backed up to OneDrive and E2E encrypted on my Macbook Pro).

Though free Dropbox keep deleted files/folders for 30 days, it's not necessarily easy to know which file to recover.

So I changed my mind and decided that it would be better to play safe by:

  • Making my main Obsidian vault on my local drive
  • During using and after quitting Obsidian on my laptop, I would like to sync the main Obsidian vault to my local Dropbox folder [>> doing this with KM]
  • In case of using my MD notes vault (from Dropbox) on my iPhone or iPad, I would like to sync back my notes vault from Dropbox to my local drive on my laptop. [>> doing this with KM]

Thanks a lot for helping!!
I understood what your macro would do, I will test it today and come back to you if I have question.

Okay. If you're happy I'm happy.

Do keep in mind that I have never used that product and I can't give any advice on its best use.