Setting Path as Folder in "Write to File" Empties the Folder?

Hi @peternlewis and others who might know,

I foolishly set the path for "Write SystemClipboard to File" to my Download folder (without specifying a file name). Now, I'm seeing an empty Download folder. I don't know where the original files and folder are (not in the trash) and how to recover them.

I don't have backups for the files and folders in the download folder. but there are a lot of files and folders there. Although I can probably redownload the files when I need them. But it will take a lot of time. Anyway to get back the files and folders?

Also, is it an expected behavior? I expect it to either do nothing and report an error or create a new file in that folder. But instead, it seems to delete all the files and folders in that folder.

Thanks!

That's curious. I got the same thing, with an error message:

using this action:

image

I'm usually wrong when I say there's a bug, but this sounds like one! I also lost three files, but they don't worry me too much.

P.S. I'm using macOS Monterey.

EDIT: I was able to get the files back by launching "Time Machine" on my Mac. (First time I've ever used that!) But the icon for the downloads folder looks different now. Not sure if that's important.

Hi @Sleepy,
Thanks for testing. I got the same error code.
I tried two times.
It's ok to fail; the big problem is that I've lost everything in the Downloads folder!

I tried the action again with a file name in the dropbox folder. The action also failed. It removes the entire Dropbox folder to Trash. This is fine; at least I can recover it.

I used Disk Drill to search for lost files. It finds only the existing Disk Drill dmg file in the Downloads folder which I just downloaded and have not removed. Not a single lost file was found. :joy::joy::joy:

My Error Code in Engine Log:

2021-08-17 10:02:58 Write File failed to write file to /Users/username/Downloads with error Error Domain=NSCocoaErrorDomain Code=513 "“Downloads” couldn’t be removed because you don’t have permission to access it." UserInfo={NSUserStringVariant=(
Remove
), NSFilePath=/Users/username/Downloads, NSUnderlyingError=0x6000007bd2c0 {Error Domain=NSPOSIXErrorDomain Code=13 "Permission denied"}}

The first one has "Permission denied" error. I added the Engine to Full Disk Access.

2021-08-17 10:02:58 Write File failed to write file to /Users/username/Downloads with error Error Domain=NSCocoaErrorDomain Code=513 "“Downloads” couldn’t be removed because you don’t have permission to access it." UserInfo={NSUserStringVariant=(
Remove
), NSFilePath=/Users/username/Downloads, NSUnderlyingError=0x6000007bd2c0 {Error Domain=NSPOSIXErrorDomain Code=13 "Permission denied"}} In macro “Accordance Files Jian2Fan” (while executing Write System Clipboard to File).

The 2nd time I execute it, I lost all of my files in the Download folder.

2021-08-17 10:06:53 Write File failed to write file to /Users/username/Dropbox/Atlas.string with error Error Domain=NSCocoaErrorDomain Code=512 "“Atlas.string” couldn’t be moved to “Dropbox”." UserInfo={NSSourceFilePathErrorKey=/var/folders/vl/x6z6g6y54gvg60cxn85jgnj80000gn/T/TemporaryItems/NSIRD_Keyboard Maestro Engine_2KcumV/Atlas.string, NSUserStringVariant=(
Move
), NSDestinationFilePath=/Users/username/Dropbox/Atlas.string, NSFilePath=/var/folders/vl/x6z6g6y54gvg60cxn85jgnj80000gn/T/TemporaryItems/NSIRD_Keyboard Maestro Engine_2KcumV/Atlas.string, NSUnderlyingError=0x600000504480 {Error Domain=NSPOSIXErrorDomain Code=20 "Not a directory"}}

I found out I forgot to put the file name. So I added it. This time, I changed to Dropbox folder. But I'm getting "Not a directory" error, and the file was created and placed in the root folder. Later, I found out the Dropbox folder has been removed to Trash.

2021-08-17 10:06:53 Write File failed to write file to /Users/username/Dropbox/Atlas.string with error Error Domain=NSCocoaErrorDomain Code=512 "“Atlas.string” couldn’t be moved to “Dropbox”." UserInfo={NSSourceFilePathErrorKey=/var/folders/vl/x6z6g6y54gvg60cxn85jgnj80000gn/T/TemporaryItems/NSIRD_Keyboard Maestro Engine_2KcumV/Atlas.string, NSUserStringVariant=(
Move
), NSDestinationFilePath=/Users/username/Dropbox/Atlas.string, NSFilePath=/var/folders/vl/x6z6g6y54gvg60cxn85jgnj80000gn/T/TemporaryItems/NSIRD_Keyboard Maestro Engine_2KcumV/Atlas.string, NSUnderlyingError=0x600000504480 {Error Domain=NSPOSIXErrorDomain Code=20 "Not a directory"}} In macro “Trying” (while executing Write System Clipboard to File).

Hey @martin,

You overwrote the ~/Downloads folder node with a file and completely destroyed it.

@peternlewis – this is very dangerous. I did the same thing Martin did more than a decade ago and completely hosed my Downloads folder.

I was very grateful it was only my downloads folder, and I have since started using a dedicated (and expendable) folder for testing.

Perhaps you should prevent the action from overwriting a folder.

-Chris

I thought that was what happened. Is there anyway to recover the files and folders in Downloads?

I excluded the Downloads folder from Time Machine because I thought I would never need it! :sob:

You used a shell process to overwrite the whole node, so the whole thing is completely hosed.

You proved this for yourself with DiskDrill.

Deleting something with the Finder is a different story.

It's a bummer, but there's many a cautionary tale about this sort of thing on Unix.

This is why I hammer on people to examine their path constructions before they try to use them in an action.

What you did is an all too common mistake – as I said I've made it myself...

-Chris

The Write File action trashes the existing destination file/folder before attempting to write the file.

Unfortunately in this case, Apple presumably disallowed trashing the Downloads folder, but then allowed the NSFileWrapper writeToURL action to overwrite the Downloads folder - its not really a case I ever expected, for the system to refuse to trash a file, but allow it to be destroyed instead. And indeed, an older OS would probably not have this issue, since it would have allowed the Downloads folder to be trashed.

I will add additional protections to verify the success of trashing the file/folder before proceeding.

But if what is described is what happened, there is no recovering from it except for backups.

Full, complete, and current backups are a minimum requirement for safety when using any scripting tool.

1 Like

Thanks, Peter, for confirming the issue. It is the only folder that I did not think worth backing up and turns out to be the only backup I needed in this case. But that's ok. Everything here is probably downloaded from elsewhere, and I was able to recover quite a few files from my browser download history.

Is there any reason for this behavior? I'd rather have KM to create a new file in this folder than have the folder trashed. As the latter is almost never intended. Alternatively, KM can abort the macro and alert the user that the path is a folder and not a file.

Keyboard Maestro would need to know what name to use.

The system API is exceptionally violent (as you discovered), which is why Keyboard Maestro actively trash the target before writing.

Some forms of writing files produce folders (for example RTFD), so simply refusing to replace a folder is not a practical option.

1 Like

I thought of the name before but I thought you probably have some idea already. Any unique name will do. The name can include "Keyboard Maestro" or can include the date and time or a hash number etc.

Or, before trashing, make an alert window to let users know Keyboard Maestro is going to trash a file or folder if they proceed forward. Of course, a cancel button must be present. This alert window will only show up when Keyboard Maestro is going to trash a file or folder in this situation.

Then you would not be able to overwrite an RTFD or other “file” that is stored as a folder.

Keyboard Maestro is a scripting tool, there are thousands of ways to destroy data. Keyboard Maestro tries to protect you from some of them, but in this case Apple also tried to protect you, and it was that combination that lead to disaster.

I can't add alerts to every possible dangerous action, I'd have to alert for every Execute Script action. Nor can I protect users from every possible misunderstanding or error. No more than Terminal, AppleScript, Automator, Shortcuts, etc can.

If you want an alert, you can make one, add an If Then Else action with a Path condition to test whether the destination exists, and an Alert action.

If you want a unique name, you can make one of them to, using the RandomUUID token or any of the other tokens.

Keyboard Maestro provides the tools, but it cannot protect you from the consequences.

2 Likes

I'm not saying that we should forbid this. I'm just asking for an alert when KM is going to trash a folder. The alert can allow users to continue if they want. Something like this:
image

That's exactly what I'm asking for, namely, a protection measurement to alert users in a "Write to file" action that a folder is going to be trashed.

I'm not asking for an alert for every Execute Script action. I'm asking for the "Write to file" action only.

I'm asking KM to do it, because this will protect from unperceivable damages. If a user is aware of this danger, it's less likely they'll need to build this extra action. Even if they need it, they may not always remember to do it when testing. For me, it's the first item I encounter this issue. In the future, I'll be much more careful with this action. But how about others who did not know this danger?

Again, I'm asking for a protection measurement for the "Write to file" action only. You probably have a broader view and want to add it to other similar actions as well.

Is it possible to set the alert I talked above by default and allow those in need to turn it off? The protection layer is not for veterans but for the novices like me.

Disclaimer: I suffered the consequence and want a way to prevent it. I'm just trying to offer some ideas. They may be useful and may be not, as I'm fully aware my limited knowledge about anything in coding and have to put my trust in you for decision making.

Thank you very much for taking my ideas seriously and taking time to respond. I really appreciate that!

Please don't add an alert. Minimization of keystrokes is one of the main reasons I use KM. I don't want to have to hit a keystroke every time I correctly use overwrite, just to prevent an occasional accident by other people.

2 Likes

Oh, no. I'm not asking for an alert for every action. I'm asking for an alert only when a folder is going to be trashed and replaced with a file.

For instance, if I do this:

My entire "Dropbox" folder is trashed. I would never want to have done that.

Also, as I said above,

Yeah, I would have no problem if it's included as one of the special preferences, but I suspect that Peter won't do that.

As I said to Peter above,

I think there must be a way to prevent accidental wiping out data in a unrecoverable state.

I can accept that a folder is trashed, because I can put it back. But using the "Write to file" action to trash a fold is not something I imagine myself will do. I will certainly have to go to Trash and put the folder back, which takes more time and confirming/dismissing an alert. It's worse if I did not know the folder was put to trash and emptied the trash. I'd rather to have KM alert me when a folder is going to be trashed by the "Write to file" action. But I'll leave it to Peter.

With my experience, I will be more careful to not to make this mistake again, and I will also back up everything in case this happens again so that I can recover the data. Therefore, my suggestion is mainly for the benefit of those who might make the same mistake as I did and have no backups for data.

Hey Peter,

That seems reasonable to me.

Although I wonder if you can easily check for the distinction between an actual folder verses a package-folder (pseudo file) and balk if the file in question is actually a real folder.

-Chris

I will adjust it so that if Keyboard Maestro can figure out that it is writing a file, and not a folder, and if a folder already exists at the destination, then the action will fail rather than trash the folder.

2 Likes

Thanks, Peter. That’s great!