Move All Files in Selected SubFolders to Parent Folder [Example]

Use Case

  • Normally I like to use native, non-scripting, KM Actions when I can, as long as they are effective and reasonably efficient.
  • The main challenge in this case was identifying the subfolder which ended in a specific suffix. The AppleScript for this is easy:
    • set subFolderList to folders of mainFolderAlias whose name ends with subFolderSuffix
  • However, use of the Finder for this can be very slow.
  • Macro/script has been updated to use System Events -- much, much faster.
    • set subFolderList to (disk items of mainFolderAlias whose name ends with subFolderSuffix and visible is true)
  • In this case, I could not find the equivalent KM Actions to perform the same function as the below AppleScript.
  • If anyone knows of a non-scripting solution, please feel free to post below.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

UPDATED: 2019-06-17 23:19 GMT-5

~~~ VER: 1.1    2019-06-17 ~~~

  • Replaced script section that used Finder with System Events to greatly improve the performance.
    • Based on scripts below by @CJK and @ccstone
    • To move files in a main folder of ~350 items takes only ~0.07 sec
  • Will now process all subfolders with the specified suffix
  • Now provides option to replace identical files in the Main folder

MACRO:   Move All Files in Selected SubFolders to Parent Folder [Example]

~~~ VER: 1.1    2019-06-17 ~~~

DOWNLOAD:

(Note download file (macro) name is different from above title)
Move All Folders in Source Folder to Different Folder [Example].kmmacros (21 KB)
Note: This Macro was uploaded in a DISABLED state. You must enable before it can be triggered.


ReleaseNotes

Author.@JMichaelTX

PURPOSE:

  • Move All Items in SubFolder with Suffix to Main Folder
    • SubFolders identified by a given sufix
    • May have more than one subfolder with this suffix
    • Option to Replace Items with the same name in Main Folder (else error)

HOW TO USE

  1. First, make sure you have followed instructions in the Macro Setup below.
  2. Trigger this macro.

MACRO SETUP

  • Carefully review the Release Notes and the Macro Actions
    • Make sure you understand what the Macro will do.
    • You are responsible for running the Macro, not me. ??
      .
  1. Assign a Trigger to this maro.
  2. Move this macro to a Macro Group that is only Active when you need this Macro.
  3. ENABLE this Macro.
    .
  • REVIEW/CHANGE THE FOLLOWING MACRO ACTIONS:
    (all shown in the magenta color)
    • Set Variable "Local__MainFolder"
      • POSIX path to folder
    • Set Variable "Local__SubFolderSuffix"
      • All subfolders with this suffix will be processed
    • Set Variable "Local__ReplaceFiles"
      • Set to "Yes" to replace existing files in Main Folder

REQUIRES:

  1. KM 8.2+
  2. macOS 10.11.6 (El Capitan)

TAGS: @Finder @Files @Folder @AppleScript

USER SETTINGS:

  • Any Action in magenta color is designed to be changed by end-user

ACTION COLOR CODES

  • To facilitate the reading, customizing, and maintenance of this macro,
    key Actions are colored as follows:
  • GREEN -- Key Comments designed to highlight main sections of macro
  • MAGENTA -- Actions designed to be customized by user
  • YELLOW -- Primary Actions (usually the main purpose of the macro)
  • ORANGE -- Actions that permanently destroy Variables or Clipboards,
    OR IF/THEN and PAUSE Actions

USE AT YOUR OWN RISK

  • While I have given this a modest amout of testing, and to the best of my knowledge will do no harm, I cannot guarantee it.
  • If you have any doubts or questions:
    • Ask first
    • Turn on the KM Debugger from the KM Status Menu, and step through the macro, making sure you understand what it is doing with each Action.

image

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

property ptyScriptName : "Move All Items from SubFolder to Main Folder"
property ptyScriptVer : "1.1" --  CHG Method Used to Select Items to use System Events
property ptyScriptDate : "2019-06-17"
property ptyScriptAuthor : "JMichaelTX" -- based on scripts by @CJK & @ccstone

(*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PURPOSE:  
  • Move All Items in SubFolder with Suffix to Main Folder
     • SubFolders identified by a given sufix
     • May have one or more subfolders with this suffix
     • Option to Replace Items with the same name in Main Folder


RETURNS:
  • "OK" on the first line
  •  TBD on subsequent lines
  • OR, "[ERROR]" & error details
  
KM VARIABLES REQUIRED: (must be set before calling this script)
  • Local__MainFolder
  • Local__SubFolderSuffix
  • Local__ReplaceFiles
  
REQUIRED:
  1.  macOS El Capitan 10.11.6+
      (may work on Yosemite 10.10.5, but no guarantees)
      
  2.  Mac Applications
        • Keyboard Maestro 8.2+
        

  1.  2019-06-17, ccstone, Keyboard Maestro Discourse
      Move All Folders in Source Folder to Different Folder [Example]
      https://forum.keyboardmaestro.com/t/move-all-folders-in-source-folder-to-different-folder-example/14016/7?u=jmichaeltx
  
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~  
*)
use AppleScript version "2.5" -- El Capitan (10.11) or later
use scripting additions

## Some Scripts may work with Yosemite, but no guarantees ##
#  This script has been tested ONLY in macOS 10.14.5 (Mojave)

property LF : linefeed
property CR : return
global gCurrentApp
set gCurrentApp to path to frontmost application as text -- use for dialogs

set scriptResults to "TBD"

try
  --~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  
  ### Requires Keyboard Maestro 8.0.3+ ###
  set kmInst to system attribute "KMINSTANCE"
  tell application "Keyboard Maestro Engine"
    set mainFolder to getvariable "Local__MainFolder" instance kmInst
    set subFolderSuffix to getvariable "Local__SubFolderSuffix" instance kmInst
    set replaceFiles to getvariable "Local__ReplaceFiles" instance kmInst
  end tell
  set replaceFilesBool to false
  if (replaceFiles = "Yes") then set replaceFilesBool to true
  
  (*
    ### FOR TESTING ###
    set mainFolder to POSIX path of (path to home folder) & "Documents/TEST"
    set subFolderSuffix to "mkv"
    set replaceFilesBool to true
    ### END TESTING ###
  *)
  
  if ((mainFolder = "") or (subFolderSuffix = "")) then
    error "The following KM Variables must bet set before calling this script:" & ¬
      LF & "  • " & "Local__MainFolder" & ¬
      LF & "  • " & "Local__SubFolderSuffix"
  end if
  
  set mainFolderAlias to POSIX file mainFolder as alias
  
  (*
    --- Using Finder Works, but can be very slow ---
    tell application "Finder" to set subFolderList to folders of mainFolderAlias whose name ends with subFolderSuffix    
    
    --- Using System Events is Almost as Fast as using ASObjC ---
            But you still have to use Finder to move to trash (security limitations)
            
    --- My upmost thanks to Chris Stone (@ccstone) for helping me work out the details
            of using System Events to get file list to move
  *)
  
  tell application "System Events"
    
    set subFolderList to (disk items of mainFolderAlias whose name ends with subFolderSuffix and visible is true)
    repeat with oFolder in subFolderList
      set itemList to (disk items of oFolder whose visible is true)
      
      --- IF USER WANTS TO REPLACE EXISTING FILES ---
      
      if (replaceFilesBool) then
        repeat with oItem in itemList
          
          try -- IF Creating Alias works, then the file exists in the Destination Folder --
            set oFile to ((mainFolderAlias as text) & (name of oItem)) as alias
            tell application "Finder" to move oFile to trash --   can't use System Events for this
          end try
          
        end repeat
      end if --  replaceFilesBool
      
      --~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      move itemList to mainFolderAlias
      --~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    end repeat
    
  end tell --  "System Events"
  
  set scriptResults to "OK"
  
  --~~~~~~~~~~~~~ END TRY ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  
on error errMsg number errNum
  
  if errNum = -128 then ## User Canceled
    set errMsg to "[USER_CANCELED]"
  end if
  
  set scriptResults to "[ERROR]" & return & errMsg & return & return ¬
    & "SCRIPT: " & ptyScriptName & "   Ver: " & ptyScriptVer & return ¬
    & "Error Number: " & errNum
end try
--~~~~~~~~~~~~~~~~END ON ERROR ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

--- RETURN THE RESULTS TO THE KM EXECUTE SCRIPT ACTION ---
return scriptResults

Hi! I think this is close to what I need, but I'm sorry - I'm struggling to work out what to change and how. I want to hit a key combination and move anything on the desktop to a specific folder (the same one every time). I changed the first purple box to the desktop, but I can't work out how to change the second one. I'm sure this is super basic - I'm sorry!

Charlotte

Hey Charlotte,

Don't apologize. @JMichaelTX is doing something a bit more complicated than the thread title suggests.

Here's a macro that allows you to use the direct path to the source-folder and the destination-folder.

You can use either a tilde-based path, as I have done – or a full POSIX Path.

The orange actions are the user-settings.

-Chris


Move Contents of Source Folder to Destination Folder with AppleScript v1.00.kmmacros (8.3 KB)

Hey JM (@JMichaelTX),

This is one of those places where I dislike using Keyboard Maestro native actions, because it's not intuitive and no fun at all to create the necessary steps.

Find a Folder in a Directory By Its Suffix v1.00.kmmacros (7.8 KB)

I don't particularly like using vanilla AppleScript for this either, because in a folder with many files/folders whose clauses can be pretty slow to glacial .

If I know in advance the source folder will have less than 100 or so items I may resort to a whose clause, but it's still relatively slow on my older hardware with macOS 10.12.6 (Sierra).

This AppleScript takes 22 seconds to run on a folder with 100 items in it (on my system).

tell application "Finder"
   set sourceFolder to target of front window as alias
   set itemList to (items of sourceFolder whose name ends with "sufx") as alias list
end tell

By contrast this shell script is very quick and efficient:

sourceDir=~/Desktop
folderSuffixStr='*sufx'

find "$sourceDir" -depth 1 -type d -iname "$folderSuffixStr"

We can also do VERY quick with AppleScriptObjC:

------------------------------------------------------------
# Auth: Christopher Stone
# dCre: 2016/05/08 07:01
# dMod: 2019/06/16 17:25
# Appl: AppleScriptObjC
# Task: Non-Recursively Find Files Using a Regular Expression ; RegEx ; Handler ; Insertion-Location.
# Libs: None
# Osax: None
# Tags: @Applescript, @Script, @ASObjC, @Finder, @Find, @Files, @RegEx
------------------------------------------------------------
use framework "Foundation"
use scripting additions
------------------------------------------------------------

# Front Finder Window (or Desktop if no windows are open).
tell application "Finder" to set sourceFolder to insertion location as alias

set sourceFolder to POSIX path of sourceFolder
set folderSuffix to "sufx"

set foundItemList to its findFilesWithRegEx:("(?i)^.+" & folderSuffix & "$") inDir:sourceFolder

------------------------------------------------------------
--» HANDLERS
------------------------------------------------------------
on findFilesWithRegEx:findPattern inDir:sourceFolder
   set fileManager to current application's NSFileManager's defaultManager()
   set sourceURL to current application's |NSURL|'s fileURLWithPath:sourceFolder
   set theURLs to fileManager's contentsOfDirectoryAtURL:sourceURL includingPropertiesForKeys:{} options:(current application's NSDirectoryEnumerationSkipsHiddenFiles) |error|:(missing value)
   set theURLs to theURLs's allObjects()
   set foundItemList to current application's NSPredicate's predicateWithFormat_("lastPathComponent matches %@", findPattern)
   set foundItemList to theURLs's filteredArrayUsingPredicate:foundItemList
   set foundItemList to (foundItemList's valueForKey:"path") as list
end findFilesWithRegEx:inDir:
------------------------------------------------------------

-Chris

It WORKED!! (I know, you knew that was going to happen, but I wasn't sure I could implement what you gave me). A thousand thank you's :slight_smile:

1 Like

That's because one really ought not use Finder to perform file system operations if it's not needed. System Events will be a lot quicker—not as quick as the shell or ASObjC, but if people are still depending on Finder to enumerate disk items, then they will find System Events very performant.

Having just run your code snippet on my ~/Pictures folder, asking it to isolate the suffix "jpg", Finder took a painstaking 37 seconds before realising that they all end in "jpg" (except the folders). I then did the same with System Events, which for reference, used this code:

tell application "Finder"
	set sourceFolder to target of front window as alias
end tell

tell application "System Events"
	set itemList to (aliases of the folder named sourceFolder whose name ends with "jpg")
end tell

It retrieved the result in 0 seconds, which is approximately infinity times faster than Finder.

There are a bunch of other reasons to use SE over Finder, the two next and most compelling being that the file operations won't block Finder while being performed, so can happily take place in the background without disturbing one's workflow in Finder; and that SE handles posix paths sublimely, including tilde expansions. I actually also like that it doesn't bitch at me when I ask to make a new file or folder knowing it might already exist, whereas Finder thinks it's genuinely worth throwing an error and terminating the script to make a big deal about it (that's more of a personal preference, I imagine).

The situations I can think of to use Finder are largely limited to the use of these commands/properties/objects: reveal, delete (which goes to trash vs. SE = immediate deletion), front Finder window, and insertion location. Even then, I always limit its involvement to the one specific expression, coercing stuff to alias class, so everything before and everything after is done by SE.


All that being said, I'd definitely go with the shell script.

2 Likes

Hey @CJK,

Well done!

This is the first time I've seen someone squeeze an alias list directly out of System Events.

This script runs on my 500 file test folder in ~ 0.07 seconds:

tell application "Finder" to set sourceFolder to target of front window as alias
tell application "System Events" to set itemList to (aliases of sourceFolder whose name ends with "sufx")

It's not as fast as the AppleScriptObjC code I posted (on my system), but it's good enough for most uses.

-Chris

1 Like

Just update my OP above with revised script.

@CJK, thanks for a great idea, and big improvement in performance.

However this script returns the hidden, invisible files that we don't want to move, so I had to adapt (with @ccstone's help) this to use:

  tell application "System Events"
    
    set subFolderList to (disk items of mainFolderAlias whose name ends with subFolderSuffix and visible is true)
    repeat with oFolder in subFolderList
      set itemList to (disk items of oFolder whose visible is true)
      
      --- IF USER WANTS TO REPLACE EXISTING FILES ---
      
      if (replaceFilesBool) then
        repeat with oItem in itemList
          
          try -- IF Creating Alias works, then the file exists in the Destination Folder --
            set oFile to ((mainFolderAlias as text) & (name of oItem)) as alias
            tell application "Finder" to move oFile to trash --   can't use System Events for this
          end try
          
        end repeat
      end if --  replaceFilesBool
      
      --~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
      move itemList to mainFolderAlias
      --~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    end repeat
    
  end tell --  "System Events"

This seems just as fast. Searching a folder with ~350 items (folders & files) still took only ~0.07 sec to process.

BTW, in case you, or anyone is wondering, you can't use System Events to move items to trash, at least on Mojave.

Indeed. I do have a specific set of instances where I do use Finder out of necessity, and one of those is for trashing files that can be easily untrashed, and I used it to overwrite the rm command in my shell after one time obliterating most of my system files with a find -exec rm {}+-natured command and a regex typo. Also, ㎞ is great for letting one redefine how ⌘-backspace operates in Finder, but even simple deletion commands can be slow, unfortunately, but Finder remains a necessary evil.

There were a couple of things I noticed about the latest version of the script:

[ Preamble: I actually got confused for a while by the title of the thread being Move All Folders in Source Folder to Different Folder. Under the release notes, the purpose of the script (or macro) is stated to be Move All Items in SubFolder with Suffix to Main Folder. Could the title of the thread by updated for clarity ? ]

subFolderList is storing a collection of disk items, which will include files, when I think you only want folders to be filtered by the suffix.

The structure of the script also has a fair bit of inefficiency built into it:

  • The two move commands are each performed once per iteration, which will be slower than taking them outside of a repeat loop in which you can instead build a list of file items that the move command can operate on in one go. It also means that the trashed files can be untrashed as a group if needed, rather than one-by-one in reverse order.

  • A feature of any script that always gets my attention is having a repeat loop inside a repeat loop. Sometimes, this is unavoidable, and it's certainly not an inappropriate nesting of loops in this case because I can see why the inner loop is necessary to check which items in a list exist and which don't. It's not obvious, but we can bypass these checks.

With this in mind, plus a tweaking of the filter, you're welcome to inspect the changes I've made and, of course, use the script below if you like. I don't think ~0.07s is something that particularly needs improving upon, so this might be more of an interest read than anything:

tell application "System Events" to tell the folder named mainFolderAlias
   set my text item delimiters to linefeed
   
   tell (a reference to (items in (folders of it whose visible = true ¬
      and name ends with the subFolderSuffix and true is in the ¬
      visible of aliases) whose visible = true)) to tell {name, ¬
      paragraphs of (path as text)} to ¬
      set {_trash, subFolderItems} to it
   
   if subFolderItems = {} then return 0
   if not replaceFilesBool then return move ¬
      the subFolderItems & item 1 to it
   
   set my text item delimiters to linefeed & path
   
   set _trash to paragraphs of (path & _trash)
   move the _trash to it
   tell me to tell application "Finder" to delete the result as alias list
   move the subFolderItems to it
end tell

How it works:

The core refactoring takes place in having System Events return everything as a path, which is literally just a string. String paths in AppleScript are, in many situations, parsed and treated just like a file reference, so you can pass them to relevant commands to have file operations performed on them. But unlike file object specifiers (file, alias, folder, disk item, etc.), they aren't explicitly pointed at a file on the filesystem, and presumably AppleScript can simply parse an invalid string path as a regular string. Being a unary value rather than a complex object, they are faster to retrieve and store, less taxing on AppleScript and more space efficient in terms of storage during execution.

The rather complex-looking/-sounding filter is really just a the original two filters combined into one. This can only be done, however, if one can mitigate the need to iterate through the lists, which are nested and potentially contain a lot of empty lists that can be very obstructive.

  • This is System Events' fault, because of the way it returns and processes references to file system objects in a persistently recursive and self-referential manner. It's very odd, given that similar references to, say, UI Element object lists are handled very gracefully and are easily iterated through irrespective of the level of nesting.)

The trick is to add an additional condition to the filtering of the subfolders:

true is in the visible of aliases

that forces the enumeration of the disk items within those subfolders to be performed only on the subfolders we know will contain items, and thus eliminate any empty lists from the result. The result is still a nested list of lists, but every item is a list that contains only string values (the disk item paths), and that means it is now easy to concatenate into a text list, then split apart again to yield a flattened, one-dimensional list of paths.

This is what allows individual checks for file existence to be bypassed if you take advantage of one of the features/side-effects of System Events' move command that will happily take a list of file paths that may or may not exist, and when you attempt to move them onto themselves, it scrubs the invalid file paths from the list and returns to you only those that could be operated upon.

The other side-effect of move is a small hindrance when its typical graceful handling of non-existent file paths has the caveat that it must contain at least one path that is valid. It cannot understand or act upon {}. This extends to attempting to use move to overwrite files that already exist at the destination: at least one path in the list must be able to undergo a valid move operation. So, employing move's first side-effect once more, you might notice that if the replaceFilesBool is true, the script takes the list of subfolder items it intends to move and adds an additional item to the list, namely an item that lives in the destination folder that is guaranteed to be a valid file operation when it gets moved onto itself, allowing the rest of the paths to be processed without complaint.

Known issue:

  • If multiple subfolders contain items with the same name that both need to be moved up into the main folder, only one set will be able to move successfully. The others will be left in their subfolders. As the original script is iterating through the folders individually and continually moving and replacing, I shouldn't expect this is a phenomenon that arises with it. But if one is content with emulating this behaviour, then one can consider that whatever files remain in their subfolders unmoved are disposable, and deleting them is trivial (or, easier still, deleting the subfolders with the suffix, which will either be empty or contain only disposable files).
1 Like

Thanks, @CJK. I always appreciate your attempts at optimizing my scripts. :wink:

I do like learning new things, but from a practical POV when developing a script, I generally observe these self-imposed rules:

  1. First, get the functionality working correctly
  2. Then, optimize only as much as is needed for practical applications.

I'd rather be on to solving other problems rather than reducing the script time by 0.01 sec. :wink:

I did observe some further optimizations that could be done, but chose not to.
Also, my objectives when writing any script, for just myself, but particularly for public consuming, include:

  1. Write code that is easy to read and modify by others
  2. Provide ample comments to make things clear

It is always good to see how others might solve the same problem. :+1:

I understand. Your objectives and your rationales all make perfect sense to me and are very pragmatic. I posted it as more of an interest read for the geeks on this forum.

A post was split to a new topic: How Do I Move Files in Download Subfolders Up to Parent Folder?