Creating a zip file and recursing through subfolders

JMichaelTX · December 21, 2016, 1:17am

Wow! This is very impressive:

Zip 3.0 (July 5th 2008). Usage:
zip [-options] [-b path] [-t mmddyyyy] [-n suffixes] [zipfile list] [-xi list]
  The default action is to add or replace zipfile entries from list, which
  can include the special name - to compress standard input.
  If zipfile and list are omitted, zip compresses stdin to stdout.
  -f   freshen: only changed files  -u   update: only changed or new files
  -d   delete entries in zipfile    -m   move into zipfile (delete OS files)
  -r   recurse into directories     -j   junk (don't record) directory names
  -0   store only                   -l   convert LF to CR LF (-ll CR LF to LF)
  -1   compress faster              -9   compress better
  -q   quiet operation              -v   verbose operation/print version info
  -c   add one-line comments        -z   add zipfile comment
  -@   read names from stdin        -o   make zipfile as old as latest entry
  -x   exclude the following names  -i   include only the following names
  -F   fix zipfile (-FF try harder) -D   do not add directory entries
  -A   adjust self-extracting exe   -J   junk zipfile prefix (unzipsfx)
  -T   test zipfile integrity       -X   eXclude eXtra file attributes
  -y   store symbolic links as the link instead of the referenced file
  -e   encrypt                      -n   don't compress these suffixes
  -h2  show more help

ComplexPoint · December 21, 2016, 12:57pm

You could experiment with building and executing the zip command line that you want,
using an Execute JavaScript for Automation (or AppleScript or Bash) action in Keyboard Maestro.

The elements of a JS action might look a bit like this:

(function () {
    'use strict';

    // selectedPaths :: () -> [pathString]
    var selectedPaths = function () {
        return Application('Finder')
            .selection()
            .map(function (x) {
                return decodeURI(x.url())
                    .slice(7);
            });
    };

    var strCMD = 'zip -r ~/testArchive3 ' + selectedPaths()
        .map(function (s) {
            return '"' + s + '"';
        }) // paths quoted
        .join(' '); // spaces between quoted paths

    var a = Application.currentApplication(),
        sa = (a.includeStandardAdditions = true, a);

    return strCMD + ' -->\n\n' + sa.doShellScript(strCMD);
})();

JMichaelTX · December 21, 2016, 6:13pm

Thanks for the script Rob. It looks very interesting.
However, I'm not sure where to "add the zip flags".

Could you please provide an explicit example?

Thanks.

Kurt_Kessler · December 21, 2016, 6:43pm

Rob, correct me if I'm wrong, but are you talking about the zip parameters in the following line?

var strCMD = 'zip -r ~/testArchive3 ' + selectedPaths()

Where the "-r" = recurse into directories. You can the parameters in the string to "zip -rm" to recurse and move.

Is there a way to change the "testArchive3" to the folder name? So the the zip will create .zip?

This looks very interesting....

ComplexPoint · December 21, 2016, 7:53pm

That’s right – I’ve added the -r flag already, and you can add others to construct a command line in strCMD that is tailored to what you need.

You may also want to arrange for an alternative output path, for example.

rjames86 · December 22, 2016, 9:47pm

AppleScript seems so heavy handed for this. Since you're executing a shell script, why not just use bash? Here's a working example I quickly wrote up:

The script:

SAVE_DIR=`dirname "$KMVAR_Path"`
BASE=`basename "$KMVAR_Path"`
BASE_NO_EXT=${BASE%.*}

pushd "$SAVE_DIR"

zip -rm "$BASE_NO_EXT" "$BASE"

popd

If you want to change the name of the zip file, just set a new variable in KM and then change the line to zip -rm "$KMVAR_<thevarname>" "$BASE" where <thevarname> is the name of your variable

JMichaelTX · December 22, 2016, 10:40pm

Thanks for sharing your macro/script.

I'm very much a Bash novice, so I hope you'll excuse my basic questions.

This looks like it will execute the script for each file.

Is that correct?
If so, wouldn't it be better to zip all files at once? Or is this not possible?
.

How do you pass a list of files to the zip command?

rjames86 · December 22, 2016, 10:56pm

To answer your questions:

It does it one at a time. Based on the request, he wanted to compress a single folder into an archive. The bash script doesn’t handle this, the for-loop does within KM
The syntax for compressing multiple files with zip is zip file.zip file1 file2 file3

Here’s a little more context into what’s happening with the script:

SAVE_DIR=`dirname "$KMVAR_Path"`

Gets the directory name for the Path. If the path is /Users/myname/Desktop/My Folder then the dirname is /Users/myname/Desktop

BASE=`basename "$KMVAR_Path"`

This gets the name of the folder without the dirname. From the example above, we’d get “My Folder”

BASE_NO_EXT=${BASE%.*}

This isn’t necessary unless you’ve selected a file with an extension. This will strip off the “.txt” from a file name like “file.txt”

pushd "$SAVE_DIR"

push the save path into the directory stack

zip -rm "$BASE_NO_EXT" "$BASE"

zip up the folder

popd

pop the last path added to the directory stack. In our case here, it’s “$SAVE_DIR”

For compressing all of the selected files into a single archive, that’d take a bit more work. If someone’s looking for this particular solution, I’ll write it up, otherwise I’ll save myself some work

JMichaelTX · December 22, 2016, 11:09pm

If that be the case, then it would be easy enough using the KM For Each to build the file list, then pass the entire list to the Bash Script. See anything wrong with that?

Thanks. I would like this, but if what I stated above will work, I think I can modify the macro to do it.

But if it is not too much trouble for you, I'd love to learn from an expert.

BTW, for any other Bash novices like me out there reading this, here is the complete, very detailed, Apple manual on Zip:

zip(1) Mac OS X Manual Page

Although this page states it is for macOS 10.9, the zip version from doing a man zip in Terminal is "ZIP(1L)", the same as in the online manual. I hope that means they are the same version?

JMichaelTX · December 22, 2016, 11:10pm

Forgot to mention this. Many, many thanks for the detailed step-by-step description. It really helps me.

rjames86 · December 23, 2016, 12:16am

Very welcome!

Yep exactly. Here's a screenshot of my solution:

Here's the script at the bottom:

# creates an array from the multiple paths variable
IFS=';' read -r -a array <<< "$KMVAR_MultiPaths"

BASE_PATH=`dirname "${array[0]}"`

PATHS_TO_ZIP=''

# Loop through the array, get the base and base name just like before, and concatenate to our PATHS_TO_ZIP variable
# Concatenating with a new line is important. You'll see why below
for element in "${array[@]}"
do
    BASE=`basename "$element"`
    BASE_NO_EXT=${BASE%.*}
    PATHS_TO_ZIP="$BASE_NO_EXT\\n$PATHS_TO_ZIP"
done

pushd $BASE_PATH
# zip didn't like passing the paths as a variable because of spaces. Echoing the paths and piping it to zip solves this
# -@ archives all of the paths passed in through stdout
echo $PATHS_TO_ZIP | zip -r -@ archive.zip
popd

Hope that helps!

JMichaelTX · December 23, 2016, 12:24am

@rjames86, many thanks again for your macro/script.

I've enhanced it a bit, to

add some documentation
Ask User to Confirm
Make it clear that the original source folder will be DELETED
Provide a nice report, copied to clipboard

Example Results

##Macro Library @ZIP Folder (& all Sub-Folders) & DELETE


####DOWNLOAD:
<a class="attachment" href="/uploads/default/original/2X/7/77efe4d6b43590ae28a8072e31feaa223d3ffabb.kmmacros">@ZIP Folder (& all Sub-Folders) & DELETE.kmmacros</a> (11 KB)

---

###ReleaseNotes

HOW TO USE:

1. Select one or more FOLDERs in the Finder
   • Each Folder wll be put in a separate Zip file
2. Trigger this Macro

Author:	 @rjames86 -- Original Macro/Script Posted below 
                @JMichaelTX -- this macro based on the one by @rjames86

WARNING!

The Folder selected in the Finder will be PERMINATELY DELETED.
(not just moved to Trash)

If the Zip File already exists, it will be OVERWRITTEN.


REFERENCE

Topic:		Creating a zip file and recursing through subfolders
Forum:		Keyboard Maestro Discourse, general

Post by:		rjames86
Post Date:	2016-12-22
Script Lang:	Bash
Post URL:	https://forum.keyboardmaestro.com/t/creating-a-zip-file-and-recursing-through-subfolders/5811/15?u=jmichaeltx

If you want to change the name of the zip file, 
  • just set a new variable in KM and 
  • then change the line to 
     `zip -rm "$KMVAR_<thevarname>" "$BASE" `

   where `<thevarname>` is the name of your variable

For a detailed, step-by-step description of the script, see:
[The above post by rjames86](https://forum.keyboardmaestro.com/t/creating-a-zip-file-and-recursing-through-subfolders/5811/17?u=jmichaeltx)

---

<img src="/uploads/default/original/2X/d/da09f3e2b3fe7f3b46c0a6260dc5304ee2e6d27d.png" width="595" height="895">

JMichaelTX · December 23, 2016, 12:34am

BTW, to address concerns about deleting the folder/files without first expanding the zip, I did expand the zip files a number of times in the process of testing the above macro, and it worked flawlessly.

Having said that, if the files were very important, or hard to replace, then I might just zip them normally (no deletion) and test the zip integrity before i deleted the original source files.

Again, this is an individual decision.

rjames86 · December 23, 2016, 12:35am

Great writeup! I personally don’t like the idea of deleting the original folder. Seems too dangerous.

I’d personally change the line zip -rm "$BASE_NO_EXT" "$BASE" to be:

zip -r "$BASE_NO_EXT" "$BASE"

It would be easy to add another confirmation if you wanted to delete or not. A way to do this could be to ask, and if they do, set a variable DELETE_CHECK to “true”. The code would then look like this:


if [[ $KMVAR_DELETE_CHECK == 'true' ]]; 
then 
    ZIP_ARGS="rm";
else
    ZIP_ARGS="m";
fi

zip -$ZIP_ARGS "$BASE_NO_EXT" "$BASE"

JMichaelTX · December 23, 2016, 12:38am

OK, I'm open to that.
How would you verify the integrity of the zip, before deleting the original folder? Or maybe just move the folder to Trash, rather than delete? Could this all be done in one macro?

rjames86 · December 23, 2016, 12:40am

oops. See above

JMichaelTX · December 23, 2016, 12:56am

OK, here's a version that moves the folder to trash, AFTER the zip has completed.

Still would like a way to verify the zip integrity. Any ideas?

Here are the main changes:

##Macro Library @ZIP T -- Zip Folder (& all Sub-Folders) & Move to TRASH


####DOWNLOAD:
<a class="attachment" href="/uploads/default/original/2X/4/467bd257d4dda460190f94134e6a4b8f745af4d4.kmmacros">@ZIP T -- Zip Folder (& all Sub-Folders) & Move to TRASH.kmmacros</a> (12 KB)

---

###ReleaseNotes

HOW TO USE:

1. Select one or more FOLDERs in the Finder
   • Each Folder wll be put in a separate Zip file
2. Trigger this Macro

Author:	 @rjames86 -- Original Macro/Script Posted below 
                @JMichaelTX -- this macro based on the one by @rjames86

WARNING!

The Folder selected in the Finder will be MOVED TO TRASH.

If the Zip File already exists, it will be OVERWRITTEN.


REFERENCE

Topic:		Creating a zip file and recursing through subfolders
Forum:		Keyboard Maestro Discourse, general

Post by:		rjames86
Post Date:	2016-12-22
Script Lang:	Bash
Post URL:	https://forum.keyboardmaestro.com/t/creating-a-zip-file-and-recursing-through-subfolders/5811/15?u=jmichaeltx

If you want to change the name of the zip file, 
  • just set a new variable in KM and 
  • then change the line to 
     zip -rm "$KMVAR_<thevarname>" "$BASE" 

     where <thevarname> is the name of your variable

For a detailed, step-by-step description of the script, see:
https://forum.keyboardmaestro.com/t/creating-a-zip-file-and-recursing-through-subfolders/5811/17?u=jmichaeltx

---

<img src="/uploads/default/original/2X/4/436a24f734c8243e2ad494ff783cd83898e0cebc.png" width="592" height="1590">

rjames86 · December 23, 2016, 12:58am

Add the T flag to zip.

zip -rT '$BASE_NO_EXT" "$BASE"

-T test zipfile integrity

JMichaelTX · December 23, 2016, 12:59am

@rjames86, et al:

How can we detect if either:

The zip fails
The zip lacks integrity

and thus NOT do the Trash?

rjames86 · December 23, 2016, 1:06am

You can add the following command after the zip, but before popd

if [[ `unzip -tq "${BASE_NO_EXT}.zip"` =~ "No errors detected" ]]; 
then
    osascript -e 'tell application "Keyboard Maestro Engine" to setvariable "DeleteAfterZip" to "true"';
else
    osascript -e 'tell application "Keyboard Maestro Engine" to setvariable "DeleteAfterZip" to "false"';
fi

you should then have a variable available to you in KM called DeleteAfterZip. If its set to the string “true”, you can delete it

Creating a zip file and recursing through subfolders

Example Results

Options