I’m trying to build a Keyboard Maestro macro that compresses folders selected in Finder into ZIP files, using ChatGPT external shell script which imitates parallel processing. It mostly works, but I need help wiring it together properly in KM and adding a progress bar.
The shell script I have (/usr/local/bin/parazip.sh):
Usage:
It takes one source folder, compresses it in parallel, and writes one .zip file.
What I want the macro to do:
Take the selected folders in Finder (left window).
Detect the destination path from the second Finder window (right window).
For each selected folder, call the shell script with arguments:
SRC_DIR = the selected folder
FINAL_ZIP = <DEST_DIR>/<basename>.zip
WORKERS = 12
While this runs, show a Custom HTML Prompt that displays progress (e.g. a progress bar and status text).
My idea was to have the shell script update KM variables PARAZIP_STATUS and PARAZIP_PROGRESS using osascript,
and then the HTML prompt could refresh and display them until finished.
What I’ve got so far:
I can grab the destination folder with AppleScript:
set destPOSIX to ""
tell application "Finder"
try
if (count of windows) ≥ 2 then set destPOSIX to POSIX path of (get target of window 2 as alias)
end try
end tell
tell application "Keyboard Maestro Engine" to setvariable "DEST_DIR" to destPOSIX
If I were building this workflow, I would consider alternative methods for setting your destination path. While this is certainly possible, it would require a non trivial amount of conditional error checking. Do you often compress these zip files to the same folder, or a folder based on your source folder's name?
I have a similar macro to send zipped folders to my clients, and my flow is like this:
Select folders in finder window.
Trigger Keyboard maestro macro, which passes folder paths to a shell script and creates the zip files in their folder's source directories, storing the destination zip files paths into a variable.
Once finished, pass zip file paths to an uploader subroutine (this could be any thing you want to do "next" with your zip files)
After that is completed successfully, zip files are deleted.
Depending on your needs, that might not work for you, but I'd still recommend thinking about setting a programatic destination path, rather than relying on a secondary finder window, or at the very least, putting a step in to "confirm" your destination path.
Which part of the macro is giving you trouble right now?
It works this way but I see no progress. There is a progress when I run the script in the terminal.
So I would like to run the terminal window for the every selected folder and to make the terminal close after it finishes them. I can't figure out how to do it.
Is it your echo statements that you want as "progress", or something else? (It doesn't look like it's the output from zip since you've got the -q in there.)
Those echos will be returned to KM, but only after the script has completed -- so not much use as a progress indicator.
You could post those echos back to KM variables using osascriptand then constantly refresh your "Custom HTML Prompt" to pick up any changes. Or you could use the "Execute a JavaScript in a Custom Prompt" action to push changes to the Prompt -- either passing the values to a macro as a parameter or by creating and running the macro on-the-fly within your shell script (see A KM Prompt in a Python Script for an on-the-fly example, albeit in a python script).
I didn’t check macro yet (I don’t have my mac here), but looked into script (probably generated by AI, terrible) and it use GNU parallel with bar parameter, which mean showing status of child jobs (progress bar).
Never heard of it -- but when has that ever stopped me?
The man page section for the --bar option shows how to pass progress output to zenity (which I've also never heard of...). zenity is available via homebrew and that might be enough for OP's purposes, otherwise you could probably replace the zenity section of the pipeline with a call to osascript and carry on as in my final paragraph above.
Thanks for you replies! I am not a specialist in scripting. What I need is to emulate the 2 pane manager with the Finder's windows.
This also includes FAST archiving to the seciond pane. I use Bandizip but via clicking on it's GUI and it's not really convinient. So I asked ChatGPT is it possible to split the files inside the folder into a number of chunks and archive them in parallel with built in acrhiver. It created this ugly script
Currently with his help I was able to achieve some stable results by using this version of ugly script
#!/usr/bin/env bash
set -euo pipefail
export LC_ALL=C
export PATH="/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin"
# ------ config ------
WORKERS_DEFAULT=20
ZIPLEVEL_DEFAULT=5
CLOSE_TERM="${CLOSE_TERM:-1}" # 1 = закрити вікно Terminal після завершення
# --------------------
SRC_DIR="${1:?need SRC_DIR}"
FINAL_ZIP="${2:?need FINAL_ZIP}"
WORKERS="${3:-$WORKERS_DEFAULT}"
[[ "$WORKERS" =~ ^[0-9]+$ ]] || WORKERS=$WORKERS_DEFAULT
(( WORKERS > 0 )) || WORKERS=$WORKERS_DEFAULT
ZIPLEVEL="${PARAZIP_ZIPLEVEL:-$ZIPLEVEL_DEFAULT}"
case "$ZIPLEVEL" in 1|2|3|4|5|6|7|8|9) : ;; *) ZIPLEVEL=$ZIPLEVEL_DEFAULT ;; esac
command -v zip >/dev/null || { echo "[ERR] zip not found"; exit 1; }
if command -v zipmerge >/dev/null 2>&1; then
ZIPMERGE="$(command -v zipmerge)"
elif [ -x "/opt/homebrew/opt/libzip/bin/zipmerge" ]; then
ZIPMERGE="/opt/homebrew/opt/libzip/bin/zipmerge"
else
echo "[ERR] zipmerge not found (install libzip)"; exit 1
fi
TMPDIR="$(mktemp -d)"; cleanup(){ rm -rf "$TMPDIR"; }; trap cleanup EXIT INT TERM HUP
( cd "$SRC_DIR" && find . -type f -print ) > "$TMPDIR/filelist.txt"
TOTAL=$(wc -l < "$TMPDIR/filelist.txt" || echo 0)
[ "$TOTAL" -gt 0 ] || { echo "[ERR] no files to archive"; exit 1; }
echo "[PARAZIP] files: $TOTAL workers: $WORKERS ziplevel: $ZIPLEVEL"
# split
if command -v gsplit >/dev/null 2>&1; then
gsplit -n l/"$WORKERS" "$TMPDIR/filelist.txt" "$TMPDIR/chunk."
else
awk -v n="$WORKERS" -v pfx="$TMPDIR/chunk." '
{ fn = sprintf("%s%02d", pfx, (NR-1)%n); print >> fn }
END { for (i=0;i<n;i++) close(sprintf("%s%02d", pfx, i)) }
' "$TMPDIR/filelist.txt"
fi
CHFILES="$(ls "$TMPDIR"/chunk.* 2>/dev/null | sort || true)"
[ -n "$CHFILES" ] || { echo "[ERR] no chunks created"; exit 1; }
i=0
echo "$CHFILES" | while read -r ch; do
[ -n "$ch" ] || continue
cnt=$(wc -l < "$ch")
printf "[CHUNK %02d] files: %s\n" "$i" "$cnt"
i=$((i+1))
done
export SRC_DIR ZIPLEVEL
echo "$CHFILES" | xargs -I{} -P "$WORKERS" bash -c '
set -euo pipefail
CHUNK="$1"; OUTZIP="${CHUNK}.zip"
[ -s "$CHUNK" ] || exit 0
echo "[CHUNK $(basename "$CHUNK")] start"
cd "$SRC_DIR"
if [ "${PARAZIP_VERBOSE:-0}" = "1" ]; then
zip -"${ZIPLEVEL}" -X -@ "$OUTZIP" < "$CHUNK"
else
zip -q -"${ZIPLEVEL}" -X -@ "$OUTZIP" < "$CHUNK" >/dev/null 2>&1
fi
sz=$(wc -c < "$OUTZIP" 2>/dev/null || echo 0)
echo "[CHUNK $(basename "$CHUNK")] done zip=$OUTZIP size=${sz}B"
' _ {}
ZIPS="$(ls "$TMPDIR"/chunk.*.zip 2>/dev/null | sort || true)"
[ -n "$ZIPS" ] || { echo "[ERR] no part zips created"; exit 1; }
echo "[MERGE] parts=$(echo "$ZIPS" | wc -l | tr -d " ") -> $(basename "$FINAL_ZIP")"
first=1
while read -r z; do
[ -n "$z" ] || continue
if [ $first -eq 1 ]; then
cp "$z" "$FINAL_ZIP"
first=0
else
"$ZIPMERGE" -iS "$FINAL_ZIP" "$z" >/dev/null
echo "[MERGE] + $(basename "$z")"
fi
done <<< "$ZIPS"
[ -s "$FINAL_ZIP" ] || { echo "[ERR] empty result"; exit 1; }
fsize=$(wc -c < "$FINAL_ZIP" 2>/dev/null || echo 0)
echo "[DONE] $FINAL_ZIP size=${fsize}B"
# Закрити Terminal у фоновому процесі з невеликою паузою
close_term_bg() {
[ "$CLOSE_TERM" = "1" ] || return 0
command -v osascript >/dev/null 2>&1 || return 0
nohup osascript >/dev/null 2>&1 <<'AS' &
delay 0.25
tell application "Terminal"
if (count of windows) > 0 then close front window
if (count of windows) = 0 then quit
end tell
AS
}
close_term_bg
which is ran by Automator's Apple script (probably of the same level of ugliness)
on run {input, parameters}
set createdZips to {}
tell application "Finder"
if (count of Finder windows) = 0 then return input
set srcWin to front Finder window
set srcDir to POSIX path of (target of srcWin as alias)
set sel to selection
if sel is {} then return input
set tgtDir to missing value
set frontId to id of srcWin
set allWins to every Finder window
repeat with w in allWins
if (id of w) is not frontId then
set tgtDir to POSIX path of (target of w as alias)
exit repeat
end if
end repeat
if tgtDir is missing value then set tgtDir to srcDir
end tell
tell application "Terminal"
activate
if (count of windows) = 0 then do script ""
set termWin to front window
end tell
tell application "System Events" to set frontmost of process "Terminal" to true
repeat with f in sel
tell application "Finder"
set p to POSIX path of (f as alias)
set isFolder to (class of f is folder)
set baseName to name of f
end tell
set outZipPath to tgtDir & baseName & ".zip"
set end of createdZips to outZipPath
if isFolder then
set shellInner to "export PATH=/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin; HOLD=1 /usr/local/bin/parazip.sh " & quoted form of p & " " & quoted form of outZipPath & " 12"
else
set shellInner to "export PATH=/opt/homebrew/bin:/usr/local/bin:/usr/bin:/bin; tmp=$(mktemp -d); cp -f " & quoted form of p & " \"$tmp/\"; HOLD=1 /usr/local/bin/parazip.sh \"$tmp\" " & quoted form of outZipPath & " 12; rm -rf \"$tmp\""
end if
set cmd to "/bin/zsh -l -c " & quoted form of shellInner
tell application "Terminal" to do script cmd in termWin
end repeat
-- Terminal лишається відкритим, щоб бачити вивід/помилки
-- М’який чек появи архівів, щоб підсвітити їх у Finder
set deadline to (current date) + 5
repeat
set allExist to true
repeat with z in createdZips
try
set _a to (POSIX file (z as text)) as alias
on error
set allExist to false
exit repeat
end try
end repeat
if allExist then exit repeat
if (current date) > deadline then exit repeat
delay 0.2
end repeat
tell application "Finder"
set zipAliases to {}
repeat with z in createdZips
try
set end of zipAliases to (POSIX file (z as text)) as alias
end try
end repeat
if (count of zipAliases) > 0 then
reveal zipAliases
activate
select zipAliases
end if
end tell
return input
end run
If you could help with more elegant solution I would be very thankful!
You want to run potentially more than one window with parallel jobs which zipping files from different catalogs (loop over selection from Finder). It will be hard to trace status of all windows in the same time - you can trap in practice only one output in time in KM. The solution presented here: Run command in Terminal.app window Macro (v11.0.4) assume, that only one terminal window is running asynchronously and KM will wait for the results from that one instance. Solution for multiple child windows requires different way.
Window doesn't close because it is NOT TERMINAL WINDOW but KM window with results (after finishing the task - no progres in result, because job was already finished)
To observe what's happens you need real terminal window which should be triggered by f.ex. AppleScript (see below) and transfer to it command to execute (with arguments). But it means that such window will stay open after finishing script unless you finish the script with exit (or add exit after name of the script). In that case you will see the progres but loose window after script finish (you can add small delay using sleep before exit).
Example code to instance Terminal window:
-- KMINSTANCE is part of created window name
set kmInst to system attribute "KMINSTANCE"
-- Get KM variable with KMFIFO node name/path
tell application "Keyboard Maestro Engine"
set localFile to getvariable "LOCALFILE" instance kmInst
set localFilePath to getvariable "LOCALFILEPATH" instance kmInst
set localFileName to getvariable "LOCALFILENAME" instance kmInst
set localBaseName to getvariable "LOCALBASENAME" instance kmInst
set destDir to getvariable "DEST_DIR" instance kmInst
set out to destDir & localBaseName & ".zip"
end tell
-- after finishing jobs wait 10 seconds
set exitcmd to "; sleep 10; exit"
-- uncomment below to see results (window will not close)
-- set exitcmd to ""
tell application "Terminal"
activate
-- "exit" command after 1st command will close the window
set result to do script "~/Documents/kmmacros/Forum/parazip.sh " & localFile & " " & out & exitcmd
--set result to do script "echo " & kmfifo in theTab
return result
end tell
Below my proposal - variant with windows showing progress and closing after 10 seconds since process finished.
Parallelising is easy -- just call zip multiple times. In KM, call zip via an async "Execute a Shell Script" action. You can even control the number of cores used.
Proof of concept that zips the currently-selected Finder items, creating an archive for each item in that item's parent directory:
What do you mean by this? If you have 20 files and a maximum of 4 cores to use, do you want the first 5 files zipped to archive1.zip, files 6-10 in archive2.zip, and so on? Or is it something else?
What you want to do will decide how progress can be reported. If every item selected in Finder is zipped to its own archive you can just display a count of items processed. If you are doing "chunks" as described above you'll probably want percentage processed of the least complete archive. And I'm sure there are many other ways.
Once you've got your temporary archives you can merge them -- I've never used zipmerge, does that output progress in any way? Or you could repeatedly merge, displaying a progress count.
I'm not saying that this approach is any better than what you've tried so far -- but we've got KM actions, let's use them!
I have 10 (8 performance and 2 efficiency) cores, so I guees I can use more. than 4?
I have a folder which contains some files and some folders inside. That subfolders contain an audio and video files. I need all of this to be split on a few chunks, to be zipped in parallel and after that combine into one zip file that preserves initial folder structure. Does it make sense?
Why mess around with "chunks"? Why not just zip every item in the folder to its own .zip file, then combine all those files into the master?
Continuing from the previous macro and still using the Finder selection for our source, but this time using a temporary folder to zip to -- and the master file will end up there too. That makes it easier to troubleshoot and reset -- once we're happy that's working it will be easy to set the source and target directories based on your Finder windows. But one step at a time!
Note the explicit path for zipmerge in the last action -- I'm on an Intel but I think you're running Apple Silicon so you'll need to change that for your platform.
Yes, we're building this in baby steps. That's partly because I think that's a good way to make a macro -- solve one part of the problem at a time, then expand to include the next thing. But it also makes feedback easier because only a few things change -- and might be wrong -- between one version and the next.
But in this case it will not treat the files in subfolder as a separate files, right? If I have a subfolder in the source folder, it will treat it as one file?
Currently it creates the zip file in the temp folder. Just slow
UPD. 2 files. The needed one and the MasterFile.zip
Erm... Why would creating zips in /tmp be any slower than creating them elsewhere? Unless you've got a dog-slow System drive but your usual test are done on fast externals, or perhaps a malware scanner checking writes to /tmp but not to where you normally work.
If /tmp is a problem then point it to your own scratch space -- you just need somewhere where you can create the temporary folder to hold the intermediate zip files, and you only really need that because it is much easier to delete that temporary folder and its contents than to sort through a bunch of files deciding whether to delete them or not!
And remember that, as written, the macro is limited to 2 cores -- it'll run roughly half the speed of your 4-core shell script.
