How to turn a System Service that runs a shell script into a KM macro that runs a shell script?

Questions & Suggestions
1

I know next to nothing about shell scripts. Somehow I found a Ruby script that easily converts specific strings of text into other specific strings of text. I got it working as a System Service, but I always need to use it as part of a KM macro. I would prefer to just run it in KM, but I am missing some critical step.

Here is the Service (which works fine):

And here is what I have so far in KM (which [obviously] doesn’t do anything):

What am I missing?

p.s. Yes, I know I can just do a “search & replace” in KM, but I have several of these scripts and some of them run 50+ search & replace actions. I also know I can just leave them as Services and call them from KM, but they are clogging up my Services menu.

2

The circled parts are the parts you are missing.

  • Receives selection as text

You do the Copy, so presumably your intention is to then use the clipboard. So you have that covered.

  • Shell /usr/bin/perl

Contrary to what you said, this is a perl script, not a ruby script. You indicate this in the script by putting

#!/usr/bin/perl

as the first line of the script.

  • Pass input to stdin

This is the main problem. The input for the service is extracted from the text and then passed as stdin to the script.

Keyboard Maestro wont pass anything as stdin to the script, so the script will need to read the dat some other way. The data is in the clipboard, and Keyboard Maestro can easily access that.

You could package your script into a file and to:

pbpaste | /usr/bin/perl ~/Scripts/myscript.pl

Alternatively, you can have perl read from a variable by making a little addition to the script. Replace the first line of the script (while (<>) {) (which basically means "read from the stdin, line by line, and process each line") with:

#!/usr/bin/perl

use IO::Scalar;
$data = `pbpaste`;
$INPUT = new IO::Scalar \$data;
while (<$INPUT>) {

which means "executing the script in perl, using the IO::Scalar module; read the clipboard into variable $data; create a new input handle for that $data variable, and finally read from the variable, line by line and process each line".

20170316-220424.png974×458 32.2 KB

and finally

  • Output replaces selected text

You have the action configured for paste results, so that is all set.

2 Likes
3

Thank you so much!

I tried to follow your second set of instructions (to keep it self-contained), but I can’t figure out where to put the list of strings I want changed. I tried putting it after the “while” line but before the “print” line:

#!/usr/bin/perl

use IO::Scalar;
$data = `pbpaste`;
$INPUT = new IO::Scalar \$data;
while (<$INPUT>) {

~s/”|“/\"/g;
~s/‘|’/\'/g;
~s/–/\--/g;
~s/…/\.../g;

	print "$_";
}

but it doesn’t seem to do anything…

Also, What is “Got line:” at the end? All that did was add “Got line:” to every line in the text.

I apologize in advance for my total lack of knowledge. As I said, I don’t know anything about scripting, I just found something somewhere and adapted it.

4

Have you considered translating these scripts to JavaScript?
JavaScript has a very powerful and easy to use RegEx engine.

They would be fairly easy to implement in a KM Execute JXA script.
You could even design the JXA script to read a KM variable that contains the RegEx replace instructions, and process it in JavaScript.

5

I know about as much about JavaScript as I know about Perl. That is, just about nothing. My goal is to set up a template that can be used for multiple search-and-replace. Is there an easy way to do that in JavaScript?

6

I think so.
If you can give me some real-world examples, I'll see if I can write the JavaScript for you.

Here's what I need:

  1. Source Text -- the text to be changed
  2. Find Text -- the characters to be found and replaced
  3. Replace Text -- the characters to change the found text to.

Since you already have #2 and #3 in your perl scripts, you can just use that format.
Please use the code format, like this, using triple back quote marks:

enter your example here

Example:

SOURCE TEXT:
This text contains some ” quotes and some “ quotes to be replaced.

FIND/REPLACE:
~s/”|“/\"/g;
~s/‘|’/\'/g;
~s/–/\--/g;
~s/…/\.../g;
7

The source text is not static. What I’ve been doing until now is using selected text in the System Service.

I assumed that in KM I could the contents of the clipboard or a variable set from the contents of the clipboard.

SOURCE TEXT:
contentsOfClipboard

FIND/REPLACE:
~s/”|“/"/g;
~s/‘|’/'/g;
~s/–/--/g;
~s/…/.../g;

(I really appreciate the help)

8

I understand that. But to develop a tool, I need some representative, real-world, examples. The source text will NOT be embedded in the tool. It will be an input to the tool, just like your System Service.

In your example, you just posted the exact Find/Replace I used.
I need for you to give me other examples of Find/Replace.

9

Hey Folks,

The big question here is why are people fooling with a script when Keyboard Maestro can more simply do this kind of find/replace itself?

If @wogo wants to simplify things then that's the way to go.

Copy Text ⇢ Search & Replace the Clipboard ⇢ Paste.kmmacros (3.7 KB)

It'll be easier for @wogo to learn a little about regular expressions than to learn about a scripting language and regular expressions.

-Chris

10

Your script looks fine.

The input is the text in the clipboard.

pbpaste is a system command that reads the text from the clipboard and prints it out. So

$data = `pbpaste`;

sets the perl variable $data to the text on the clipboard.

If you prefer, put it in a Keyboard Maestro variable (say “Input Data”), and then the lines:

$data = `pbpaste`;
$INPUT = new IO::Scalar \$data;

would be changed to:

$INPUT = new IO::Scalar \$ENV{KMVAR_Input_Data};

The output is pasted according to your action, or you can send the output somewhere else.

1 Like
11

@ccstone

The big question here is why are people fooling with a script when Keyboard Maestro can more simply do this kind of find/replace itself?

I am using the KM search-&-replace (with regex) for simple things, but I often have to change a long list of characters or strings into another long list of characters or strings.

For example, I often deal with small bits of Hebrew text sprinkled within large blocks of English text. There can be encoding issues when moving the text between programs or platforms. The best system I've come up with to avoid problems is to convert all 30 some-odd Hebrew charachters into HTML entities before moving the text around and then converting back to make it human-readable form after it is settled. It is a simple find-&-replace but done 30 times for each conversion.

Doing this as a script in a System Service was relatively easy once I had the right magic incantation. Doing it using KM's built-in actions will involve a ton of clicking and switching and copying and pasting.

@peternlewis

Your script looks fine.

When I ran it, it didn't do anything. The output still had smart-quotes and em-dashes. I must be missing something.

@JMichaelTX (& everybody): I am currently on my phone, but I can give more details and screenshots when I get back to a computer. Thanks again for all the help.

1 Like
12

Hey @wogo,

Thank you for explaining; that makes perfect sense.

For this kind of job I usually employ the Satimage.osax AppleScript Extension.

http://satimage.fr/software/en/downloads/downloads_nextsmile.html

The Satimage.osax has the advantage of being fully Unicode-compliant – is easy to use – and is quite versatile.

------------------------------------------------------------------------------
#### Requires Installation of the Satimage.osax AppleScript Extension ####
------------------------------------------------------------------------------

set someText to "Now is the time for all good men to come to the aid of their country."

# Regular Expression:
set newtext to change "(N|t)\\w+" into "•••" in someText with regexp without case sensitive

--> "••• is ••• ••• for all good men ••• come ••• ••• aid of ••• cou•••."

# Literal Text:
set newtext to change "Now" into "Who" in someText

--> "Who is the time for all good men to come to the aid of their country."

# Groups of Literal Text
set newtext to change {"N", "o", "w", "men"} into {"W", "h", "o", "women"} in someText

--> "Who is the time fhr all ghhd women th chme th the aid hf their chuntry."

------------------------------------------------------------------------------

On my system I use a handler to make the calls a little less verbose:

The syntax of the call:

set someText to cng("findPattern", "replacePattern", dataVariable) of me

What a script with it looks like:

------------------------------------------------------------------------------

set someText to "Now is the time for all good men to come to the aid of their country."

# Regular Expression:
set newtext to cng("(N|t)\\w+", "•••", someText) of me

------------------------------------------------------------------------------
--» HANDLERS
------------------------------------------------------------------------------
on cng(_find, _replace, _data)
   change _find into _replace in _data with regexp without case sensitive
end cng
------------------------------------------------------------------------------

Now we'll work with the clipboard:

------------------------------------------------------------------------------

set someText to the clipboard
set newtext to cng("(N|t)\\w+", "•••", someText) of me
# Add more "cng" statements as needed...

------------------------------------------------------------------------------
--» HANDLERS
------------------------------------------------------------------------------
on cng(_find, _replace, _data)
   change _find into _replace in _data with regexp without case sensitive
end cng
------------------------------------------------------------------------------

From here we can set the clipboard to the massaged data, or we can output to a variety of scriptable applications or a file.

If you run a compiled-script instead of a text-script you can open it in the Script Editor.app (or Script Debugger if you have it).

This offers greater flexibility while testing and offers a better editing experience complete with find/replace.

I believe you mentioned you might like to have dynamic options based on the clipboard content.

AppleScript + the Satimage.osax will give you many options in that regard, and you should be able to learn how to do quite a lot for yourself without too much study.

Here's a trivial example:

------------------------------------------------------------------------------

set the clipboard to "What is the meaing of הָאֱמֶת?"

set someText to the clipboard

if someText contains "הָאֱמֶת" then
   
   set newText to cng("הָאֱמֶת", "emet", someText) of me
   
end if

------------------------------------------------------------------------------
--» HANDLERS
------------------------------------------------------------------------------
on cng(_find, _replace, _data)
   change _find into _replace in _data with regexp without case sensitive
end cng
------------------------------------------------------------------------------

Much more sophisticated things are possible of course.

-Chris

13

When you get a chance please post the examples I asked for above.
With those, I'm sure I can build a user-friendly JXA script that does the work, but uses KM for all the UI, so you won't have to learn or modify JavaScript.

14

I too have a lot of existing Perl scripts that do multiple subsitutions on a text selection. And I've found they work just fine in Keyboard Maestro, which makes them even more accessible.

What I do is select some text and run the macro. Period.

The macro copies the selection to the clipboard and then copies it to a variable which is then passed to the Perl script. After the script runs, the new text replaces the old text automatically. In the blink of an eye.

The attached macro has your code (cleaned up a bit) that I tested on this sample text:

“This is ‘a’ test.” And – if I say so myself – not finished …

to get:

"This is 'a' test." And -- if I say so myself -- not finished ...

To use the macro with any other set of regexps, just replace the existing s///g; lines with the ones you need. And change the trigger.

Here's what the macro looks like:

If you use the same trigger, you'll get a conflict palette that will let you choose the conversion you want.

This is stripped down (if doesn't check for a selection and I'm intrigued by Peter's use of pbpaste to read the clipboard instead of assigning a KM variable. But that seemed to confuse things, so maybe this approach (if less efficient) is easier to apply.

Perl Substitution.kmmacros (4.2 KB)

1 Like
15

@mrpasini and @wogo:

I think that is a great solution.
The trick is to name each Macro in a way that clearly identifies the changes to be made, and puts them in the order you want.

For example, creating 3 macros, all with the same trigger:

Replace Text: Quotes (all forms) with Std Quote
Replace Text: Hebrew Text to HTML Encode
Replace Text: Spaces with Underscores

would produce a Conflict Palette like this:

You then choose the macro by typing the character that is in grey.

16

Or you could just dump all three sets into one macro. There’s not much of a penalty for failing to find a match.

My workhorse style macro has just about 200 regexps in it and only a few have any work to do in any particular document. Still runs in the blink of an eye.

Nice having options.

17

That would be true for use cases where the RegEx is simple, and the text to be searched is short.

But if dumping all into one big search works for you, then you're right, keep it simple with one macro.
You can always start with that, and then, later, if you find things are a bit slow, you can optimize.

18

@mrpasini: Your perl script worked great of the non-ASCII punctuation, but it choked on the Hebrew.

Here is sample text in Hebrew including some Hebrew-specific symbols:

כך התרסק נפץ על גוזל קטן שדחף את צבי למים ₪ תנ״ך גמ׳

The perl script returned a bunch of lines like this:


Useless use of 1's complement (~) in void context at /var/folders/p3/9s42jc4j3cl1dcbqklj3c07w0000gn/T/Keyboard-Maestro-Script-F0809E3C-5615-4BB5-A174-40962DA0C3BF line 8.

Part of the problem is that Hebrew text reads from Right-to-Left, and when displaying it — particularly combined with punctuation or numbers — it switches to a different position in the line. I’m not sure how to explain it, but it looks like this:

With tabs between code and the find_text and replace_text:

With tabs removed:

You can see that some of the characters look like they’re switched around, but the script works fine in the System Service.

Here are the services I am using successfully to convert the Hebrew to code and back (note: When converting from code back to Hebrew, I only convert the alphabet, leaving the punctuation and currency symbol as is).

Heb txt>code.workflow.zip (38.5 KB)

Heg code>txt.workflow.zip (88.6 KB)

Also, here is a Numbers.app file with tables of all the conversions of Hebrew & code, and Hebrew & QWERTY-equivalent:

hebrew alphabet.zip (166.3 KB)

I expect this table (and the sample text above) will help @JMichaelTX with the JavaScript, but it looks like the Applescript solution may be the best option. I am about to test that myself now.

To be continued…

19

In my macro, I changed the syntax of the regex (tidied it up, as I put it). You'd have to do the same with the Automator services regexp. I've done one for you in this revised macro and added it to the original one that handled punctuation.

The test text was:

“Jofi,” said the guy with a ₪ on his cap.

and converted to:

"Jofi," said the guy with a ₪ on his cap.

which I believe is what you're after.

Reading direction won't matter. And neither will language, frankly. Syntax does, though. :smile:

Perl Substitution.kmmacros (2.5 KB)

20

YES! That seems to do it.

I will do a bit more testing to make sure is works as expected without any hiccups, but it looks perfect.

Thank you thank you thank you!

And thanks to everyone else who offered suggestions as well.