MACRO: Get List of RegEx Capture Group of Multiple Matches

MACRO: Get List of RegEx Capture Group of Multiple Matches

DOWNLOAD:

[JS] Get List of JavaScript Functions in Selection.kmmacros (33 KB)

I wrote this macro to extract a list of functions from a JXA/JavaScript script library, but it serves as a good example in general of how to process multiple RegEx matches, get the same Capture Group from each match.

The macro is setup to use either KM built-in Actions, or a custom JXA script.
It was not obvious to me how to use KM Actions at first, but with some great help from Peter (@peternlewis), I was able to design this macro.

If you want to Get List of RegEx Capture Group of Multiple Matches, then here is the key:

  1. Use the "For Each" Action to get each match into a variable
  2. Then use a "Search Variable" Action to get the Capture Group for that match

Both Actions can use the same RegEx pattern.


VER: 1.1 Last Update: Tue, Feb 16, 2016
Author: @JMichaelTX

PURPOSE: Extract a list of functions from Selection, Put on Clipboard

HOW TO USE:
(1) Select the text that contains the actual functions (like a Script Library file)
(2) Run this macro

NOTE: Macro is Setup in Demo Mode

You will need to enable/disable certain Actions for production use.
See the Comments in the macro for details.



JXA Script (initially disabled)

function run() {
'use strict';

var app = Application.currentApplication()
app.includeStandardAdditions = true

var myString = app.theClipboard();
var myRegEx = /^\s*?(function[ ]?\w*\(.*\)).*[\n\r]/gm;

// Get an array containing the first capturing group for every match
var matches = getMatches(myString, myRegEx, 1);

var matchesStr = matches.join("\n");

return (matchesStr)
//~~~~~~~~~~ END OF MAIN SCRIPT ~~~~~~~~~~~~~~~~~~

function getMatches(string, regex, groupIndex) {

  groupIndex || (groupIndex = 1); // default to the first capturing group
  var matches = [];
  var match;
  while (match = regex.exec(string)) {
    matches.push(match[groupIndex]);
  }
  return matches;
}	// END function getMatches

}	// END function run
7 Likes

I have no words to describe to you how grateful I am for this post. I managed to stumble across it (eventually), and it showed me how to solve almost all the issues I was having. A little Googling and I solved everything else.

I’m sitting here around 3am, which should tell you something already about how much this has been festering in my brain.

The getMatches() function was a godsend, because I could not get non-capturing groups, or lookbehinds, to work.

Here’s what I ended up using for my purposes (slightly edited), thanks to you and Google:

function run() {
'use strict';

var _kme = Application("Keyboard Maestro Engine");
var _prefix = _kme.getvariable("thePrefix");
var _string = _kme.getvariable("theSource");

var _re = new RegExp("([>%])(" + _prefix + "[^<%>=+-/\.&]*)", "gm");

var _matches = getMatches(_string, _re, 2).filter(onlyUnique).sort().join("\n");

return _matches;

function getMatches(string, regex, groupIndex) {
  var _matches = [];
  var _match;
  while (_match = regex.exec(string)) {
    _matches.push(_match[groupIndex].trim());
  }
  return _matches;
}

function onlyUnique(value, index, self) { 
    return self.indexOf(value) === index;
}
}

Again, seriously undying gratitude!

1 Like

Like @DanThomas DanThomas, I have no words to express my gratitude for your generosity in posting this script, @JMichaelTX. I’ve been surprising myself lately, but things like macros, RegEx, JavaScript, and other things not covered in a standard Word Nerd degree are befuddling to me. :slight_smile: This example macro and the fabulously clear documentation within it has provided an eleventh-hour rescue from utter and certain disgrace in the eyes of my peers. Thank you so very much!

I’m wondering if either you might provide some guidance on whether it’s possible to include a location or maybe even the text that surrounds the matched string in the results window?

I used the For Each method in the example because… well… Javascript :dizzy_face:

Thanks!

1 Like

I'm not sure I understand what you are asking for, but if you just want to see, for learning purposes, the text around the matched text, then I'd suggest that you use the RegEx tool provided by regex101.com.

It is a great tool for developing, testing, and learning RegEx. It shows you the RegEx pattern, the source text highlighted in colors with the matching text, and an Explanations panel that tells you what each piece of the RegEx pattern does.

Check it out, and let us know if you have any questions.

Hi there,
I've been trying to wrap my brain around multiple matches and this seems like the answer but I'm still so confused. I have two main questions that are getting in my way and two boring, straightforward (if possible) answers would be wonderful.

  1. What is a MatchString? Is that the name you've chosen for a variable or some kind of KM command?
  2. Where do you put the regex you want to be executed? I've got working code (\w+ (\D+)) on regex101 but I can't get it to work on KM. What's the difference between the substrings regex and the regex in the searching a variable section?

Sigh, this is where I always break down with KM, the syntax. I know exactly what I'd like to do and have no idea how to find out what the commands even mean. I'm reading the tutorial and then suddenly there's "MatchString". I get that this is written for someone more advanced but it takes so many hours to find out things like, you have to change the modifiers on regex.

I'm not complaining about KM or anyone's posts; mostly sharing this to make my perspective and knowledge-level clear since I don't know where to even start with making my code problem clearer.

Wow, and as usual, just when I give up and post, I test one more thing and it works. This app is both the greatest thing I've ever found for my Mac and the most frustrating part of my computing life.

If you are referring to this:

then, yes, it is just a KM Variable name that I have assigned to this Action. For more info see For Each action.

It depends on the purpose/objective of your workflow, and what specific results you want. KM supports RegEx in a variety of Actions. For more info see Regular Expressions . This KM article shows examples of three of the most common uses of RegEx in KM.

I'm not sure if you are referring to KM in general, or to RegEx. RegEx is a complex language separate and apart from KM, and is NOT easy to learn.

The way to learn the syntax of any language, including KM, is to study the tutorials and user guides for that language, and then to actually write programs (macros, scripts, code) using that language. Just like in school, the real learning comes with the doing, and doing your homework, first with simple examples, is a good approach.

You say you " have no idea how to find out what the commands even mean".
KM provides lots of places to easily get help/info on the objects (Macros, Actions, Tokens, Functions, etc) that it uses:

  • In the Action Gear menu (top right of Action) there is a "help" item
  • When you view the menu to select an Action, Token, or Function, hold down the OPT key to get a link to the KM Wiki Help for that item
  • Then there is the entire Keyboard Maestro Wiki itself. Starting on its home page, or searching using its Search tool, you can easily find details on every KM entity.

To learn RegEx, I suggest starting with Regular Expression Tutorial .

Finally, I'll leave you with this:

Getting Started with Keyboard Maestro

For more help, see Getting Started with Keyboard Maestro and the Forum .

3 Likes

@dduncombe and @JMichaelTX I think some of the issue with this one is that there is a variable in another variable. Which is ok but can be confusing at first encounter? Am I right? Variables can be like Russian Dolls and nest inside each other. Tokens are very much like variables that can be used inside other variables for example.

Not really. There is an exception where the value of ONE Variable contains the name of another Variable, but in general, KM Variables are simple, scalar, text variables, that can contain ONLY text.

KM Tokens provide a means to get information about objects (like web pages and windows) in the user's environment. A KM Variable is one such object. So you can refer to a KM Variable in two ways:

  1. Directly using just the Variable Name in certain fields, like a Calculation field or a Variable field
  2. Using the Token format of %Variable%YourVariableName% in Text Token fields

So, generally, Tokens can NOT be used "inside of other variables".

For more details, see Variables in the KM Wiki.

1 Like

OK, I see what you mean.

I appreciate this response a lot, I was whining and genuinely did not want to take that much of your time.

Thank you for clarifying that MatchString was a variable name, I did not follow that at all.

I was asking about placing the regex in your action specifically; I thought maybe the first one was a set of instructions setting up the variable for the second regex function. All in all your regex and the macro in general took me some time to peel apart, even with your outstanding notation.

I was referring to KM rather than regex. I think I follow your advice for learning syntax almost to the letter. I spent hours reading, testing, playing. You're right, there is a ton of info. More than one could expect or hope for both in the wiki and the forum. What I was getting at was the way KM seems to be just out of reach sometimes. I'm definitely a slow learner and this might just be me but it feels like I'm missing a glossary that everyone else has. I don't think this is anyone's responsibility but my own.

That said, I have been working on macros for pretty much three days straight and finally tackled some regex. In the end I was able to update my whole workflow and finally, finally gotten the ability to parse variables. It was amazing and life-changing, literally. Your posts were an essential part of that; I know when I see your avatar I'm going to get reliable info.

Also, thank you for all the links!

1 Like

@JMichaelTX and others take a lot of time and trouble to help other users and have great knowledge. If it is any comfort I have had a lot of trouble with variables in Keyboard Maestro myself and am still struggling with them at times in new contexts. I have exactly the same problems you do to, in some cases 'had' I guess. Identifying variables in macros posted by other people is still tricky for me. This one confused me too. It sounded like an instruction, 'match the string to ' or something like that.

2 Likes

One way to view KM is like an electronic Lego set. :wink:
The way KM is designed is such that you can combine its various elements (Actions, Tokens, Functions) in an almost infinite number of combinations -- giving you great flexibility.

Of course, the chief challenge with that is how to combine these elements.

There is no silver bullet here, except maybe by examining and analyzing the many Macros in the Forum Macro Library, and especially in one of the topics in that section, Best Macro List .

I can share with you that some elements are harder to understand at first exposure.
Although I have a long history of programming, and fully understand the many different types of programming loops, it took me a while, with some encouragement from others, to fully grasp and know when to use the very powerful KM Action For Each action.

When designing a macro, I knew I needed a loop, but for some reason, the For Each Action would just not pop into my head. It was only have having used it many times that finally I have internalized it.

So, I know I've said this before, but nothing replaces actual doing. :wink:
The Forum is definitely here to help you, but, to be honest, you will learn, and retain more, if you can just keep slugging away at a problem until you resolve it yourself.

OTOH, don't keep beating your head against a brick wall until its bloody and you're ready to quit. The key is in finding the right point at which to ask for help. I've got to say there is much satisfaction in figuring out a solution to the problem yourself.

I don't know how many of you have experienced this, but I, and some of my friends, have noticed that sleeping on a problem will often bring new insights, even solutions, either during the night, or the next morning. Worth trying.

I can tell you this: If you will keep on trying, the day will come when designing a KM Macro, and even writing a RegEx pattern, that it will become almost as natural to you as speaking your own native language. However, I will quickly follow that with the well known adage "Use it or lose it". This doesn't mean you have to spend hours every day writing KM Macros, but finding the time to spend at least a few hours each week should keep you sharp.

Good luck to all!

5 Likes

I forgot to mention, that KM does indeed have a glossary.
If you find terms that need to be added to it, please let us know.

Glossary in the KM Wiki.

2 Likes

The feeling I got when I finally got everything working was maybe the most satisfying moment of the year! I can't believe I made progress with regex.

The miserable letter you saw originally was definitely being typed by someone with a bloody head! Figuring out what and when to ask is maybe the hardest part; I swear I've posted like three times and each time I figure it out after I type out the question... hmmmm. I did sleep on it a few times too.

Thank you again or all the help and your follow up with the KM overview; it's reassuring to know that you face some of the same boundaries and problems. I'm really looking forward to looking through the macros for idea and to add a few layers of possibility to my imagination.

1 Like

Thanks @JMichaelTX for the encouraging reply. How about "Variable" in the glossary :+1:

The kind of comment you and I made are important here. It can seem at times, if you look at some posted macros that the whole thing is totally unreachable. The Lego analogy is an excellent one: I am learning slowly regex and so on but you know, my own use of Keyboard Maestro doesn't really use it much. My own use is focused on palettes, opening sets of websites, sets of apps, spell checks. I find I use the 'if then' control features a lot. Some other users who are advanced, David Sparks for example, say they hardly use them. Go figure. Usage can, I think, become very idiosyncratic and that is, well, that is what it is for.

1 Like

Done. Please review and provide any comments to improve clarity.

1 Like

Very clear what was put in the Glossary, it seems just right to me. The whole page is useful and I wasn't really aware of it to be honest!

Hello @JMichaelTX,

I was searching for this function and found a few relevant posts including this. First, I want to thank you for posting the example. It is very helpful.

I do have a question. The following question seems to be redundant to me:
image

You already have a match here:
image

Why then search for the same pattern in itself (the first screenshot in this reply), which will surely return itself, and save to the same variable name?

Maybe this will help:

1 Like