Regex Using a Variable as Replacement Including Capture Group

[Augustin]

Hello everyone,

My apologies if this question has already been asked and solved, but I haven't had any luck with a search of the forum.

For my teaching and my research, I use a macro to try to change texts written in Middle French into modern spelling.

For this, I have a (long) list of Search & Replace statements using regexes placed one after the other in a text file. They are listed following the same pattern for each line: mySearch/myReplace.

The macro gets the contents of each line, declares the variable mySearch, declares the variable myReplace, and performs a replacement by calling these two variables.

My problem is that it doesn’t work when there are groups captured in the myReplace variable. (It’s not the same behavior when I number ($1) or name (${name}) the groups, but it doesn’t work in both cases.

So, is it possible to use the captured groups in a variable that will be called as a replacement in a search and replace command ?

I guess there are conflict issues that I am not aware of.

Here is my test macro with just a few examples.

Thanks in advance for your help !
Augustin

---

Mise en forme | Anciens | Substantifs & adjectifs TEST.kmmacros (7.2 KB)

[thoffman666]

I don't think I understand why you're using nested loops and substrings. Why not just do this?

[tiffle]

Hi @Augustin, welcome to the KM Forum!

I've looked at your macro and can offer 2 comments:

1. The method you use to extract the search and replace patterns from your mySearchReplaceList is overly complicated and can be simplified considerably by using KM Arrays with a custom delimiter. I've done this for you in the new version of your macro below.
2. I've tested your last search pattern ([dl][’'])escri(?![m]) in regex101.com which shows there is something wrong with it. Unfortunately I have to leave right now so I can't offer a solution but maybe someone else will be able to.

Here is my updated version of your macro for you:

Download Macro(s): Mise en forme | Anciens | Substantifs & adjectifs TEST v2.kmmacros (4.4 KB)

Macro-Image

Macro-Notes

• Macros are always disabled when imported into the Keyboard Maestro Editor.
  • The user must ensure the macro is enabled.
  • The user must also ensure the macro's parent macro-group is enabled.

System Information

• macOS 10.14.6
• Keyboard Maestro v10.2

Apologies for not being able to go "the whole way" for you!

[Augustin]

Oh, thanks so much to both of you for your far more elegant suggestions than mine! I'm just starting out with Keyboard Maestro, and I'm still trying to get the hang of all the possibilities it offers.

I have tried both of your suggestions, but each time it does not solve my first issue.

The replacement capture groups ($1, $2, etc.) do not work any more when they are placed in a variable. But maybe it just can't be done this way...

Thanks again for your time and effort in answering my questions!

Here is an updated version of my macro:

Mise en forme | Anciens | Substantifs & adjectifs TEST v3.kmmacros (4.1 KB)

Macro-Image

[tiffle]

Ok @Augustin I have the solution for you and it is based on the contents of this discussion in the KM forum:

Filter Variable to Escape Regular Expression Special Characters

Is there any way to filter Variable to escape RegEx special characters?

I'll leave it to you to read and digest that discussion so you'll understand what's going on but the modifications to your macro consist of the two red-coloured actions in this new version:

Download Macro(s): Mise en forme | Anciens | Substantifs & adjectifs TEST v4.kmmacros (5.3 KB)

Macro-Image

Macro-Notes

• Macros are always disabled when imported into the Keyboard Maestro Editor.
  • The user must ensure the macro is enabled.
  • The user must also ensure the macro's parent macro-group is enabled.

System Information

• macOS 10.14.6
• Keyboard Maestro v10.2

[Augustin]

Thank you very much @tiffle for taking the time to help me!

At first I thought that your macro solved my problem, but in fact it doesn't: in your macro none of my regex with special characters work anymore (whereas before it was only those with capturing groups).

If I understand correctly, the two actions you added disable all the special characters in my regex. But, I need precisely the opposite: I need them to be active and especially the capturing groups in the replacement regex ($1, $2, etc.), which is not the case at the moment.

Thanks again for your help, I appreciate it!

[tiffle]

@Augustin

but in fact it doesn't

Oops.

@Augustin

the two actions you added

were a red herring - sorry about that.

Looking closely at what you've provided I see that the third search/replace pattern is incorrect (as I said in my original reply to you). It should read:

(?<name>[Cc])hresti/${name}hréti

which is the syntax supported by KM and documented here: Regular Expressions | ICU Documentation.

However, I have to ask the question: why do you need a named capture group there when this will do just as well:

([Cc])hresti/$1hréti

so, making changes to the macro as follows:

1. Change the first 2 Set Variable actions to "process nothing" - otherwise the \b somehow gets lost.
2. Simplify the regex that extracts the search and replace patterns to (^.+?)/(.*)
3. Changed some variables to Locals (which I prefer).

gives this result:

bâtir
bêtes
$1hrétiens
$1criture

So close yet so far.

I'm afraid I do not know why KM is behaving like this so I'm going to ask Peter...

@peternlewis - do you know a reason why a search pattern of ([Cc])hresti with a replace pattern of $1hréti (both held in KM variables) when applied to the text string Chrestiens should return $1hrétiens as opposed to Chrétiens?

Both this example and the last example in this test macro exhibit the same puzzling behaviour.

Download Macro(s): Mise en forme | Anciens | Substantifs & adjectifs TEST v5.kmmacros (5.8 KB)

Macro-Image

Macro-Notes

• Macros are always disabled when imported into the Keyboard Maestro Editor.
  • The user must ensure the macro is enabled.
  • The user must also ensure the macro's parent macro-group is enabled.

System Information

• macOS 10.14.6
• Keyboard Maestro v10.2

[Augustin]

Again @tiffle, many thanks for your investigation on this!

@tiffle

However, I have to ask the question: why do you need a named capture group there when this will do just as well:

([Cc])hresti/$1hréti

Just to answer that: I have no real reason to use a named capture group here, I was just trying another syntax to see if I had better luck with it. But in my use of regular expressions, I use exclusively the numbered capture groups ($1, $2, etc.).

At last, the result you get is indeed the result I struggle with! You have perfectly summed up my issue (and greatly simplified what I had unnecessarily complicated).

[ccstone]

Has anyone here been able to make a named capture group work properly under any circumstance?

I cannot get them to work in CotEditor (on Mojave) – period.

[tiffle]

Yes, I get them to work on regex101.com and also RegExRX app.

[Augustin]

Thank you @ccstone for your contribution, but in my case the numbered capture groups ($1, $2, etc.) don’t work either…

[tiffle]

@Augustin

I use exclusively the numbered capture groups

Personally, I have never found a need to use named capture groups so I do as you do.

I should add, by the way, that even using the KM-supported syntax that I posted previously, the replacement still fails which is why I asked whether you needed to use named groups. So your response of

@Augustin

I was just trying another syntax to see if I had better luck with it

was really the same thing I was trying. I guess we were both getting a bit desperate

[ccstone]

@tiffle

Yes, I get them to work on regex101.com and also RegExRX app.

Neither one of them support ICU RegEx...

[tiffle]

@ccstone

Neither one of them support ICU RegEx...

OK, but here’s the test I did regex101: build, test, and debug regex using the ICU syntax I posted previously.

I assumed since it worked under Pcre and was also ICU-correct then it should work in KM. Silly me ?

[Nige_S]

@tiffle

So close yet so far.

I'm guessing you're going one level of indirection too far -- by the time the "replace" variable is expanded it's treating the $1 as literal text. Compare it to removing the $1s from your list text and then using $1%Variable%LocalmyReplace% as the replace -- works fine when there is a capture group but fails and uses "$1" as a literal when there isn't.

[tiffle]

@Nige_S

too far

[tiffle]

Hi Chris,
I thought I'd do a test of named capture groups in KM. Here's the test macro:

Download Macro(s): Test Search Replace with Named Groups.kmmacros (3.6 KB)

Macro-Image

Macro-Notes

• Macros are always disabled when imported into the Keyboard Maestro Editor.
  • The user must ensure the macro is enabled.
  • The user must also ensure the macro's parent macro-group is enabled.

System Information

• macOS 10.14.6
• Keyboard Maestro v10.2

Running it gives the result:

Search Pattern:	(?<name>[Qq])ui
Replace Patten:	${name}ua
Input:			The quick brown fox
Output:			The quack brown fox

so it does seem to work OK.

[peternlewis]

@tiffle

@peternlewis - do you know a reason why a search pattern of ([Cc])hresti with a replace pattern of $1hréti (both held in KM variables) when applied to the text string Chrestiens should return $1hrétiens as opposed to Chrétiens?

Sure. Because no further processing happens after the token expansion that expands the replacement variable. So just like if the replacement text was %LongDate%, your result would be %LongDate% not the date, the $1 has no meaning within the variable.

[tiffle]

Hi @Augustin - so from what @peternlewis (thanks!) is saying, it is not possible to achieve what you want. My old brain cells are having a hard time getting to grips with the explanation but regardless - the fact is it does not work

[Nige_S]

@tiffle

Hi @Augustin - so from what @peternlewis (thanks!) is saying, it is not possible to achieve what you want.

I think it is @tiffle, by riffing off your original idea. A single replacement pattern should be quite simple...

Go through each off the search/replace pairs, splitting them out. If the replace variable contains replacement patterns, pseudo-array on those and build your replace text field using eg %Variable%Local_replace[1]$1%$1%Variable%Local_replace[2]$1% But if the replace variable doesn't contain a $1 use the variable as usual.

A multiply-occurring single replacement pattern or multiple patterns will be more difficult and will probably need a different approach...