Removing duplicate strings from Array stored as KM Variable

I have an array where the strings are separated by ;

I want to remove the duplicate strings in the variable e.g.

My array1:

123 ; 123; 213;214 ; 123;

should give me the result of 123; 213;214
blank spaces do not make a difference to my intended use.

Thank you in advance for your assistance.

Hey Ali,

NOTE – the Perl script expects variable “theText” from Keyboard Maestro – i.e. KMVAR_theText.

To alter this you only need to change that variable name in one place.

Remove Duplicates from Array v1.0.kmmacros (2.4 KB)

The Perl is portable and should work on any macOS system back quite a ways.

I would use AppleScript and the Satimage.osax myself:

------------------------------------------------------------------------------
# REQUIRES the Satimage.osax to be installed.
------------------------------------------------------------------------------

tell application "Keyboard Maestro Engine"
   set theText to getvariable "theText"
end tell

set theText to fnd("\\d+", theText, true, true) of me
set theText to sortlist theText comparison 2 with remove duplicates
set theText to join theText using ";"

------------------------------------------------------------------------------
--» HANDLERS
------------------------------------------------------------------------------
on fnd(_find, _data, _all, strRslt)
   try
      find text _find in _data all occurrences _all string result strRslt with regexp without case sensitive
   on error
      return false
   end try
end fnd
------------------------------------------------------------------------------

-Chris

1 Like

Here is a JavaScript for Automation (JXA) solution.
Please let us know if this works for you.

BTW, you called your string an “array”. Do you plan on splitting the string into an array?


<img src="/uploads/default/original/2X/9/9fa034801953dfe7d1745afcb7087c7530a06e23.gif" width="70" height="17"> 2017-04-06 14:36 CT


###Example Results
<img src="/uploads/default/original/2X/3/3ecf6a850df6f30dbdc5745ca200107aae73bcd4.png" width="316" height="184">

<img src="/uploads/default/original/2X/b/b4f7d2fc5f658ab156ae4fb4e12c9d0c1f8e6d93.png" width="487" height="184">

###MACRO:&nbsp;&nbsp;&nbsp;Remove @Dups from String @Example

~~~ VER: 1.1&nbsp;&nbsp;&nbsp;&nbsp;2017-04-06 ~~~

####DOWNLOAD:
<a class="attachment" href="/uploads/default/original/2X/c/ca1b55b2b24319e3f4d9d1931605e4ad31fe140e.kmmacros">Remove @Dups from String @Example.kmmacros</a> (5.5 KB)

---

###ReleaseNotes

**NOTICE: This macro/script is just an _Example_**

* It has had very limited testing.
* You need to test further before using in a production environment.
* It does not have extensive error checking/handling.
* It may not be complete.  It is provided as an example to show you one approach to solving a problem.

**PURPOSE:**

* **Remove Duplicate Values from String Separated by Semicolons (`;`)**

AUTHOR:		@JMichaelTX

REQUIRES:

* macOS 10.11.6+
* Keyboard Maestro 7.1+
* Uses ES6 "Set" function
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set


REFERENCE:

This macro built in response to this topic/post:

Topic Title:	Removing duplicate strings from Array stored as KM Variable
URL:		https://forum.keyboardmaestro.com/t/removing-duplicate-strings-from-array-stored-as-km-variable/6761?u=jmichaeltx

Date:		2017-04-06
Author:		demirtas1


---

<img src="/uploads/default/original/2X/4/498ec33e104126ccab7ce3d1a1967f25019617a4.png" width="503" height="1429">

###JXA Script
```javascript

'use strict';
(function run() {      // this will auto-run when script is executed

/*
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
PURPOSE:  Remove Duplicate Values from String Separated by Semicolons (;)
VER:      1.1    2017-04-06
AUTHOR:    @JMichaelTX
REQUIRES:
  • macOS 10.11.6+
  • Keyboard Maestro 7.1+
  • Uses ES6 "Set" function
    https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Set
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*/
//--- GET SOURCE STRING FROM KM VARIABLE ---
var kmeApp = Application("Keyboard Maestro Engine");
var sourceStr = kmeApp.getvariable('SourceStr');

//--- For Testing only ---
// sourceStr = "; 123 ; 123; 213;214 ; 123;"
// sourceStr = "; 07405574904; 07738528636; 07931797376; 07931797376; ;; 07789102621; 07899786063;"

//--- REMOVE SPACES ---
sourceStr = sourceStr.replace(/ /g,"")


//--- SPLIT STRING INTO ARRAY ---
var valueList = sourceStr.split(";")

//--- GET UNIQUE VALUES ---
// (Requires macOS 10.11.6+ for ES6 feature)
var uniqueList = Array.from(new Set(valueList));

//--- REMOVE EMPTY VALUES ---
var uniqueList = uniqueList.filter(function(val) {
  return val !== "";
});


return (uniqueList.join(";"))

}  // END of function run()
)();
```
1 Like

Hi, thank you for the macro its does not work with:

; 07405574904; 07738528636; 07931797376; 07931797376; ;; 07789102621; 07899786063;

Please assist
Thank you once again.

This is a different pattern than in your original request:

What are the expected results for this new pattern?

1 Like

Duplicate mobile numbers should be removed, sorry about my earlier response i thought that the same pattern would apply here. In some cases the numbers would start with 44 and not 0

That is a big change in your requirements.
I’m not sure I fully understand what you want.

Please provide real-world examples (for all possible patterns) of:

  • Source data BEFORE any processing
  • Expected results
1 Like

Hi,
Sorry about the confusion the expected results would be as follows:

; 07405574904; 07738528636; 07931797376;; ;; 07789102621; 07899786063;

In this example there was only one duplicate number: 07931797376
I just deleted one of the duplicated to keep a unique

So, do you really want to keep all of the extra spaces and extra semicolons (;) in the final output? What is the point of that?
How are you going to use the output of this process?

1 Like

Isn't this correct?

1 Like

Hey Ali,

Both my KM macro and the Satimage.osax-based AppleScript work fine with that dataset.

-Chris

1 Like

I will test this tomorrow morning and respond, many thanks for all your assistance.