Keyboard Maestro “03)Link - Load Variables” Macro

Quinn · August 16, 2017, 7:59am

Hi, Everyone!

This macro is just one of over a dozen that I'm working on as part of an effort to automate a huge part of my fledgling business. I'm not any kind of a programmer, so it's been a steep learning curve. Fortunately for me, this forum is full of very knowledgeable people whom I have stolen countless ideas from. Of course, I still have so much to learn.

So, I was wondering if anyone could help me figure this out.

I'm using a regex to search text formatted thusly:

``First Name:%<>%
Last Name:%<>%
Cell#:%<<Cell#>>%
Office#:%<<Office#>>%
Business Email:%<>%
Business Address:%<>%
RE/MAX Website URL:%<<RE/MAX Website URL>>%
Facebook URL:%<>%
Youtube URL:%<>%
LinkedIn URL:%<>%
Twitter URL:%<>%
Business Hours:%<>%
About Me:%<>%```

As you can see, each entry will be formatted Thing Name:%About Thing% (the <<>> are tags that get replaced by other data. The "%"s are there to make it easier to capture the text between them since it could be absolutely anything spanning more that one line.

I want to take each line entry and break it up into its two parts for further manipulation.

For example:

First Name:%<<First Name>>%

gets broken up into

"First Name" and "<<First Name>>"

The macro then sets, creating if needded, the variable "First Name" to the text, "<<First Name>>"

Where I'm a little stuck is on making the macro get and process each match.

I have two questions:

Where am I going wrong with this approach?

and

Is there a simpler way that I'm just not seeing?

Please bear in mind before commenting that I will never admit to how much time I spent fiddling with regular expressions in order to produce the one you see here.

Thanks in advance, Guys.

Keyboard Maestro “03)Link - Load Variables” Macro

03)Link - Load Variables.kmmacros (5.4 KB)

peternlewis · August 16, 2017, 8:50am

Are the fields always in that order? Are they always fixed? Are all fields always there?

If so, then you can use a single regex to capture the whole lot into the set of variables.

If not, then you will have to parse the data, extracting both the field name and the value.

Can any of the data ever have a percent character in it?

JMichaelTX · August 16, 2017, 11:08pm

The major flaw in your script is that you need to do a For Each on each line on the Clipboard.

Here's my revision to your Macro.
My Changes:

I've added a prefix of "LLV__" to all of your variable names.

This is to avoid conflicts with variables from other macros, and to make it much, much easier to identify and clear/delete when needed.
Used a For Each Action on Lines on Clipboard
Delete all Variables (if any) at start of Macro
Revised AppleScript
- Add Prefix
- Use latest method of setting KM variables
Add Log of Results

One Benefit of Using a Variable Prefix

Please let us know if this works for you and/or if you have further questions.

###MACRO: 03)Link - Load Variables REV by @JMichaelTX

~~~ VER: 1.0 2017-08-16 ~~~

####DOWNLOAD:
03)Link - Load Variables REV by @JMichaelTX.kmmacros (9.2 KB)
Note: This Macro was uploaded in a DISABLED state. You must enable before it can be triggered.

###Example Results

Quinn · August 16, 2017, 11:02pm

Holy smokes! That looks great. I’ll test this out and get back to you.

I like your way of naming variables. I’m going to start doing that consistently in my other macros.

Thank you for this!

Quinn · August 16, 2017, 11:41pm

Thanks, JMichaelITX!

Okay, so this works nicely. I now need to extend it so that it can parse the text when entries may or may not span multiple lines.

To make it more clear, I can control everything except what ends up between the two % signs. That’s all submitted by the client in an electronic form. So these (and many other fields that I haven’t mentioned) could be lists, paragraphs, or whatever. Any text really. It doesn’t matter to me what it is. I just need to capture it in a variable so I can put it somewhere else specific.

The reason for the % signs is just to provide an easy way to bound the text if I end up needing to, as there shouldn’t ever be percent signs in that content that would throw the expression off.

I have a regex that handles all this (gloriously hideous as it is) either with or without the percent signs. It matches lists, paragraphs, single lines and every other configuration it’s likely to encounter. But it doesn’t work with this macro so I’m I’m guessing it has something to do with how the FOR EACH and SEARCH are working. Does it step through line by line. How would I go about making this able to handle text spanning multiple lines as one thing so that it steps from match to match instead?

Thanks

JMichaelTX · August 16, 2017, 11:50pm

Please post a real-world example of your multi-line values, and the RegEx you have so far.

You are correct that my solution requires each field and value to be on one line. But I’m sure we can adjust that.

BTW, this is why it is important at the beginning to post real-world examples of your source data, and expected result. Had I known from the beginning that you needed to handle multiple lines, I could have saved time by writing the RegEx and macro to handle that.

Quinn · August 17, 2017, 12:13am

Sorry about that. I thought I had mentioned that detail.

Anyway, here is an actual example. My regex works on this, I just don’t know how to get each pair of capture groups.

The regex: (?m)^(\w.+):((.+\n*)+)|(?m)^(?:\s+)|(\w.+):(?:\s|\S)(.+\w)

The text

Neighbourhood Name:<<Neighbourhood Name>>

First Name:<<First Name>>

Last Name:<<Last Name>>

Page HTML:<!—main photo-->
<div style="text-align: center;">&lt;Main Photo Here&gt;</div>%
<!--Request More Information-->
<div style="text-align: center;"><img src="http://150116017.homesconnect.com/AccountData/150116017/btn_request_more_info.png" alt="btn_request_more_info.png" title="Request More Information" style="margin: 0px;" width="279" height="59" border="0">
</div>
<div style="text-align: center;"><br></div>
<!--introduction-->
<div style="text-align: center;"><span style="color: rgb(0, 0, 0); font-family: &quot;Open Sans&quot;, Arial, sans-serif; font-size: 14px; text-align: justify;"><<introduction>></span></div>
<div style="text-align: center;"><br></div>
<!--table for gallery, brochure, property search, map, and schedule a showing buttons-->
<div style="text-align: center;">
  <table style="width: 100>>;" align="" cellspacing="1" cellpadding="1" border="0">
    <tbody>
      <tr>
        <!--Photo Gallery-->
        <td style="text-align: center; ">
          <!--link to gallery-->
          <a href="" title=""><img src="http://150116017.homesconnect.com/AccountData/150116017/PhotoGallery.png" alt="PhotoGallery.png" title="Photo Gallery" style="margin:0 0 0 0" height="100" align="middle" width="100" border="0"></a><br></td>
        <!--Brochure-->
        <td style="text-align: center; ">
          <!--link to brochure-->
          <a href="" title="Download pdf brochure" target="_blank"><img src="http://150116017.homesconnect.com/AccountData/150116017/Brochure.png" alt="Brochure.png" title="PDF Brochure" style="margin:0 0 0 0" height="90" align="middle" width="90" border="0"></a><br></td>
        <!--Schedule a showing-->
        <td style="text-align: center;">
          <a href="/contactme/default.aspx" title="Schedule an appointment"><img src="http://150116017.homesconnect.com/AccountData/150116017/ScheduleShowing.png" alt="ScheduleShowing.png" title="Schedule a Showing" style="margin:0 0 0 0" height="90" align="middle" width="90" border="0"></a><br></td>
      </tr>
</html>

What I’m doing is getting the information from the text file to the appropriate field in a browser-based program. Some of the values are also used for web navigation, SEO, and the like. The pages I’m navigating are company pages, and so have consistent URLs. I’ve therefore taken advantage of this with Keyboard Maestro and use the values for that as well.

So, once I’ve read the file I’ll be performing a bunch of work using the information and then eventually reading another file with different information in the same format and performing the exact same work. Rinse and repeat.

If I can get my macro to handle the above text, I can get it to do pretty much anything I need it to do. I just need it to be flexible and general enough to handle any changes to the text format that might be imposed on me in the future.

I appreciate your time. Thanks.

JMichaelTX · August 17, 2017, 12:59am

Quinn, I sense that we are dealing only with a part of your complete workflow. If you don’t mind laying out your complete workflow, starting with source data that you obtain from some source (where?), any revisions you need to make to the data, and then submittal to some other app/web site., then I might be able to suggest some improvements in your process.

AppleScript and JXA, combined with JavaScript insertion in a browser, is very powerful. There have been many AppleScript written to move data between apps and/or web sites.

JavaScript for Automation (JXA) is very powerful, and an excellent RegEx engine, and can make great use of JSON in moving data around.

So, if you don’t mind, let’s start at the beginning:

What is the primary source of your data?

How are you currently retrieving this data from the source?
How would you like to retrieve the data?
Do you need to store any of this data locally? If so, where?

I have an AppleScript SQL Library (based on Obj-C), that is very fast and very powerful, if you need to create, put, or get data in SQL databases.

If this seems like I’m making it more complicated than you want, then we can just deal with your immediate technical issues with KM.

In that context, I don’t understand your last example data
It doesn’t have all of the fields that your first example set did
What is the purpose of the “Page HTML:” ?
- Do you just want to extract that into a field, or is it the source of something?

I think it would be best to start with a real-world example that has all possible fields in it. Don’t use the “<>” unless it is actually part of the real data.

Also, will you real-world data sets be limited to one entity (e.g. one person) per set/file, or might it have multiple entities? This can make a huge difference in the processing.

OK, I apologize for a bit of rambling, but I think I’ve stated all of the questions (for now) that I need answers to. Please let me know you’d like to proceed.

peternlewis · August 17, 2017, 1:01am

Yes, you can’t use For Each Lines in Clipboard because of the multi-line nature of the data.

Instead you just need to use For Each match in Clipboard, and have the matches use a similar regex to the Search Variable.

It should use the (?s) flag so “.” matches anything, and (?m) flag so ^ and $ match start/end of line within the string. It should use non-greedy search so that it captures the minimum amount. And since you control the output, you should enclose the values in something that clearly cannot be contained in the value. Something like KMVALUEKM%value%MKEULAVMK. So a regex like:

Something like (?sm)^(\w.+)?: KMVALUEKM(.*?)MKEULAVMK$

And then the same regex can be used in the Search Variable action.

Quinn · August 18, 2017, 2:56am

Thanks for the feedback. I’m going to take you up on your suggestions and I am currently working on putting together a complete rundown of the workflow. I welcome any suggestions you may have.

I hope you don’t mind holding tight for a little bit while bang this out. I’ll return with answers to all your questions and you can have a look at my plan. Just promise not to laugh.

Thanks!

JMichaelTX · August 18, 2017, 3:38am

No problem, take your time. I’m in no rush – I’ve got plenty to do.

Keyboard Maestro “03)Link - Load Variables” Macro

Options