Folding long lines to a maximum length

[truecolor]

Hi there,

First of all ... I´m a absolutely beginner. Keyboard maestro is so awesome, no way .. I have to buy it and learn how to work with it and use it.
Maybe ´cause my small english mistakes ... I´m from germany (please no expressions of pity. Here in germany it's bad enough as it is )

I came up with a nice little project to get to know you that could be very useful in my everyday life. It's about the fact that I often send text messages that are a little longer than the 140 characters allowed per message. So I thought why not let Keyboard Maestro do the work for me.

My key data were: Texts longer than 140 characters should be divided into 140 character blocks and the individual blocks should be provided in the clipboard. The separation should not take place within words, but in the spaces before the 140 character limit. Paragraphs should be retained. No Apple Automator should be used, but RexEx should be used instead.

Since I'm still pretty much at the beginning, I thought it would be a good idea if I added ChatGPT to help. But after hours of testing, I got the feeling that ChatGPT is overpowered.

Towards the end, it feels like ChatGPT was desperately trying to search for the solution like a needle in a haystack. I was so frustrated, I took all my anger out on ChatGPT first.

Of course, that didn't help because it's a machine. But after that I felt visibly better. I can laugh about it now.

Nevertheless, I would be happy if my macro would finally work. So I had a second idea: there's a forum for everything. And here I am, hoping that someone can help me. If it helps, I can upload the macro.

Best regards from Germany, Frank

[Airy]

Most apps don't have a 140 character limit anymore. Which app are you using that has a limit? I suspect it will help you if you upload your macro so we don't have to ask you basic questions like this.

This is a text manipulation puzzle, which are the kind I enjoy. In fact, macOS has some text utilities that may help solve this problem.

You said, "paragraphs should be retained." How do you "define" a paragraph? Does a carriage return "define" a paragraph?

And what do you want the code to do if a paragraph or a word breaks the 140 character limit. Do you want the macro to terminate with an error, or do you want it to break up the text anyway?

EDIT: (30 minutes later) I think have a solution, assuming I understand your problem correctly, (which may not be the case.) My solution is only three actions long! But I see Nige is working on the problem and his solutions are usually twice as good as mine.

[Nige_S]

@truecolor

Keyboard maestro is so awesome

KM is, indeed, awesome. Automation is awesome. And the most efficient automation of all is to let someone else do the hard work, then use it yourself.

In this case the hard work can be found in the fold utility, which comes installed with macOS. If you open Terminal and check folds man page:

man fold

...you'll see that there are options to fold after the last blank character that's still within your width limit (-s) and that you can set the width to use (-w 140 in this case).

And KM has an "Execute a Shell Script" action... Demo time!

Fold Demo.kmmacros (2.7 KB)

That's the easy bit done. Now for the difficult bit...

Any macro starts with a plan. You then break that plan into well defined steps, that break each of those down and further define them until you are left with a series of steps that can be easily converted into KM actions.

The best plan is one that you can put in front of someone else who has no prior idea what you are trying to do, no knowledge of the applications you are using, and they can follow it line by line and achieve the goal without asking any other questions.

All we know about your plan is that you want to split some text into chunks of 140 characters or fewer, splitting on the nearest blank. Where's the text coming from? Where is it going to? What other factors might be in play?

An easy way to sort all this out is to work through your process manually, taking notes as you go. Tidy that up, make explicit any assumptions you've made, post the plan here -- and watch the suggestions roll in!

[truecolor]

Which app are you using that has a limit?

Simple mobile phone SMS (Short Messages).

BTW. There is a macro collection here in the forum. Many users have uploaded their macros there as screenshots. These are very long screenshots. How do I get my macro as a screenshot saved completely as a photo?

You said, "paragraphs should be retained." How do you "define" a paragraph? Does a carriage return "define" a paragraph?

Simply paragraphs, as they are also used here in the forum.

I used "Lorem Ipsum" for the test runs.

And what do you want the code to do if a paragraph or a word breaks the 140 character limit. Do you want the macro to terminate with an error, or do you want it to break up the text anyway?

The text should be separated after 140 characters at the latest.

If there is a word at this point, (e.g. RegEx) should search for the next space from back to front and then separate the text from the whole text when it finds a space.

[hemicyon]

@truecolor

How do I get my macro as a screenshot saved completely as a photo?

With the macro selected in the macros pane of the editor, you can select Edit -> Copy as -> Copy as Image. This menu item is also available for selected actions inside a selected macro.

You can also upload your macro and image together using the share function, detailed here in the wiki.

[Airy]

@truecolor

Simple mobile phone SMS (Short Messages).

That's not an app. In order to make a macro for you, I must know the name of the app.

I will assume that your app is the Messages app in macOS. In that case, here's a full solution. Basically, Nige had the same idea as me.

Try my macro, after filling your destination in the orange action. It will break up your message into chunks of a size that you specify (in the second parameter) and send each chunk out using the app Messages.

Of course, this is completely useless because Message has no limit on message length.

[truecolor]

This is what ChatGPT tried to sell me as an efficient solution.

[Airy]

@truecolor

This is what ChatGPT tried to sell me as an efficient solution.

ChatGPT came up with a really long, complicated solution. Why not try Nige's solution or my solution and see if that meets your needs?

P.S. In your code, you define a paragraph by a double carriage return, which is not what you said earlier, when you said, "Simply paragraphs, as they are also used here in the forum". On this forum a single carriage return separates paragraphs. This is a big difference and could make my code (and Nige's) incorrect for your needs. This is why it's important to make your requirements clear.

[truecolor]

Actually my brain is empty. Here is 11:13 pm, so tomorrow (in 10h) I will start again ... together with you both solutions.

Many thanks to you all

[Airy]

Ok! Thanks.

[Nige_S]

@truecolor

so tomorrow (in 10h) I will start again

When you do, start with @Airy's "What's a paragraph?" question. Then decide how you want those paragraph breaks handled -- if you have the text

This is paragraph 1.
This is paragraph 2.

...should these be one message or two?

If two then you can use fold to split the text then process it one line at a time, sending a message with each. If one then you will need to first process your text, replacing each paragraph delimiter (probably either \n or \n\n) with something the same length that isn't already present in the text, do the fold, then process each line and replace your substitute with \n before sending the message.

[Airy]

There are several questions he will have to answer before a solution can be written. Some of the solutions may require logic that backtracks. Hopefully the "fold" solution we offered is good enough. It would be good enough for me.

He still hasn't said which app has a 140 character limit, and his code doesn't answer that either, so I think this is just a programming exercise for him. Another reason I wanted to know which app he is using is because I think there's an entirely different solution for this problem.

[hemicyon]

Even though you've already received excellent suggestions from Airy and Nige, I simply cannot resist a regex puzzle.

This expression (?s).{0,140}(?:[^\S]|$) works even better than my previous regex to capture the 140-character text blocks as I understand the requirements to be.

You can live test the regex here https://regex101.com/r/fZqLKI/1 if you'd like, but I've also attached the macro if you prefer. Alles Gute!

Text in 140-Zeichen-Blöcke mit Regex.kmmacros (4.2 KB)

Old, near-perfect solution

This expression [\s\S]{0,140}[^\S] works almost perfectly. The only thing I can't seem to solve for are characters/words immediately preceding the end of the text. That is to say, the very end of the text must end with a space or ⮐ or it will be left out.

For example, notice how the last word and punctuation mark are not matched without the space proceeding them:

Text in 140-Zeichen-Blöcke mit Regex.kmmacros (4.2 KB)

@Airy

He still hasn't said which app has a 140 character limit,

I think he did. Simple Mobile appears to be a mobile service provider, and I'm guessing Short Messages is the name of their messaging app. iPhones are not nearly as prevalent in Europe as they are in the US, and due to current events many Europeans have been seeking alternatives in order to move away from services provided by the United States (like WhatsApp for instance).

[Airy]

Your regex is interesting (and can be fixed by first adding a space to the end of the text before you perform the regex) but has a problem... he said "paragraphs should be retained" and not split up. Your code splits paragraphs. He currently seems to be defining paragraphs as double carriage returns.

I think I have a solution for that, but I don't want to write any more code unless he answers some questions first.

[hemicyon]

@Airy

he said "paragraphs should be retained" and not split up.

Hm, I took retained to mean preserved, which is why I solved it like that.

@Airy

(and can be fixed by first adding a space to the end of the text before you perform the regex)

Yeah, I did figure that bit out; it's just niggling at me that there doesn't seem to be a neat and tidy expression to do the whole thing through the end of the text regardless of what's there (I tried to make use of \Z but was unsuccessful) — Although, this is still vastly better than the path I nearly fell down with an if-statement...I don't even know where I would've ended up.

[Nige_S]

@hemicyon

This expression [\s\S]{0,140}[^\S] works almost perfectly.

I think that the fix is:

(?s).{0,140}([^\S]|$)

...because you don't care what the intervening characters are until you're reaching the 140 character limit. The (?s) flag makes .-matches include new-lines, taking care of the paragraph problem. And the last bit lets us match "either a non-whitespace character or the end of the string".

I didn't think of using a "For Each... substring match" to do this -- nice one!

[hemicyon]

@Nige_S

I think that the fix is:

(?s).{0,140}([^\S]|$)

Ah-ha, the missing link! The key to the fix is actually the end grouping because (?s).{0,140}([^\S]|$) and [\s\S]{0,140}([^\S]|$) give the same results, which is so frustrating knowing how close I was and not getting there because I would've sworn I tried every combination of [^\S] and $ back there. I'd make one small change though: I'd make it a non-capturing group because — even though it doesn't make a single bit of difference in this case in KM — my brain craves order.

@Nige_S

I didn't think of using a "For Each... substring match" to do this -- nice one!

High praise indeed! I find that I naturally gravitate towards regex to solve very many problems. It isn't always the best solution in every case, but they're the ones I have the most fun doing.

[truecolor]

My original requirement for ChatGPT:

There is a text in the clipboard in macOS.

Lorem Ipsum serves as sample text. (attached below)

Permitted software: Keyboard Maestro
Permitted tools: regular Expressions (RegEx ... is already implemented in Keyboard Maestro)
Forbitten tools: Scripts e.g. Python, Bash
Forbitten: separation within words.

A macro is to be created with Keyboard Maestro that performs the following task:

Why the effort: the global SMS service only allows a maximum of 140 characters per message due to uniform standards.
Keyboard Maestro should therefore divide a long text into 140 character blocks so that the individual text blocks can be sent one after the other via SMS without any problems.

Using RegEx, it should be possible for KM to stay below 140 characters and to separate the text at the last possible space within the 140 characters. If the space is found, the text is copied from the first character up to this space and pasted into a separate variable. This text section is given the heading Block01. The variable is filled with further text blocks in the course of the runs. KM remembers the space from the first pass and starts counting 140 lines again from this space. RexEx checks whether there is a space at this position. If not, RegEx searches backwards again until it finds the next space. If the space from the second pass is found, the text between the first and second space is copied and pasted into the separate variable. The second section is now given the heading Block02

This continues until KM determines that the number of characters from the previous space to the end of the original text is <140 characters. The last part is entered into the separate variable at the end.

At the end, the separate variable is copied to the clipboard with all the text blocks it contains.

//—————

Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet. Lorem ipsum dolor sit amet, consetetur sadipscing elitr, sed diam nonumy eirmod tempor invidunt ut labore et dolore magna aliquyam erat, sed diam voluptua. At vero eos et accusam et justo duo dolores et ea rebum. Stet clita kasd gubergren, no sea takimata sanctus est Lorem ipsum dolor sit amet.

Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit augue duis dolore te feugait nulla facilisi. Lorem ipsum dolor sit amet, consectetuer adipiscing elit, sed diam nonummy nibh euismod tincidunt ut laoreet dolore magna aliquam erat volutpat.

Ut wisi enim ad minim veniam, quis nostrud exerci tation ullamcorper suscipit lobortis nisl ut aliquip ex ea commodo consequat. Duis autem vel eum iriure dolor in hendrerit in vulputate velit esse molestie consequat, vel illum dolore eu feugiat nulla facilisis at vero eros et accumsan et iusto odio dignissim qui blandit praesent luptatum zzril delenit augue duis dolore te feugait nulla facilisi.

Nam liber tempor cum soluta nobis eleifend option congue nihil imperdiet doming id quod mazim placerat facer possim assum.

Lorem
—————//

[Airy]

Yesterday, you said you would test our solutions. Did you test them? Did they work?

[Nige_S]

@truecolor

sent one after the other via SMS without any problems

Sent how? Are you just wanting something you can paste into eg TextEdit from which you can work through the blocks manually? That would be a good first step anyway, allowing you to check things are working the way you want before going full automation.

@truecolor

This text section is given the heading Block01.

"Heading" means different things in different contexts -- you need to be more explicit. But it would be a good idea to make your "headings" such that they are very unlikely to appear in the "normal" text, eg ===Block1===.

Line separation -- i.e.

===Block1===
Some text here
===Block2===
Some more text

vs

===Block1===Some text here===Block2===Some more text

...will make manual use of the blocks easier, but could make later automation harder.

Is the numbering a requirement, for example to allow recombination of messages that may be received out of order? Does that information need to be included in the SMS and therefore accounted for in the 140 character limit? Is number padding required and, if so, what's the maximum expected number of blocks (9, 99, 999,...)?

When splitting the text, each block (except the last) will have a trailing white-space character. Do you want to keep that so you are able to accurately recombine the blocks, or trim it off?