An Automatic URL Decrufter for Defined Hosts

Downloaded and expanded just fine here on two Macs.

I don't think there are actually any upgrade instructions in the macro. Basically, you'd need to copy any custom domains from macros [12] and [14] in the old version, as well as any custom filters (in the [20s] section) that you had created and paste them in the new version. It should then "just work."

If you haven't added any custom stuff, then there's no work required. Someday, I should rewrite the thing to use a persistent file on the disk instead of variables in RAM. Someday.

-rob.

2 Likes

This is more of a Keyboard Maestro question than a "Decrufter" question, but is there some way to resize the full text fields of the Comment macros in the "00] The Decrufter" macro? I thought there was a way, but I can't find/remember it. Not being able to do that is really making customizing more challenging. Those steps for stubborn URLs are tucked away pretty good in there.

Sorry, still stumped with 8.3.
I want to fix some crufty Youtube links I get sometimes.
Here's the regex (looks right in simple regex test):
^[^&]*

Domain is youtube. I need the cruft including and after the "&" removed, not the "?".

Here's a close but phony sample:
https://www.youtube.com/watch?v=i9AoBLgKyns&list=PLi4l3wxwkqyy9EDOH4VzhhEL-mBFIoaa8&index=12

Can you give it a try? Thanks.

Not that I'm aware of. You can use the gear icon to copy the contents of the field, but I'm not aware of any way to expand it.

-rob.

1 Like

Remember you need a capture group in your regex to capture to the d_trimmedurl variable. Here's the Facebook version:

So it should work fine if you just add the capture group:

^([^&])*

-rob.

1 Like

The final URL becomes

https://www.youtube.com/watch.

I think I'm stumped. If I delete the entire 8.3 macro group, then add it again from the latest download, copying this URL:

https://www.youtube.com/watch?v=i9AoBLgKyns&list=PLi4l3wxwkqyy9EDOH4VzhhEL-mBFIoaa8&index=12

results in https://www.youtube.com/watch

before I make any changes, and nothing I try that allows any decrufting operation results in anything else for the final URL. It shouldn't even try to decruft a youtube URL before it's specifically added, should it? (It doesn't appear in the list.) It seems to me that something would have to be disabled for the URL in my example before it can be added with the correct treatment. Is that incorrect? Does some kind of cache need to be cleared so Decrufter won't attempt to decruft a Youtube URL before the domain is added to the list?

Maybe I just need instructions on how to force First Run again. When I looked at the value of the d_nonconcurHosts_added, what appeared was "youtubeyoutubeyoutubeyoutubeyoutube".
When I cleared that from variable contents field the

https://www.youtube.com/watch?v=i9AoBLgKyns&list=PLi4l3wxwkqyy9EDOH4VzhhEL-mBFIoaa8&index=12

URL was decrufted as https://www.monoprice.com/product?p_id=36045. That's something that I can't begin to explain. (That's still before I attempted to make any changes after deleting and reinstalling again.)

Sorry, I wasn't clear: YouTube isn't set up to be decrufted, because normal URLs to/from there don't have any cruft in them. You'd need to set it up as a custom host in macro 12 (list youtube in the box), and add it to macro 14, as it doesn't need to be fed through curl. Be sure to enable the steps where you list youtube.

You'd then duplicate one of the 24] macros, edit to to be named YouTube, and put your regex in the regex box.

Finally, in macro 22, you'd need to add your own host rules in the green box. You'd be checking for d_extractHost is youtube, and in the execute step, you'd put in the execute macro step, pointing at the 24] macro you earlier created.

-rob.

1 Like

Thanks very much, and I really hope I didn't sound huffy. :smile:

Nope, not at all. It's just that what's there is there because they're URLs I often run into; I'd never seen any YouTube things that needed fixing, so didn't try to include it.

-rob.

1 Like

Thanks again.

Well, I went through those steps, but my example crufty youtube URL is still decrufted to https://www.youtube.com/watch. But now I'm sure something is wrong, because even though I enabled the macro and customized it to establish the d_linkHosts_added variable for "youtube", no such variable existed after I ran Decrufter by copying the crufty Youtube URL to the clipboard. Neither did any variable named d_linkHosts exist until I used "try" on that macro in macro 12.

I think I'll need to create a new user in order to use Keyboard Maestro in an environment with no variables, then install Decrufter to get a list of variables created before and after I configure it to handle the crufty Youtube URLs. That's the only course of action I can think of that will allow me to troubleshoot further.

After a flurry of farreaching attempts at troubleshooting, I discovered that my existing configuration of Decrufter which had been working well for me with the youtube URLs stopped working, too. However, I exported the previously existing configuration while booted from a backup (one example of how a bootable backup can come in handy, glad it still works as of 13.3.1), so having revived that one, I'll probably give it a rest until I can devote proper attention to it. Thanks again.

I got it working. The regex you were using wasn't actually workable (I only eyeballed it the first time through). This version works for your URL example:

(.*?)&.*

But then, if you copied the Facebook routine as I suggested, I found that there was a typo in its variable name (trimmmed vs trimmed), which meant you'd have the same typo in the YouTube routine. And that typo meant that the already-trimmed URL was then passed to the generic trimmer (because it didn't think the final variable had been set), and that generic trimmer crops everything after the "?". Once I fixed the typo, it worked as expected.

I'll just go ahead an include this basic YouTube version in the next release, which I'll have out sometime today. (Only changes are the YouTube addition and the Facebook typo fix.)

-rob.

1 Like

Thanks again, Rob. You're the best!

OK, one more thing (again). Have you ever seen XCRegex? It's in the MAS. It tells me that

^[^&]*

yields the match I want, while the one you provided

(.*?)&.*

matches the entire string of the typical youtube URL that I routinely decruft. What do you think?

Anyway, thanks again!

I don't use any apps for regex, I just use various sites to test my strings. Here's your example string with my regex code:

And here's what it shows for yours, with the added group capture as required by The Decrufter:

It seems yours doesn't capture what we need to capture ... and I am nowhere near a regex wizard, so I won't even hazard a guess as to why. Typically, I go to regex101.com, enter some example strings in the input area, then build the regex that gets what I want as shown in the substitution area.

-rob.

1 Like

Very helpful, as always.

I want to identify the problem if possible, because here's what I'm seeing for the regex snippet originally submitted here by me at regex101.com:

Could it be Safari's text encoding? Something else?

Thanks!

Oops, I think the group capture is what I'm missing. Sorry.

Yep, the group capture is the key bit—that's what gets the URL that the macro then processes.

-rob.

1 Like

Thanks. I finally got 8.3 to be reliable performing the decrufting the way I wanted for all the URLs that I had hoped to decruft, including those unusual ones for Youtube. There were still some hurdles. Most of my issues appeared to be related to the "cruft" in KM caused by unneeded and disruptive remnants of older versions, a caveat that I've previously encountered when testing different versions of complex macros. I was able to eliminate the problems by creating a new user in System Settings>Users & Groups, and customizing the Decrufter while logged in as that user, then noting which variables were persistent so that I could eliminate them from my permanent configuration.

The adverse effects of these variables, notably the misspelled Facebook variable that you mentioned, were quite significant. But the regex for Youtube decrufting seemed more forgiving than expected, and the working macro contains the following (which was based on the same regex I had been using before 8.3):

image

I discovered that either my original regex or the one you provided (^([^&])*) appears to work successfully. It'll be interesting to find out whether that remains so. I have tested both forms on a couple of the crufty URLs. I'll install your next version of Decrufter whenever it's available . Thanks again!

-Lantro

Sorry that was so much hassle. To make such troubleshooting easier in the future, here's a tip: The only variables that The Decrufter creates that are "permanent" (though most are deleted at the end of the run) all start with d_. So an easy way to wipe everything clean is to just delete any d_ variables you see ... unless you have some other macro installed that happens to use the same prefix, of course.

As for the regex, I honestly have no idea why it works—and this morning, your version is working fine in regex101.com as well. Bizarre. When I build regex equations, I tend to go for the versions that seem simplest in my mind; I'd never have discovered your form in 1000 attempts :).

-rob.

1 Like