Replace NULL characters

I have this AppleScript that I downloaded a while ago that I used to backup all notes from the Notes all and it saved each note as html files.

Now I'm working on a macro that reads those html files and converts them to .md files to use with Obsidian. When I open those files I notice that when I hit the left/right arrows it always needs 2 hits to move the cursor. When I pasted into the website I see this:

After some research I got to this forum reply:

When I go back to regex101 and add \0 and use the Substitution option I can see that it indeed replaces them, but then when I use that in the Search and Replace action and Display the text, if I copy and paste into regex101, I can still see the NULL characters.


How can I get rid of them completely?
I'm almost sure that \0 is not the right thing to use, but I'm not familiar with this stuff so I don't know which one I should use or if there's any other way to achieve this...

One option is to run your variable through an Execute Shell Script using Perl:

perl -pe 's/\000/ /g'
1 Like

Thank you.
That got rid of the NULL characters indeed, but it just replaced them with spaces.
I have K e y b o a r d M a e s t r o
instead of Keyboard Maestro

This is how I used it, because it wasn't working with the default action settings (no input, and ignoring the results)

Here's the AppleScript I'm using to convert the notes in Notes app. Maybe you can spot something that can be changed in there originally that could fix the issue?
If not, is there a way to make that perl command remove the NULL, but take the space with it as well?

set rootFolder to "Macintosh HD:Users:dannywyatt:My Files:Inbox Global"

tell application "Finder"
	set exportFolder to make new folder at folder rootFolder with properties {name:"Convert to Obsidian"}
	set exportFolder to exportFolder as text
end tell

-- Simple text replacing
on replaceText(find, replace, subject)
	set prevTIDs to text item delimiters of AppleScript
	set text item delimiters of AppleScript to find
	set subject to text items of subject
	set text item delimiters of AppleScript to replace
	set subject to "" & subject
	set text item delimiters of AppleScript to prevTIDs
	return subject
end replaceText

-- Get an HTML file to save the note in.  We have to escape
-- the colons or AppleScript gets upset.
on noteNameToFilePath(noteName)
	global exportFolder
	set strLength to the length of noteName
	if strLength > 250 then
		set noteName to text 1 thru 250 of noteName
	end if
	set fileName to (exportFolder & replaceText(":", "_", noteName) & ".html")
	return fileName
end noteNameToFilePath

tell application "Notes"
	repeat with theNote in notes of folder "Notes"
		set oldDelimiters to AppleScript's text item delimiters
		set AppleScript's text item delimiters to "/"
		set AppleScript's text item delimiters to oldDelimiters
		set fileName to name of theNote as string
		set filepath to noteNameToFilePath(fileName) of me
		set noteFile to open for access filepath with write permission
		set theText to body of theNote as string
		write theText to noteFile as Unicode text
		close access noteFile
	end repeat
end tell

This script was downloaded from a website a while ago, I didn't make it. I'm still learning AS as I go, so I can't spot anything there that could be doing this to the text.


Update/Solution: I tried using the Search and Replace now with the option you used \000 and I was able to remove them without using the perl command.

I guess I stuck to using \0 due to my lack of understanding how these work and didn't try \000 even though I thought I did try \0 , \00 , \000 and \0000.
Maybe the issue is that sometimes KM doesn't run the macro in an updated state after we make changes, so maybe when I changed it to \000 it was actually using \00 or something like that. Go figure...

But yeah, it's now working!
Really appreciate your help on this. It was driving me crazy!

For reference, Keyboard Maestro uses these metacharacters in its regex: Regular Expression Metacharacters.

And as for the Perl solution, you merely have to take the replacement space out (// instead of / /), if all you want to do is delete the nulls:

perl -pe 's/\000//g'

Glad you worked it out!

Yes, probably an issue with text encoding. Notes does it differently to accommodate special characters.