New user, request for help with text manipulation in Bruji's BookPedia

I've decided I really want to get to know KM (I have 9 on Mojave).

Two questions, please, if I may:

1 - I have about 8,000 records (Titles) in Bruji's BookPedia.

I want to split many of them on a colon which may or may not be surrounded by one or more space(s).

So a Title like this:

Bitter: A Taste of the World's Most Dangerous Flavor, with Recipes

leaves

Bitter’ in the main/first Title field

and

A Taste of the World's Most Dangerous Flavor, with Recipes

gets pasted into the ‘Original Title' second field by this Macro after removing the first two characters (the colon and the space). That second field is the next in each record so can be reached by Tab.

Secondly, are these videos considered the best introductions to KeyBoard Maestro, please? If not, what's good?

TIA to any kind user who can help :slight_smile: .

Bookpedia looks like an app. It appears that you are trying to edit fields in an app, not text in a text file.

So what you seem to be asking for is a KM macro that will go through all 8000 titles, using the app, look for titles that require changes, and when changes are required, to make the change (and to ask the user to confirm the change?)

The price of Bookpedia is $26 on the App Store so I'm not planning to download it. I know there may be free versions from the internet but I only trust the App Store (I also trust KM, but I think I first bought it on the App Store. So it has earned my trust.) But I still might be able to help you.

My main question is can you find one example of a title that requires changes, and carefully record what you click on and what you type? If you can do it with your own eyes and fingers, then i can help you make a macro.

Since you plan to do this only once, I infer, then the solution can be a little sloppy. For example, we can hardcode some mouse positions.

I think I can see the "Title" field but not the "Original Title" field in any of the screenshots of the app online. So you might need to insert some screenshots here if you want some help.

May I ask why you are doing this? I'd guess that only 1% of regular book titles have colons in them. And of those only 25% should be broken up into separate titles. Out of 8000 titles that, hypothetically, would make only 20 that need to be split up. Personally it would take me less time to fix 20 titles manually than it has taken me to write this message. To me the hard part of the task that requires automation is flipping through the 8000 titles to find the ones with colons in them. But I can see a "Search" field in the Bookpedia app. Have you tried entering ":" in that field to see if it can find the books with a colon in the title? That's 90% of the work right there.

Sorry for being argumentative.

1 Like

Thank you very much for your response, Sleepy!

Not argumentative in the least; my apols for not being 100% clear.

Yes, BookPedia is an app; Yes, there is (plain unformatted ASCII) text in the fields.

Because only about a quarter (2,000) of the records (= books (' titles)) have colons, I am selecting those which I wish to change manually… not asking KM to do that.

Nor do I need confirmation.

And of course, you're right - this is a one-off-then-discard Macro. Just that I have arthritis and doing the following (screen grabs attached) 2,000 times manually is pretty painful!

I have found that by and large BookPedia's Edit windows - and hence these fields - are in the same position on screen, so cursor location ought to work, though if there is an alternative, that would be better :slight_smile: .

This is the procedure:

  1. select all the text from before the colon to the end of the Title (its descriptor isn't shown - but it's the first larger-font, bold-font field at the top)… : A Taste of the World's Most Dangerous Flavor, with Recipes
  2. paste it all into the 'Original Title' field (which is always empty)
  3. move the cursor to the start of that string (now in the 'Original Title field')
  4. and (forward-delete) the colon and the space*
  5. save the amended record by single-clicking (or pressing the key) Return

So the Main Title field is left with - in this instance - Bitter.
And the Original Title field (which was blank) becomes - for instance - A Taste of the World's Most Dangerous Flavor, with Recipes with any leading spaces and the colon removed.

That would be a good start… once I understand how such a Macro works, I'd try and add a command to move to the next record in the subset (a Smart Collection in BookPedia's terminology) of all the titles which contain a colon.

*The colon may or may not have one or more spaces before and/or after it.

Any help you are kind enough to offer very much appreciated!

That's excellent information, your procedure. But I would change your procedure a bit. I think my method will be simpler. But it depends a little on how your app works.

  1. Triple click on the location where the title is (in many apps, triple clicking is "select all" for that field. Not sure if your app works that way, but check it out.)
  2. Copy the selected text to the clipboard.
  3. Using a KM action, split the clipboard into two strings on either side of the colon, String1 and String2 (apologies to Dr Seuss.) You may need some advice on how to do this step.
  4. Insert String1 back to the app (while the app has the title selected, this will replace it)
  5. Send the Return key to the app (probably not required)
  6. Triple click on the Original Title text field.
  7. Insert String2 back to the app.
  8. Press Return.

This is easier than your approach because your approach has a very tricky requirement regarding clicking the mouse near the colon. Your approach isn't impossible but from my experience your approach is more tricky.

Try testing my triple click shortcut. If it works, and you like my approach, I think we can code my approach fairly easily.

I'm heading out for lunch now. They are changing the menu at my favourite restaurant tomorrow and I've got a couple of items left to try on their menu that are disappearing (after?) tomorrow.

1 Like

The triple click idea for selecting the title text is good, but this also seems like a great chance to take advantage of KM9's new OCR capabilities. Doing so would also let the insertion cursor remain in the Original Title field, ready to paste the modified title text after OCRing it to the clipboard. The exact area coordinates to use for the OCR action will vary depending on your system and where BookPedia is placed on the screen, so you'll need to tweak those (I recommend taking advantage of KM's mouse display feature to facilitate this) but this proof of concept macro should hopefully get you started:

BookPedia - Split Title at Colon.kmmacros (3.3 KB)
image

While the OCR action can also be applied to the entire BookPedia window, in this case I think you'll want to limit it to the specific area on your screen where the title text will be displayed.

As for this question:

I haven't looked at this resource myself, and it doesn't include information on all the new features and changes added in KM9, but a good other number of forum users have had positive things to say about it, so I imagine it should indeed be a great resource for getting started.

1 Like

Sleepy,

(I can see very clearly that I've joined yet another extremely friendly and helpful Mac community. Thanks!)

How was lunch? Enjoy those items for the last time?

I appreciate the time and trouble you're taking to help me here. Your suggested procedure is almost perfect, I think. I tested it with a manual sequence. Triple click works just fine.

Except:

step 3: There are really three strings: the substring which will get pasted back into the Title field at step 1; the substring which will get pasted into the Original Title field at step 6. And thirdly the need to isolate and discard/ignore one or more spaces either side of the colon.

So I'm thinking there will be some extra regex work after your step 3; and, Yes, my regex are a bit rusty… I'll give it a try with the help of this site. But I think I'd be grateful for some help, please :slight_smile:

It's also true to say that there are some long existing Title strings with two sets of colons. That's no problem, because I can do those manually - since I'm working on just one record at a time, and always confirming everything visually.

What's more your step 6 may not be completely necessary… Tab will always move from the Title (1st) to the Original Title (2nd) field :slight_smile:

I agree, I'm not sure that step 5 will be required. Can test :).

Thanks again!

Thanks for your help. I always appreciate when others chime in.

I've used OCR every day for nearly 10 years in my macros. I use the same underlying engine as the one used in KM9. But my engine is about 10 years old now, the latest engine used in KM9 is probably more reliable so I'm happy it's here. But for a case like this I would prefer a 100% accurate method.

Your regex expression looks good. So I may not have to reinvent that part, thanks. And despite the fact that it's better to use KM actions to split text, I tend to find myself piping text into a KM action to allow shell commands to do the work. I know that's not efficient, but it's usually easier for me as I'm more familiar with shell text processing methods. Since this is a one-time job macro, efficiency isn't crucial.

1 Like

Gabe,

So kind of you too to take so much trouble. What a forum. Thanks :slight_smile:!

I'm still learning KM, and so haven't used the OCR functionality at all - YET.

In addition to Sleepy's help, I'll gratefully try the Macro you wrote…!

Your regex in the second step does check for those possible spaces etc, doesn't it?

As I look through what you wrote, I can see I have a lot to learn. But the pay-off will be huge :slight_smile:.

So shall gladly try those videos.

Thanks again!

1 Like

I wouldn't call that third step a step at all. It's not required because steps one and two essentially perform that step. I know I glossed over that in my procedure. I guess I was thinking about getting lunch so I glossed over it. Or maybe that was too much detail for pseudocode.

You said my step 6 wasn't necessary. Well, yes, if you use Tab instead. The step is still necessary, you just found a different way to skin the same cat. I was thinking about tab but I didn't know how your app worked, so I figured if triple click worked once, it would definitely work twice.

It seems you are in control right now. The only thing you need is a way to split the string into two strings. GG provided one way to do that in his post. I was thinking along his lines, but regex is very flexible and I was thinking of a different way to skin that cat. But if his code works, just use it.

Apologies to Dr Seuss again for talking about skinning a cat in a hat.

Thanks for the nice things you said about me. Encouragement helps.

My lunch was fabulous. Country fried chicken with a large helping of both mashed potatoes, vegetables, and gravy. And cornbread. Today is the last today for some delicious entrees. But tomorrow they are adding some items to the menu, like brisket. To be honest I don't even know what brisket is.

1 Like

Sleepy,

Thank you again. I'll see what progress I make in building a Macro and using GG's really helpful example.

I expect I'll have to make a few detours to become really familiar with the KM basics.

But - I wonder whether you realize it - you've both done something extremely helpful that you probably take for granted: confirmed that the best way to stroke this cat (I love animals so would never dream of skinning them :wink: ) is by clicking into fields and not trying to name or script them. That gets me a long way very quickly. Thanks.

Maybe you need to talk to your restaurant manager about why they choose Tuesday to change their menus.

Encouragement? You bet!

Back soon after some investigating and trial and error.

Thanks both!

I think we've steered you in the right direction. As long as your boat doesn't have a hole in it you should make it to your destination now. That tip about clicking to activate a field seemed to be really helpful to you. If there's a hole in your boat we can patch it up. But you've got the rudder right now and the wind is at your back. Bon voyage.

1 Like

It certainly was: I was imagining all sorts of other ways to automate selection of fields in that app. But the simplest one - using pointer and caret, mouse and screen location means I can eliminate those.

Provided - as GG says - I can consistently assume (and/or if necessary account for) the position of the window etc.

Half the time my code requires windows to be in certain positions on the screen. What I tend to do is move them into the upper left corner at the start of my macro, sometimes using this code. This way my hardcoded mouse positions will generally not fail because the window will be in a known fixed location. You should consider this too.

2 Likes

I should; and I will. Thanks.

From a quick experiment, BookPedia seems to keep its edit window always in the same place. But to be double-sure is a good step.

As I said, a novice user probably imagines all sorts of ways to work specifically in a field. Conventional scripting (e.g. locating and changing the contents of a field on a webpage in JavaScript) certainly tends towards naming the fields. To know that KM takes a different approach by choice has saved me hours of waiting in blind alleys :slight_smile: .

Here a more traditional (but presumably faster) approach with UI scripting (since Bookpedia doesn’t provide any AppleScript dictionary):

Script:

set {saveTID, AppleScript's text item delimiters} to {AppleScript's text item delimiters, {":"}}

# Max. number of records to be processed with one run
set maxEntries to 3

tell application "System Events"
  repeat with i from 1 to maxEntries
    tell application process "Bookpedia"
      tell window "Bookpedia"
        tell splitter group 1
          tell splitter group 2
            tell group 1
              tell scroll area 1
                tell table 1
                  tell row 1
                    tell text field 1
                      try
                        set oldTitle to its value
                        set newTitle to text item 1 of oldTitle
                        set newTitle to my trimLeadingSpaces(newTitle)
                        set originalTitle to text items 2 through -1 of oldTitle as text
                        set originalTitle to my trimLeadingSpaces(originalTitle)
                      on error
                        set originalTitle to ""
                      end try
                      perform action "AXShowMenu"
                      delay 0.1
                    end tell
                  end tell
                  click menu item "Edit…" of menu 1
                end tell
              end tell
            end tell
          end tell
        end tell
      end tell
      tell window oldTitle
        tell group 1
          tell scroll area 1
            delay 0.1
            set value of text field 1 to newTitle
            if originalTitle is not "" then set value of text field 3 to originalTitle
          end tell
        end tell
        tell button "OK" to perform action "AXPress"
      end tell
    end tell
  end repeat
end tell

set AppleScript's text item delimiters to saveTID

on trimLeadingSpaces(str)
  repeat with i from 1 to count of str
    if str begins with space then
      set str to text 2 through -1 of str
    else
      exit repeat
    end if
  end repeat
  return str
end trimLeadingSpaces

Macro with the script:

[temporarily removed as we are debugging]

With this example dataset:

…it returns this…


It is required that in the editor window the “Original Title” is the third text field, like this:

48-pty-fs8

More Notes:

  • In the current test form the script only runs on the first three entries. You can change this in the AppleScript (“maxEntries” variable.)

  • Currently the script only removes leading spaces for each of the re-defined fields. (Trailing spaces usually do no harm, but this can be implemented too.)

  • When you test the script, make sure the Bookpedia window is open.

  • The screen position of the windows doesn’t matter at all.

  • If you get an error like “Can’t get menu item…” try to uncomment the delay 0.2 (It’s the only commented line in the script.)

  • The script is meant as a test: if it basically works for you (on a couple of entries), then we can still refine it. (I have no clue what a window with 8000 records looks like, or if your database is somehow setup differently. I just downloaded the app and used the default setup.)


Edit 2019-08-20:

Updated script and macro, see also here.

Edit 2019-08-21T00:17+02:

  • Updated script
  • Removed macro download link (for the time of the debugging stage)

Edit 2019-08-21T02:43+02:

Updated script:

  • Colon-separated text items after the first colon go all into Original Title (leaving the colons in that part untouched)
  • Added some delays
  • No longer overwrites Original Title if we don’t have a value for it. (I know, you were OK with overwriting, but it is better this way, in case you are accidentally processing records you already had processed before. No data loss.)
2 Likes

I count three different solutions from three different people in this thread that have been given to you. You have the rudder. Pick your preferred path. But we're here to help if you get stuck. Each path has its own advantages.

One of my hobbies is to take intransigent macOS apps (or websites) and automate them even if they weren't designed to be automated... even if they are designed with self-defence mechanisms to prevent automation. So I have a few tricks up my sleeves. Bookpedia is a cakewalk compared to the apps/sites I automate, but I won't tell you what apps those are. I help people with their problems on this website when my problems are too difficult, or when I have a toothache, like now. :tooth: :sob: Yes, I'm getting that addressed. Another hobby is to build robots that use cameras to move. That's also difficult. I won't say much about what those robots are doing.

I actually have a fourth solution that might work, but it requires that your app be able to back up its database into a text file. If so, there's a much simpler solution than any of our solutions here. But I have no idea if your app has the ability to back up its book database into a text file. In fact it may already be stored in a text file, or multiple files. If this is actually the case, I could solve the problem in a single KM action, and a fairly simple one at that.

1 Like

Come on :thinking: :wink:

1 Like

Indeed, manipulating for example a CSV file would be the cleanest (and fastest) way.

Edit: …and the app can do it. Just checked.

1 Like

I'm willing to tell you that the robots are made out of a bunch of $9 servos which are controlled by a Phidgets servo controller which KM sends signals to based on the images coming from a camera. The servos and the controller look like these images I just got from the web. Physically the parts look like this image I found online:

2010_1 - PhidgetAdvancedServo Kit 4-Motor

So you know based on the size of these parts that I'm not exactly building a Terminator robot to send back in time to take Peter out before he builds Skynet using Keyboard Maestro.

Does that sate your hunger for information?

Hehe, haha! So we went through all that work when a single KM action with a single sed statement could probably do everything. Nice!

No regrets. I'm sure that 30% of the solutions we provide to people aren't the best solution simply because we don't have all the information.

1 Like