New user, request for help with text manipulation in Bruji's BookPedia

I'm starting with the main BookPedia window as here:

Looks fine.

Now I do:

Seems fine, too.

When I run the macro, the Original Title field still remains empty; I also checked - and the candidate string for that field isn't getting put into any other field.

When you run the script/macro, does the edit window get opened at all?

I do notice - as will you - that the Date Modified field always contains the same date (e.g. November 5 2018): this is because of hours and hours I've spent tidying manually.

Although the app only displays the modification date/time with a resolution of days, it still is aware of the precise modification time.

I don't think it's really relevant since - as you explain - it's a way to prevent the records re-ordering themselves on processing complete.

The point is to make it reorder the records. So the most recently modified (= modified by the script) must go to the bottom. Otherwise we would run the script always on the same record. And the app does live-time reordering, at least here. (No need to exit the window or such.)

is your macro designed to work on all records - or just those with a colon?

It should be fine with either. See first paragraph here. If there isn’t any colon nothing will (should) be changed.

1 Like

What little experience I have of using SQL with BookPedia tells me that it is extremely well and robustly constructed (their developer is one of the very best).

Perhaps there are INSERTions into temporary/buffer tables.

Looking at the structure with something like Datum Lite might be revealing?

Yes, the Edit Window is opened and the menus called for the changes the macro makes.

It also puts the target record elsewhere - presumably to today's modification date.

But when I then look at it again - e.g. by searching for its title or author - the Original Title field is still blank.

Yes, the Edit Window is opened

and closed again?

and the menus called for the changes the macro makes.

What do you mean with this?

It also puts the target record elsewhere - presumably to today's modification date.

If the script has changed a record it should go to the bottom of the list, yes.

But when I then look at it again - e.g. by searching for its title or author -

You could probably also bring the last modified record to the top by – temporarily – changing the sort direction of the Modified column. (Click the column header.)

the Original Title field is still blank.

Even if there was a colon in the old title? And the Title field, was it changed by the script (if there was a colon)?

1 Like

When you run the AppleScript from Script Editor (/Applications/Utilities/), does it behave the same as when run via KM? (The script in its current form is completely stand-alone, i.e. you don’t need KM to run it.)


PS:

When running it from Script Editor, make sure the app has Accessibility permission:

47-pty-fs8

And also KM Engine should have that:

24-pty-fs8

1 Like

Tom,

Yes; it is very quick. But it does - I think - what it's supposed to do by opening the Edit window.

I see the menu appearing and the script making the same changes as I would do if I were editing manually. Quick, though.

Running it again, I see that it actually doesn't change the position of the record.

That's right, I'm afraid. The Main title string of records with one colon remain the way they were and nothing gets pasted into the 'Original Title' field :-(.

No.

Tom,

Running from the Script Editor (Accessibility permissions all set as they should be), I think reveals what's going on. I think:

  1. I have BookPedia open in the Columns view (as confirmed above)
  2. ordered/sorted by Date Modified
  3. I single-click to select the first record
  4. the standalone script, or the KM macro, executes - on that first record successfully; if the first record contains a colon in the title, the split and the strings are handled correctly

But when:

  1. I have BookPedia open in the Columns view (again as you confirmed above)
  2. ordered/sorted by Date Modified
  3. I single-click on - say - the third record to select it because it does contain a colon, it's the first visible record to contain a colon,

then:

  1. the standalone script still runs but it starts on the first record, and processes it correctly

IOW it always processes the first record.

Is that what you'd expect?

If it is, maybe that's OK because I can run the script/macro on the Smart Group I created where (virtually) all the records do need to be processed :slight_smile: .

Progress!

Running from the Script Editor

OK, since this works for you, let’s run it from Script Editor from now on. Just to exclude any additional complications by KM (not saying that KM tends to add complications :wink: , just to make it as simple as possible – during the debugging phase).

I single-click to select the first record

You don’t have to click any record. The script always works on the top-most. That’s why I emphasized that it’s important to sort by Modification Time, so that the processed record goes down. Sorry, if this wasn’t clear enough in my explanations.[1]

IOW it always processes the first record.

Is that what you'd expect?

Exactly. See above.

the standalone script still runs but it starts on the first record, and processes it correctly

Great :slight_smile:


1: Since you have stated that you only ever want to process one record at a time, I will remove the loop entirely from the script. This will make the script maybe a bit more transparent. I’ll upload it shortly.

Edit: However, I think, processing the records in junks of 5 or 10 wouldn’t be too bad either, no? Let me know what you think. That way you wouldn’t have to temporarily change the sort order for after each record for checking. (You could check the last bunch of 5 or 10.)

1 Like

Got it. Perfect. Yes!

You were clear. Thanks. Yes; if I execute the script on the Smart Collection populated only with those records which need splitting, it will be irrelevant.

One minor snag on my end is indeed to make sure that only those records which do need splitting are in that Smart Group. That means that the criterion: 'Contains :' won't work.

That was in the back of my mind when I tried to explain (sorry if I wasn't clear!) that I really wanted something that would run on records of my choosing.

I suspect that's not possible if this script only runs on the top record.

Will it also make it quicker?

Heresy to suggest it, Tom: but if I could make a BookPedia Smart Collection all of whose records absolutely did always need splitting, and when it's completely debugged, would it be possible after all to have it loop through all visible records (that is all of those in that (Smart) Collection)? I suspect it would be in the region on 2,000 - and I'd of course continue to work from a backup.

IOW the control which I want to keep - e.g. not to try and split Titles with two colons, nor to split where I didn't want to split would be up to me. I'd fill a (Smart) Collection with only records which needed splitting and we'd go back to the loop-through-them-all model?

I think in the time we have spent on the solution you would have already processed 1000 records by hand… :wink: (This was not meant to discourage you from searching for a scripted solution!)

1 Like

Yes, I like that idea - at the debugging stage.

But - once we know it's working 100% flawlessly - I am veering to the idea of looping globally through a (sub)set of all candidate records.

I'd make sure that they were all pre-screened (so to speak) by me first as being records which are to be split..

Not really - if for no other reason than that I keep working on a backup copy of the db. Not a single record has been changed in my 'production' database.

But I have learnt 1,000 things about KM :wink:

This time on Monday I only had the vaguest idea how to write a half-decent macro!

OK, I have updated the script, re-download it from above, please.

  • I have set the loop to 3. Feel free to change it 5 or 10 or whatever (or to 1). It’s the maxEntries variable at the beginning of the script.

  • The index of the processed row is no longer affected by the loop number, since we have agreed that we sort by Mod Time, so the most recently modified record goes out of the way automatically. (No need to itinerate through the rows.)

  • I’ve tested it with my 3-records-setup (colon, colon with spaces, no colon) and it still works fine here.

1 Like

Wonderful. Thanks so much, Tom!

It works perfectly.

So can you see any reason why I should not simply set maxEntries to the number which is the total of candidate records in a (Smart) Collection (currently 2,800!) and just let the script run, looping through every visible record?

Great. Worked perfectly.

Same here - although I'm not sure what would happen in the case of two (or more) colons, like:

1914-1918: The Great War: Recollections from the Front

I'm pretty sure that 90% of the time, splitting on the first colon (only) will do the job. Is that what it will do?

Even if it does something else, it would still be quicker for me to sort by Date Modified and then visually correct any which didn't look right than what I had been planning before coming to KM.

Thanks again, Tom; very close!

So can you see any reason why I should not simply set maxEntries to the number which is the total of candidate records in a (Smart) Collection (currently 2,800!) and just let the script run, looping through every visible record?

Don’t know. I remembered it was you who said that you wanted to process record by record. Probably I’m wrong.

But anyway I wouldn’t try to process “chunks” of 2800, because sometimes the script bugs out with no apparent reason. Run it again, and it’s fine. This can happen with Accessibility Scripting, it’s not proper AppleScripting (which the app doesn’t support; forward this to the dev). If it happens too often we can try to find the issue.

although I'm not sure what would happen in the case of two (or more) colons

The third part will be discarded. I knew you will ask for this. I will try to implement it properly, sorry. No matter how much colons, all after colon #1 should go to Original Title, right? Or a different solution? If all after colon #1 goes into Original Title, how should the parts (in Original Title) be separated? Keeping the colons there is OK?

1 Like

No. You were right. I did. But that was before I saw how reliably it could be made to do things, thanks :slight_smile:

That also helps: I can run it in chunks (set maxEntries to the next x number of records which I've decided all can be split! Will work very well.

I can also have two scripts side by side, one of them maxEntries=1 to iterate one record from the Smart Collection at a time.

Thanks. Yes: let's say that after the first colon everything else:

  1. retains the (second, third etc) colon, and
  2. all goes to 'Original Title'

Tom - I have an engagement tonight. Please take your time. Will look back here tomorrow. Very, very grateful!

3 posts were merged into an existing topic: Backups, iCloud, Language, & Other Stuff

OK, updated script here. See the last edit notes of that post.

Works fine here, still with my minimal 3-records setup, but with multiple colons in one of the records, and without any colons in another one.

If you have problems when processing more records in a row and your Mac is rather slow, try to increase the delays you find throughout the script from 0.1 to 0.2 or maybe even 0.3 (though 0.3 would be really much).

1 Like

Got it. Thanks, Tom.

I'll check it tomorrow morning. But I think it'll work just fine.

Very much appreciated…

Tom,

Still here: I'm currently going through all 2,000 + records in BookPedia which don't need the colon splitting. More than I thought or hoped.

But I successfully wrote KM macro which substitutes a string which won't exist anywhere else in the record (actually '777') for the colon-that-needs-to-be-retained-in-the-long-run.

That way I have isolated in a new Smart Collection all those records on which I'll reverse the substitution after I've run your script on the colon Smart Collection and put the colon back in!

That way I know I can safely run your script on the entire colon Smart Collection… that's some hours away :slight_smile: .

1 Like