New user, request for help with text manipulation in Bruji's BookPedia

Tom,

I changed maxEntries to 1, which solves both problems vis-à-vis colons present but split not required, and two colons present, thanks.

But working on one record at a time starts the process off always from the first record in the database.

I can - and will - certainly experiment on a backup subset.

I know that you don't use BookPedia, so am doubly appreciative. One thing I have done is create a Smart Group consisting entirely of titles with one or more colon(s). Could your script be made to work on just that subset? There is also a View mode much more like a spreadsheet in columns where - again - splitting, cutting and pasting would be between fields by Tabbing.

In the case of single records, if it were possible to advance the split and paste one record at a time using the appropriate BookPedia Keystroke (not sure that that can be scripted), I suspect that'd also do it!

Yep. That’s why I proposed to order the rows in a way that is independent of recent changes (maybe creation date or somtehing).

But here come another Unknowns into play: how fast is the main window re-ordering the entries?

Try to add a pause on some locations. The AppleScript syntax for pause is: delay 1.0 (which means 1s). I would say try to add the delay just before the “repeat”, pretty much at the beginning.

1 Like

No, this app is not scriptable. That’s why I proposed UI scripting. I would be glad if it was scriptable :wink:

1 Like

Sure, you don’t want an one-in-all modification of your CSV dump?

You can always correct it afterwards? (And you have your backup anyway…)

PS:

I’m saying this because any GUI actions (via AppleScript or via KM) are unreliable. And, since the app already provides the possibility of a CSV dump, why not use it? The after-works are probably easier than doing the task at once, and one-by-one.

1 Like

Mark, could you possibly create a separate Book with, say only 100 records, from your master, that we could all have for testing? If so, maybe you could zip it and post somewhere like Dropbox that we could all access.

I've got a RegEx that should work:
(?mi)^(.+?):\h*(.+)

I've got an ASObjC RegEx handler that you can use with Tom's script.
Is everyone still working with Tom's original script posted here?

BTW, Mark, the general approach here is to put the post quote first, then add your reply comments.

1 Like

I think the main thingy here is: Should we work on the CSV dump… or work with a GUI based script.

The OP’s desire (“one record by one”) seems to point into the direction of a GUI script, but I’m not sure that the OP’s desire is the optimal one.

1 Like

Using CSV is preferable from several POV, but I thought @MarkSealey said he didn't know how to UPDATE the Book with the results. Is this correct?

1 Like

We still haven’t got any CSV dump yet, no? Unless I’m missing…

1 Like

What does this mean? The database is supposed to be up to date on saving time, no?

1 Like

Tom, I think this rules out using CSV, unless he wants to export the entire book, process the data, and create a new book from the corrected data.

@MarkSealey, what do you think?

1 Like

I'm calling it a day guys. If you haven't solved it by tomorrow, I'll be glad to help if I can.

1 Like

Yes, according to what he says, it rules it out. But what he’s saying is he wants to go thru each row (record) manually, because he doesn’t know if there are maybe two colons in the first row.

This complicates the things, obviously. But it is not undoable. And even if we can’t manage to solve it with two+ colons, he could always post-treat the results in relatively less time. Well, depends on the the amounts of titles with double colons in it.

1 Like

JMichaelTX,

Attached a zip of two dumps - txt and csv - from a much cut down BookPedia library with all (Smart) Collections removed - except the one that returns which records do have a colon in them somewhere.

BookPedia exports.zip (5.0 KB)

The ideal KM macro would be one which looks at a single record. That's because not all records with a colon in their title field need splitting… I want to retain control.

And one which allows me to start iterating through the 8,000 records at any point in the database - partly because I've already fixed a couple of hundred (such) records manually; and partly because I don't anticipate being able to fix all those records which need fixing at one go/in one sitting.

The interrelatedness of (Smart) collections and possible resultant losses of data integrity in BookPedia is another reason why I've been reluctant to pursue the approach whereby a script just works its way through either an export or the entire database.

IOW, exporting-fixing-importing would work very well for a single table; but re-importing would lose all sorts of data-relationships in BookPedia.

I really do need to go record by record - thereby updating all BookPedia's (Smart) Collections etc; and giving me the final say over which record is amended, and which not - really is the best way for me, I think.

I doubt that would be possible because of all the live-updated (Smart) Collections etc. I don't believe they're exported with the dumps we've been talking about.

I have almost 100 of these (Smart) Collections with carefully-defined criteria. It would take longer for me to record and recreate them all than to clean up on colon manually. But thanks for advising that there is so much merit to that approach; I can see your point!

I really appreciate all your help. I'm learning a lot. Bottom line: I can live with having to edit records with more than one colon. But I do believe it's vital to go record-by-record within BookPedia.

Thanks for your other guidance :slight_smile:

@MarkSealey, I have updated the macro and the script, please re-download it from here. (If you are still interested.)

I have also set the maxEntries to 1 so it now processes only one row at each run.

I’ve tested it with my mini dataset and it seems to run fine, if you order the rows by modification date, so that the oldest one is on top:

(Otherwise it would always process the same row.)

1 Like

Tom,

Thanks a million!

Still very much interested :slight_smile:

I've downloaded and tried your macro. For some reason it's not quite working as I think it should.

I believe it's failing in the loop:

on error
	set originalTitle to ""
end try

because the Original Title field is remaining blank.

Would you prefer to email direct, take this offline here as it probably isn't of great interest to anyone not using BookPedia? I email you or you me with email addresses?

One quick question, first, though, please: is your macro designed to work when the records are displayed one-by-one, in the single Edit window; or as in your screenshot (above) displayed in BookPedia's spreadsheet-like view?

If in the latter, I understand the importance of moving BookPedia's columns so that at the left come: Title, Date Modified (on which the whole thing must be sorted, oldest first) then Original Title.

But I wonder whether it's not working for me the way it is for you because I have a different number of columns further to the right and different actual custom columns?

Or because of some other way in which my fields are arranged, ordered, (named even?) differently from yours?

This shouldn’t be the reason. This is meant to avoid an undeclared variable in case the old title doesn’t have a colon (= no text item 2).

No. This can still be of use for somebody with a similar database GUI.

Starting point is the window as shown in the screenshot.

In the main window (is it called like that?) the order of the displayed columns shouldn’t matter (well, try to have the Title column at the left, since that’s how I tested it). What does matter are the positions of the Title and Original Title fields in the edit window, as shown in the last image here.


PS:

Another (unmentioned) assumption of my script is that the Original Title is empty for all records. (Otherwise it will just get overwritten.)

1 Like

I just imported your exports from above and noticed that the Title field remains empty for all records.

Is this normal, or what am I doing wrong? (In the CSV there seems to be a Title…)

1 Like

Tom,

(I do want to find a more concrete way to thank you for all this. I know it's the spirit of our community; but when we (= you) get it working, it will save me so much time - and I'll have learnt so much. And you've put in so much time too!)

I'm starting with the main BookPedia window as here:

Then triggering the macro (I put a KM Display Text action in to make sure it is running - although I do see BookPedia's Edit window briefly appear, so I am sure it is running, though somewhat slowly).

But I didn't have the fields in the same order as you did.

Now I do:

Yes, the Original Title is always empty. But if it weren't, overwriting would be what I want.

When I run the macro, the Original Title field still remains empty; I also checked - and the candidate string for that field isn't getting put into any other field.

I do notice - as will you - that the Date Modified field always contains the same date (e.g. November 5 2018): this is because of hours and hours I've spent tidying manually. I don't think it's really relevant since - as you explain - it's a way to prevent the records re-ordering themselves on processing complete.

Which does bring me to another question, please, Tom: is your macro designed to work on all records - or just those with a colon? I'm happy with the latter. It will be quicker, and I do still need not to split certain records even though they have a colon :slight_smile: .

Tom,

I've always been wary of working to and out of such exports. I don't think they're really designed to act as ways to transfer data between actual BookPedia files.

I don't know. Yes, I just looked again at the exports. They're as I would expect. With titles. It's partly why I wanted to do everything inside BooKpedia.

I agree, odd…

Well, forget about it. I just downloaded it to have a more “real world” dataset. But we should also get along without it.

1 Like