Generate Random IDs to a CSV File Up to a Specific File Size


I have a macro that generates random IDs to a csv file up to a specified file size. But it is really slow when the file size is set to tens or hundreds of megabytes.

I would like to do this in a faster way, maybe using the Execute Shell Script only. I would like to prompt the user for 2 inputs:

  • Number of digits for the ID.
  • File size of the CSV.

Then just execute the shell script using the inputs. I think that would be much faster.

Random_VIDs.csv.kmmacros (5.1 KB)

Macro Image

Any help is much appreciated, TIA.

For every ID you generate you're instantiating a shell and then opening, writing, and closing a file in the "Append" action, and finally another system call to get the file size. That really slows things down.

Is there a reason to be doing x megabytes rather than y IDs? If you do need a size, could you use

desiredSizeInBytes / (VID_length + 1) get the number of IDs to generate and always be a little under the size, or add 1 to always be a bit over?

I won't pretend to understand the perl, but I think you are just writing one ID per line and nothing else in your "CSV". Is that right?

If you want speed then do all the heavy lifting in Perl.

Here's how to get Keyboard Maestro variables into your shell script:

Keyboard Maestro Variables to Perl v1.00.kmmacros (6.5 KB)

Macro Image

Keyboard Maestro Export

1 Like

Nah -- let's go Route 1...

Even on my crusty old iMac this'll churn out 526k VIDs, a 10MB file, in ~20seconds:

Generate VIDs.kmmacros (4.2 KB)


@ChrisQ -- this will let you choose an output file and then set your VID length and required file size in MBs. You'll see that, like your script, I've used "new-fangled" MBs. Change the fourth action's calculation from using 1000000 to 1,048,576 if you prefer "proper" MBs :wink:

As written this will do slightly less than the required file size -- if you'd prefer slightly more than change the fourth action's FLOOR to CEIL so that the number of VIDs generated rounds up instead of down.

I've a feeling this will be "quite a bit" faster than your original -- would love to see some numbers on that!


@Nige_S, Wow! Your macro is nice and fast. Thank you! It is exactly what I wanted. Your macro finished creating IDs for 10MB csv file in 10 seconds. Compared to my macro which I stopped after it ran for 1 minute and only generated 35KB csv file.

@ccstone, I have not tried out your method yet. I will need to learn more shell scripting before I attempt this.

Thanks again :slight_smile: ,