I have a macro that generates random IDs to a csv file up to a specified file size. But it is really slow when the file size is set to tens or hundreds of megabytes.
I would like to do this in a faster way, maybe using the Execute Shell Script only. I would like to prompt the user for 2 inputs:
Number of digits for the ID.
File size of the CSV.
Then just execute the shell script using the inputs. I think that would be much faster.
For every ID you generate you're instantiating a shell and then opening, writing, and closing a file in the "Append" action, and finally another system call to get the file size. That really slows things down.
Is there a reason to be doing x megabytes rather than y IDs? If you do need a size, could you use
desiredSizeInBytes / (VID_length + 1)
...to get the number of IDs to generate and always be a little under the size, or add 1 to always be a bit over?
I won't pretend to understand the perl, but I think you are just writing one ID per line and nothing else in your "CSV". Is that right?
@ChrisQ -- this will let you choose an output file and then set your VID length and required file size in MBs. You'll see that, like your script, I've used "new-fangled" MBs. Change the fourth action's calculation from using 1000000 to 1,048,576 if you prefer "proper" MBs
As written this will do slightly less than the required file size -- if you'd prefer slightly more than change the fourth action's FLOOR to CEIL so that the number of VIDs generated rounds up instead of down.
I've a feeling this will be "quite a bit" faster than your original -- would love to see some numbers on that!
@Nige_S, Wow! Your macro is nice and fast. Thank you! It is exactly what I wanted. Your macro finished creating IDs for 10MB csv file in 10 seconds. Compared to my macro which I stopped after it ran for 1 minute and only generated 35KB csv file.
@ccstone, I have not tried out your method yet. I will need to learn more shell scripting before I attempt this.
,