How to extract all the data from a csv file

Hi -

I would be grateful for some newbie assistance -

I have data in a clipboard, sets of numbers, that I wish to extract into KM to manipulate. But I don’t know how to do it?

The format is as follows
12.345,34.567,56.789,79.901,
23.45,56.7,31.234,79,234

So with the above sample we have two lines with 4 numbers on each line each separated with a ‘comma’. (I could change this to alternative delimiters if that would help?) And each line is also separated with a ‘comma’ (I could change this delimiter if it helps?) In a real world example, I can imagine a maximum of say 600-1000 lines x 4 numbers.

( Please note that I have created the layout so I can change it if required… eg change the line delimiter?)

So where I need help is how to extract the data… I feel it likely that ‘get substring’ is probably the way to go? for me to basically step through the four commas…
Find the first one, step back 6 steps* and then grab the next six numbers and file them away… - do a loop of 4 to get the Numbers from the first line… then repeat on the next line…
I don’t understand how to use the various options in ‘get substring’ there to enable me to do that…

I hope the above makes sense...

Assistance gratefully received. Thanks v much

Mik

*As some of the numbers are less than 6 digits inc decimal point, I may need to initially pad any of these shorter numbers.

That format is perfect, and easy to extract. Just use a Calculation and array indexing. The line feed is ignored as white space, so the comma at the end of the line is needed.

image

Note that I “corrected” your input data from “79,234” to “79.234” as you said two lines of four numbers and it looks like that was a mistake, but it would work just as well with 9 numbers or whatever. As long as they are all numbers and as long as they are all separated by commas, the array indexing in a calculation should be fine.

1 Like

Hi Peter
Thanks v much.
I will try it out later...
I can confirm that the data are all numbers, all with a decimal point.
Question-1: Can you put things into perspective for me... what sort of quantities of 'plain ascii data', can this type of storage, an array, cope with? As mentioned in my initial posting, would say 1,000 lines x 4 numbers be stretching things?
Question-2: Taking things further, could I legitimately have multiple sets of data, in multiple arrays, say 20 sets, all of similar size. And I switch to them to access alternative sets of data...?
Question-3: If yes, am I better off to have all the data in one array? or better to use multiple smaller arrays?
I hope this makes sense?
Thanks as always
Mik

Should be fine. Computers are pretty fast. How much performance do you really need? The only wat to answer this is to try it and see.

I just had a quick test, and with 10,000 numbers it was not apparently a problem (on my very fast Mac), but 100,000 seemed like it was getting a bit slow.

I presume so. You can store the data in files, and read it in to a variable when you want to use it. Make sure you use a Local or Instance variable so it's not stored, or clear it when it's done, to avoid wasting space.

It really depends on what you are trying to do.

But I would suggest you do whatever is easiest first, and see if that works for you without any undo performance issues. Only worry about performance issues when they become apparent, otherwise you're just wasting your time instead of a little CPU.

If scale, datatypes, or available functions ever proved an issue, you could also try this kind of thing:

1 Like

Thanks as always, Peter

I will go have a look - Thanks very much