Trouble with Perl Script requiring arguments

I have a perl script that I had someone create for me which converts csv files into a format that I can use. The perl script takes three arguments and I cannot for the life of me get KM to supply the appropriate arguments.

My KM macro will run this script for each file added to a specific folder. The perl scripts arguments are <template.csv> <datafile.csv> .

Two of the arguments are basically going to be static (template file will always be in same location and path of output file will always be same path).

I’ve tried passing $KM_VAR_File_Name as the datafile.csv Arg, but that doesn’t work. I’ve tried using the suggestions in this forum post (https://forum.keyboardmaestro.com/t/how-do-i-execute-shell-script-that-requires-arguments/3145/4) without success. I’ve tried using the script as file and as text, but I cannot get it to work. I could really use some help here if anyone know how I might be able to get this script to work.

Here’s the script:

#!/usr/bin/perl

use strict;
use Text::CSV;
my $num_args = $#ARGV + 1;
if ($num_args != 3) {
    print "\nUsage: $0 <template.csv> <data.csv> <output directory>\n";
    exit;
}

my $corresponding={
    '*ContactName'=>'Client',
    'Reference'=>'Project',
    '*Description'=>'Description',
    '*Quantity'=>'Hours',
    '**Date' => 'Date'
};

my $template_file = $ARGV[0];
my $data_file = $ARGV[1];
my $output_directory = $ARGV[2];

`mkdir -p $output_directory` unless -d $output_directory;


open(my $template_fh, '<:encoding(utf8)', $template_file)  or die "Could not open file '$template_file' $!";
open(my $data_fh, '<:encoding(utf8)', $data_file)  or die "Could not open file '$template_file' $!";
my @template;
my @data;


my $csv = Text::CSV->new ({
  binary    => 1,
  auto_diag => 1,
  sep_char  => ','    # not really needed as this is the default
});
 

my $header = <$template_fh>;
$header =~ s/[\r\n]+//;
$csv->parse($header);
my @header_template =  $csv->fields() ;

while(my $line = <$template_fh>){
    # chomp($line);
    $line =~ s/[\r\n]+//;
    push @template, $line;
}

$header = <$data_fh>;
$header =~  s/[\r\n]+//;
$csv->parse($header);
my @header_data =  $csv->fields() ;
while(my $line = <$data_fh>){
    # chomp($line);
    $line =~ s/[\r\n]+//;
    push @data, $line;
}

close($template_fh);
close($data_fh);

my @replacement_keys;
my $invoise_number_index;
my $contact_name_index;
my $date_index;
my $description_index;

push @header_template, "**Date";

foreach my $key (keys %{$corresponding}) {
    for(my $i=0;$i<$#header_template+1;$i++){
	$contact_name_index = $i if($header_template[$i]) eq '*ContactName';
	$invoise_number_index = $i if($header_template[$i]) eq '*InvoiceNumber';
	$date_index = $i if($header_template[$i]) eq '**Date';
	$description_index = $i if($header_template[$i]) eq '*Description';
	
	for(my $j=0;$j<$#header_data+1;$j++){
	    if($header_template[$i] eq $key && $header_data[$j] eq $corresponding->{$key}){
		push @replacement_keys, [$i,$j];
		next;
	    }
	}
    }
}
$csv->parse( $template[0].",0"); 
my @template_row = $csv->fields();

my $client_name = undef;

sub replace{
    my ($a,$b ) =@_;
    foreach my $key (@replacement_keys){
	$a->[$key->[0]] = $b->[$key->[1]] if defined($a->[$key->[0]]);
    }
    $client_name =  $a->[$contact_name_index] unless defined($client_name);
    $a->[$description_index] = $a->[$description_index] . '-' .  $a->[$date_index];
    $a->[$invoise_number_index] = 0 if defined($a->[$contact_name_index]);
    delete $a->[$date_index];
}



my @output = map { 
    $csv->parse($_);
    my @x= $csv->fields();
    my @a=@template_row; 
    replace(\@a, \@x);
     \@a;
} @data;
delete $header_template[$date_index];


@output = map { $csv->combine(@{$_}); $csv->string(); }  @output;

my ($sec,$min,$hour,$mday,$mon,$year,$wday,$yday,$isdst) = localtime(time);

#$client_name =~s /[^A-Za-z0-9\-\.]/_/g;
my $output_file = $client_name.".".($mon+1).".$mday.".($year+1900).".csv";
open(my $output_fh,">",$output_directory."/".$output_file);

$csv->combine(@header_template);    # combine columns into a string
print $output_fh $csv->string() ."\n";
print $output_fh join("\n", @output);
close($output_fh);

Thanks!

I don't know that much about Perl, but I think this should help:

It appears you need to use the format

$ENV{KMVAR_variablename}

to access KM variables from Perl.

Hmm, ok so I add that line in the script, but what do I do to pass arguments to this script via KM?

In the above example, "variablename" is the name of a variable you use set in KM. So, for example, if you have this action:

You would be able to access it in Perl (assuming my syntax is correct) like this:

$ENV{KMVAR_MyFilePath}

When KM runs your script, all KM variables are added to the environment, except for variables that contain a large amount of data (there's a limit to the size of the environment).

So if you go to the menu item Keyboard Maestro->Preferences, the Variables tab, all of those variables are available in your Perl script (with the aforementioned size provisio), by using the above syntax.

Does this make sense? I hope I didn't just confuse things.

Ok, understood. Where in the script would I make the declaration:

$ENV{KMVAR_FileName}
$ENV{KMVAR_FilePath}
$ENV{KMVAR_TemplateName}

Anywhere in the Perl script where you need the value.

I’m guessing here, because as I said, I don’t know perl (although I did use Perl once in KM, using someone else’s Perl script), but perhaps you want to replace $ARGV[0], $ARGV[1], and $ARGV[2] with the KM variables. Obviously you’d need to get rid of the code that checks the number of arguments.

There’s probably also a way to run this as a shell script, passing the variables as arguments to the Perl script so you could leave the Perl script alone, but that’s a little beyond my knowledge.

That's precisely what I want to do. Thanks for all your help, hopefully some other soul sees this post and helps out.

Cheers,

Bryan Short

I’ll bet we can get it to work. Try this:

Remove these lines:

my $num_args = $#ARGV + 1;
if ($num_args != 3) {
print "\nUsage: $0 \n";
exit;
}

Change these lines as shown:

my $template_file = $ENV{KMVAR_TemplateName};
my $data_file = $ENV{KMVAR_FileName};
my $output_directory = $ENV{KMVAR_FilePath};

Assuming I understand your your variable names, I’ll bet this works. Double-check to make sure I assigned the right KM variable to the right Perl variable.

1 Like

Well, for some reason, I keep getting the following error:

Could not open file '/Users/bryanshort/Documents/template.csv' No such file or directory at /var/folders/ly/tqxh_3yj52d6lt5rf8hbrj4w0000gn/T/Keyboard-Maestro-Script-41E2E3C5-3730-4F6C-BC1B-E8278943BE01 line 27.

Not really sure why. line 27 is this line in the code:

open(my $template_fh, '<:encoding(utf8)', $template_file) or die "Could not open file '$template_file' $!";
open(my $data_fh, '<:encoding(utf8)', $data_file) or die "Could not open file '$template_file' $!";
my @template;
my @data;

The positive news is that the variable is obviously working properly as that is the file that I am referencing in TemplateFile variable.

So something is working properly.

Yeah, at least part of the problem is solved.

A couple of obvious questions:

  1. Are you sure the file is there, and that you spelled the entire path correctly?
  2. Is the file encoded in UTF8?

Other than that, we’ve reached the end of my usefulness.

  1. Are you sure the file is there, and that you spelled the entire path correctly?

Yes, I've checked it and rechecked it.

  1. Is the file encoded in UTF8?

Yes it is. The script works fine from CLI with same parameters. Just Using KM doesn't seem to like it.

Thanks anyways.

Someone will definitely come along and help. Good luck.

Hey Bryan,

I take it you’ve used this script successfully from the command-line – yes?

Please provide a working example command-line invocation.

Please also provide a working csv sample file.

Your script uses a non-standard module that Keyboard Maestro doesn’t see, and it shouldn’t be too hard to fix given the right stuff to test with.

-Chris

The error seems pretty clear, check that the source file is actually at that location.

In the Terminal, do the command:

ls -l /Users/bryanshort/Documents/template.csv

does it show the file?

yes the file is there.

Thanks for everyone’s help, I’ve decided that rather than spending hours to fix this, I’ll just rehire the developer that created the script for me and have him modify the script such that it iterates through each file in the dir, performs operations and then deletes files in dir. Much more economical for me to pay for code than figure this out myself.

Thanks for all your help!

Cheers,

Bryan Short