Friday, July 17, 2015

Dynamically assign fileref to external file

Sometimes, you might want to assign fileref to external file which path you get to know during some complex data manipulations. In that case, filename statement is difficult to use as filepath you need to know in advance. Along with FILENAME statement, SAS offers also FILENAME function, which you can use within the DATA step.

SAS Base

data test;
  Path='/path/to/your/csv/file.csv';
run;

data _null_;
  set test;
  rc=filename("fid",Path);
run;

data csvData(drop=path);
  infile fid;
  input ...
run;

SAS Data Integration Studio

Let's assume that you already have SAS dataset created that contains path to csv file. Then you need to:
  • add "User Written Code" transformation (under "Data->User Written Code"),
  • connect SAS dataset (that contains path to csv file) to "User Written Code transformation".
  • Go to "User Written Code" properties, tab "Code" (keep "Code generation mode: User Written Body"), and in the code body write the following text right above %rcSet(&syserr... (it is sometimes hard to see, but automatically generated code has a greyish background, and your code has complete white background):
        data null; 
          set &_INPUT.; 
          rc=filename("fid",Path); 
        run;
    
  • now create a new "External File", set metadata you want (I assume you know what you expect as a format of the csv file), and
  • under "External File"s Properties, tab "File Location", under "File name:", write simply "fid" (without quotes), and under "File Name Quoting" on the same tab, choose "No quotes around file name".

Redirect libref to WORK library in SAS

Imagine you have a SAS Enterprise Guide Project, where you have been using a libref (let's name it "SAVE") very often. However, now you want to switch it in specific occassion turn to WORK library so that it does not require a specific folder for SAS datasets to be saved. Also, you would still like to have the LIBNAME statement for SAVE libref, so that you can switch easily from WORK libref to specific path. so, the following two solutions you can use:
LIBNAME save %sysfunc(quote(%sysfunc(pathname(work)))) ;
the second way to do it is the following:
%let workLocation=%sysfunc(getoption(work));
libname SAVE "&workLocation.";

Read multiple external files into single dataset in SAS

Sometimes you get to have a multiple external files, which you need to read in SAS.
For further manipulation, it looks good if these data could all be gathered in the same dataset, from which then usual data manipulation queries could be performed on.


%let dirname =/path/to/folder/to/scan;
%let ext=csv;

data dirlist (keep=filepath fext fname);
 rc=filename("filerd","&dirname.");
 if rc eq 0 then do;
  fid=dopen("filerd");
  count=dnum(fid);
  length filepath $256; 
  do i=1 to count;
   fname=dread(fid,i);
   fext=scan(fname,-1,".");
       if upcase(fname) ne upcase(fext) and upcase(fext) eq upcase("&ext.") then do;
     filepath=cats("&dirname./",fname);
     output;
   end;
  end;
  rcc=dclose(filerd);
 end;
run;

data all_text;
  length myfilename $100;
  set dirlist;
  infile dummy filevar = filepath length=reclen end=done missover delimiter=';' dsd;
  do while(not done);
    myfilename = filepath;
    input a $ b $ c $ d $;  
    output;
  end;
run;

proc print data=all_text;
run;
Now, if you would like to conditionally to read multiple files that have different structure, you can conditionally change the input statement. Let's say that file has the different structure if its name ends with "d" character:
data all_text;
  length myfilename $100;
  set dirlist;
  infile dummy filevar = filepath length=reclen end=done missover delimiter=';' dsd;
  do while(not done);
    myfilename = filepath;
 if prxmatch("/d.&ext.$/",trim(fname)) then do;
 input un1 $ un2 $ un3 $ a $ b $ c $ d $;
 drop un1 un2 un3;
 end;
 else do;
    input a $ b $ c $ d $;
 end;
    output;
  end;
run;

Tuesday, April 28, 2015

Error: Invalid business key defined

In SAS DI Studio, when you try to add the column as Business Key which is already selected as a Detect Changes, you will get the error:

Error: Invalid business key defined

In order to resolve that, you need to remove whatever business keys you want to define from "Detect Changes" tab, it error is gone.

Tuesday, March 10, 2015

Java Expert RE-Learning Javascript Part 2 - Operators

What do you thing, what does javascript do with the following ?

"5" + 1
- result? "51"
5 + "1"
- result? "51"
"5" - 1
- result? 4
"5" == 5
- result? true
null == undefined 
- result? true

How did you do? I was bad at it, especially to realize that "5" + 1 and 5 + "1" would make the same result. Javascript tries always to do conversion whenever the types of operands dont match.

Javascript is pretty easy-going language. It allows you to do many things, giving you a bit of additional freedom. BUT you have to be VERY careful, as you can imagine that in some cases, above mentioned freedom might cost you a lot of bugs :)

So, summary of rules in Javascript about operators:


  • If you give Javascript different types to operate with, it will start doing type conversion. The way it would do it is based on set of (often confusing) rules. It would check first the operator, and then try to convert operands to the type appropriate to the operator. However, Javascript sees addition operator (+) as string concatenaton first.
  • null and undefined in operation with numbers will convert to 0 and NaN respectively (examples: 8+null = 8 , 8+undefined = NaN)
  • In order to assure yourself that the two values are properly compared, you should rather use === instead of ==. === (triple equals), checks precisely if the values are equals, without doing any automatic conversions.


So, what is the difference between double equals and tripple equals? With "==", Javascript tries to do automatic conversion before comparing the values, and with "===", Javascript does the precise comparison, without trying to "be smart".

Now the most favorite new thing for me: Logical Operators!

The rule sof && and || operators are the following:

  1. Javascript will try to convert the left-hand value to Boolean type in order to decide what to do,
  2. depending on operator (&& or ||) and the result of that conversion, it will return either original left-hand or right-hand value.
  3. "" is converted to false, and "jklkjsda" is converted to true. 0 is false, and 3124 is true.
some examples:
"Carl" && "Bob"
"Bob"
"Carl" || "Bob"
"Carl"
"" && "Bob"
""
"" || "Bob"
"Bob"

Java Expert RE-Learning Javascript Part 1 - Intro

You know the time when you come across so many different programming languages, it happens that you come cross programming language you never really learn from scratch, but just "learn by doing".

It has been like that all the way until I had "awakening" call, a job interview, where I got javascript questions which I did not know at all.

The biggest issue was its similarity to Java/C programming languages, where many things I rather "assumed" it is the same, without ever really checking it out.

After sobering up, I decided to RE-learn Javascript, and started reading book "Eloquent Javascript" . after reading now already a quarter of a book, I am positively surprised how nice it was written. I strongly suggest to anyone, either a core beginner in programming (no pre-knowledge whatsoever is required to read it), or even a veteran programmer. This book has also very challenging exercises in the end which solutions (can be found on above link) are really showcase-ing you the power of javascript in its core.

Wednesday, February 11, 2015

Today I listened a really nice clean-code google talk about bad practice of using IFs, and how much polymorphism helps out there. It is amazing to see how nicer code becomes, and how easy it is for a programmers to go into routine of making if/switch statements for no reason. All methods mentioned there are not new at all, but having it here shown reminds you on what and how should you think when programming.