Generate Unique Identifiers (UID) in U-SQL on Azure Data Lake Analytics with Python extension scripts

U-SQL doesn’t support constructs to generate Unique Identifier in Text Files. The script below generates unique identifier for every row in the input file. The steps are Extract the data file with the EXTRACT statement REDUCERS are spun based on the customer code. Too little reducers or too many reducers may both cause performance issues….

0

U-SQL Script with Python extension to detect Invalid input files

  The script below validates each of input files in the folder and the python script splits and count the number of columns in each row of every. Those files that either have > or < than 9 columns in any of its rows are all logged as Invalid files.   REFERENCE ASSEMBLY [ExtPython];  …

0