Troubleshooting Oozie or other Hadoop errors with DEBUG logging

In troubleshooting Hadoop issues, we often need to review the logging of a specific Hadoop component. By default, the logging level is set to INFO or WARN for many Hadoop components like Oozie, Hive etc. and in many cases this level of logging is sufficient to trace the issue. However, in certain cases, INFO or…

1

Sliding Window Data Partitioning on Microsoft Azure HDInsight

HCatalog is a table and storage management layer for Hadoop that enables users with different data processing tools like Pig, Mapreduce, Hive, and Oozie to read and write data. HCatalog’s table abstraction presents these tools and users with a relational view of data in the cluster. HCatalog Integration was made available starting with Apache Oozie…


Oozie sqoop action hits primary key violation

We have seen multiple customers contact us where an oozie job appears to hang. The oozie job involves a sqoop action which is exporting data from a file in HDInsight to a table in a SQL Azure database. For background on Sqoop see Getting Started with Sqoop . We will use this blog to help…

1

HDInsight News – New Articles to read

Hi Folks, I’m Jason from the Microsoft Big Data Support team. Thanks for reading our blog, and for trying out HDInsight in your own business. I want to share some new articles Microsoft just published that will be helpful for getting started with HDInsight in your business. To help folks who are not so familiar…

0