Skip to content Skip to sidebar Skip to footer
Showing posts with the label Hadoop

Python Udfs In Pig

I've seen the documentatio here, but I confess that I feel it rather lacking. I was wondering … Read more Python Udfs In Pig

Can Pyarrow Write Multiple Parquet Files To A Folder Like Fastparquet's File_scheme='hive' Option?

I have a multi-million record SQL table that I'm planning to write out to many parquet files in… Read more Can Pyarrow Write Multiple Parquet Files To A Folder Like Fastparquet's File_scheme='hive' Option?

Encountered Ioexception While Registering Python Udf In Pig. File Helloworld.py Does Not Exist

Pytjon UDF : @outputSchema('word:chararray') def helloworld(): return 'Hello, World&#… Read more Encountered Ioexception While Registering Python Udf In Pig. File Helloworld.py Does Not Exist

Running A Job Using Hadoop Streaming And Mrjob: Pipemapred.waitoutputthreads(): Subprocess Failed With Code 1

Hey I'm fairly new to the world of Big Data. I came across this tutorial on http://musicmachin… Read more Running A Job Using Hadoop Streaming And Mrjob: Pipemapred.waitoutputthreads(): Subprocess Failed With Code 1

Connecting To Kerberized Hadoop Cluster Using Python Module Impyla

I am using impyla module to connect to kerberized hadoop cluster. I want to access hiveserver2/hive… Read more Connecting To Kerberized Hadoop Cluster Using Python Module Impyla

Pyhive, Sqlalchemy Can Not Connect To Hadoop Sandbox

I have installed, pip install thrift pip install PyHive pip install thrift-sasl and since pip ins… Read more Pyhive, Sqlalchemy Can Not Connect To Hadoop Sandbox

Remove Empty Line Printed From Hive Query Output Using Python

i am performing a hive query and storing the output in a tsv file in the local FS. I am running a f… Read more Remove Empty Line Printed From Hive Query Output Using Python

Transport Exception

I'm trying to import happybase but gets the following error message while connecting. I have Ha… Read more Transport Exception

Delimiting Carat A In Python

I have data in form: 37101000ssd48800^A1420asd938987^A2011-09-10^A18:47:50.000^A99.00^A1^A0^A 37101… Read more Delimiting Carat A In Python

Launch Hadoop Mapreduce Job Via Python Without Putty/ssh

I have been running Hadoop MapReduce jobs by logging into SSH via PuTTy which requires that I enter… Read more Launch Hadoop Mapreduce Job Via Python Without Putty/ssh

Ambari Server Setup: Oserror: [errno 2] No Such File Or Directory

I'm trying to setup Hadoop on my EC2 instance using this tutorial. I'm trying to setup the … Read more Ambari Server Setup: Oserror: [errno 2] No Such File Or Directory