Skip to content
Big Data Interview
The Interview Hacker and Technical guide
  • Home
  • Blogs
  • About Us
  • Contact Us
  • Privacy Policy

Month: October 2020

Difference between IN operator and EXISTS operator in HIVE or SQL.

October 26, 2020 admin Leave a comment

EXISTS EXISTS operator will be used when we need to check if there is any row exists with a condition.…

Continue Reading →

Posted in: Hive, Spark SQL Filed under: EXISTS operator, IN and EXISTS in SQL, IN Operator, SQL

Difference between dropDuplicates() and distinct.

October 10, 2020 admin Leave a comment

To delete duplicate records in a DataFrame we can either use distinct or dropDuplicates method. But dropDuplicates method comes with…

Continue Reading →

Posted in: Spark Filed under: Distinct, dropDuplicates

Difference between withColumn and select() on DataFrame in Spark.

October 5, 2020 admin Leave a comment

While working with Big Data applications we might have used the methods withColumn and select(). They both are few of…

Continue Reading →

Posted in: Big Data, Spark Filed under: select(), withColumn, withColumnHiddenCost

What is the difference between cache and broadcast variable?

admin Leave a comment

Spark has lot of performance enhancement techniques. Two of them are cache() and broadcast variables. Although they both used to…

Continue Reading →

Posted in: Big Data, Spark Filed under: Broadcastvariable, Cache

Recent Posts

  • Save action in Spark takes too long time/Save operation spills huge data on to disk and fails with the error “No space left on device”
  • How to set configuration to start Reduce jobs after completion of certain proportion of the Map jobs in Hive or Hadoop?
  • HDFS commands

Recent Comments

  • curry 7 sour patch on Spark groupByKey vs reduceByKey vs aggregateByKey
  • jordan 4 on Hive – Order By vs Sort By vs Cluster By vs Distribute By
  • louboutin shoes on Spark RDD vs Dataframe vs Dataset

Archives

  • August 2021
  • June 2021
  • May 2021
  • January 2021
  • December 2020
  • October 2020
  • July 2020
  • May 2020
  • April 2020
  • March 2020
  • November 2019
  • July 2019
  • June 2019
  • May 2019

Follow Us

Contact Us

  • Email
    sparkandbigdatainterview@gmail.com
Privacy Policy
Copyright © 2023 Big Data Interview