Skip to content

Big Data Interview

The Interview Hacker and Technical guide
  • Home
  • Blogs
  • About Us
  • Contact Us
  • Privacy Policy

Blog

  1. Pages:
  2. «
  3. 1
  4. 2
  5. 3
  6. 4
  7. 5
  8. 6
  9. 7
  10. 8
  11. 9
  12. »

How can we Pivot and Unpivot data in Spark (or) What is Pivot in Spark?

December 4, 2020 admin Leave a comment

In Big Data applications we rarely get the requirement to pivot the data because making transpose of billions of rows…

Continue Reading →

Posted in: Spark Filed under: Pivot, Unpivot

Difference between IN operator and EXISTS operator in HIVE or SQL.

October 26, 2020 admin Leave a comment

EXISTS EXISTS operator will be used when we need to check if there is any row exists with a condition.…

Continue Reading →

Posted in: Hive, Spark SQL Filed under: EXISTS operator, IN and EXISTS in SQL, IN Operator, SQL

Difference between dropDuplicates() and distinct.

October 10, 2020 admin Leave a comment

To delete duplicate records in a DataFrame we can either use distinct or dropDuplicates method. But dropDuplicates method comes with…

Continue Reading →

Posted in: Spark Filed under: Distinct, dropDuplicates

Difference between withColumn and select() on DataFrame in Spark.

October 5, 2020 admin Leave a comment

While working with Big Data applications we might have used the methods withColumn and select(). They both are few of…

Continue Reading →

Posted in: Big Data, Spark Filed under: select(), withColumn, withColumnHiddenCost

What is the difference between cache and broadcast variable?

admin Leave a comment

Spark has lot of performance enhancement techniques. Two of them are cache() and broadcast variables. Although they both used to…

Continue Reading →

Posted in: Big Data, Spark Filed under: Broadcastvariable, Cache

How to delete duplicate records in Hive (or) How to extract unique records in Hive using analytical functions.

July 14, 2020 admin Leave a comment

In this post we will see how we can extract unique records from a Hive table. This can be achieved…

Continue Reading →

Posted in: Hive

RANK() vs DENSE_RANK() vs ROW_NUMBER() in Hive (or) Differences between RANK(), DENSE_RANK() and ROW_NUMBER() (or) Ranking window functions in Hive

admin Leave a comment

One of the most frequent questions during Data Engineering interviews. These are called Ranking functions in Hive. These are the…

Continue Reading →

Posted in: Hive Filed under: DENSE_RANK() in Hive(), RANK() in Hive, RANK() vs DENSE_RANK() vs ROW_NUMBER() in Hive, ROW_NUMBER() in Hive

How to calculate moving sum or moving average in Hive?

admin Leave a comment

This post will focus on calculating moving average or sum using Hive queries. We might have come across this question…

Continue Reading →

Posted in: Hive Filed under: Moving Average in Hive, Moving sum in Hive

Hive CLI vs Beeline (or) Difference between Hive CLI and Beeline

July 13, 2020 admin Leave a comment

Hive CLI and Beeline both can be used to interact with Hive execution engine. But there are few differences between…

Continue Reading →

Posted in: Hive Filed under: Beeline, Hive CLI, Hive CLI vs Beeline

Data Engineer interview preparation/Bigdata Interview Questions/Data Engineer Interview Questions

admin Leave a comment

If you are an aspiring Data Engineer and reading this article implies that you have landed on a wonderful website.…

Continue Reading →

Posted in: Big Data, Data Engineering Filed under: BigData Interview Questions, Data Engineer Interview Question, Scenario based Bigdata interview questions
  1. Pages:
  2. «
  3. 1
  4. 2
  5. 3
  6. 4
  7. 5
  8. 6
  9. 7
  10. 8
  11. 9
  12. »

Post navigation

Page 2 of 9
← Previous 1 2 3 … 9 Next →

Follow Us

Contact Us

  • Email
    sparkandbigdatainterview@gmail.com
Privacy Policy
Copyright © 2021 Big Data Interview