Assume that we have an RDD of m partitions and another RDD of n partitions. When we perform union operation on first RDD with second RDD we will get a new RDD of partitions m+n.
Let's take an example scenario and do it. We have an RDD of some number of partitions. Here I have a repartitioned the RDD to 1 partition and stored in a variable. And I have repartitioned the same RDD to 4 partitions. Then I have performed union operation on these two RDDs. Then I got a new RDD of 5 partitions.
Look at the screenshot of the operations performed.
If you liked this post or if you feel anything can be enhanced in this post please let us know..
My brother recommended I may like this blog. He was totally right. This submit actually made my day. You cann’t consider simply how so much time I had spent for this info! Thank you!| а
Like!! I blog frequently and I really thank you for your content. The article has truly peaked my interest.
Like!! I blog quite often and I genuinely thank you for your information. The article has truly peaked my interest.
Thanks for ones marvelous posting! I really enjoyed reading
it, you can be a great author. I will ensure that I bookmark your blog and will eventually come back in the future.
I want to encourage you to ultimately continue your great posts,
have a nice morning!
Thank you so much. Please comment if we have missed any Bigdata concepts on our blog.