This white paper proposes that shuffle time in Hadoop can be reduced by using an SDN-enabled infrastructure. The paper highlights the techniques to measure network traffic for shuffle phase of Hadoop MapReduce jobs running over the SDN-enabled topology.
The paper sheds light on the Lambda architecture- a combination of batch and real time processing capabilities built on a big data platform and how it was used to enable real time decision making and boost campaign performance in the mobile advertising domain.