COMSYS TALK : " Networking for Big Data: Theory, Algorithms and Applications "

Professor Edmund Yeh from Northeastern -
Communication systems

Date: -
Location: Eurecom

Abstract : In the era of big data, experts in various fields are facing unprecedented challenges in data access, distribution, processing and analysis, and in the coordinated use of limited computing, storage and network resources. To address this, we present new frameworks for the optimization of key network functionalities, which are broadly applicable to content delivery networks, wireless heterogeneous networks, and distributed computing networks. The frameworks enable joint (in-network) caching, request routing, and congestion control for content distribution over general network topologies, optimizing metrics including routing costs, data retrieval delay, and content-based fairness. We meet the challenge of the underlying NP-hard problems by exploiting submodularity, matroid structure, DR-submodularity, and by leveraging tools including concave relaxation, stochastic gradient ascent, continuous greedy and Lagrangian barrier algorithms. We develop polynomial-time approximation algorithms with proven optimality guarantees, with particular emphasis on adaptive and distributed implementations. We further discuss the extension of these frameworks for jointly optimal wireless user association and content caching in wireless heterogeneous networks, and for jointly optimal computation scheduling, caching and request forwarding in distributed computing networks. Finally, we discuss an ongoing project which applies the optimization frameworks and algorithms to facilitate data distribution and computation in the Large Hadron Collider (LHC) high-energy physics network, one of the largest data applications in the world.