Computer Science Ph.D. Student Receives Best Paper Award at International Conference
Deep-learning algorithms for supercomputing will be more efficient due to research conducted by an Illinois Institute of Technology doctoral student, who was recognized at a prestigious international computing conference.
Hariharan Devarajan (Ph.D. CS ’21) earned a best paper award at the Institute of Electrical and Electronics Engineers/Association for Computing Machinery CCGrid conference hosted in Melbourne, Australia, for his innovations in identifying data bottlenecks that hinder input/output (I/O) performance in deep-learning algorithms. By identifying these bottlenecks, scientists will be able to extract more performance from supercomputers in their research.
“Consumption of large amounts of data is necessary for deep-learning techniques,” Devarajan says. “Due to the explosion in big data’s volume, there is an I/O bottleneck on large-scale distributed deep-learning training. However, there is a lack of understanding of the I/O behavior of large-scale deep learning applications. A deep understanding of the I/O behavior of deep-learning applications can enable us to reduce the I/O bottleneck through various data-access optimizations.”
Devarajan’s adviser, Ron Hochsprung Endowed Chair and Distinguished Professor of Computer Science Xian-He Sun, explains advancements in deep learning have been reliant on the availability of high-performance computing power and big data. A combination of increasing volume in big data sets and supercomputers running more applications with different designs has exasperated the I/O bottlenecks.
“As a result, deep learning has not found success in traditional scientific domains such as climate modeling or DNA folding,” Sun says.
Devarajan’s paper provides a detailed investigation of the I/O behavior of various scientific deep-learning workloads running on Theta, a production supercomputer at the Argonne Leadership Computing Facility. The investigated workloads belong to various projects, such as Argonne Data Science Program, Aurora Early Science Program, and Exascale Computing Projects. Devarajan studied and characterized the I/O behavior of eight scientific deep-learning applications running on Theta supercomputer.
“Based on the characterization, we developed a novel representative benchmark suite called DLIO: A Data-Centric Benchmark for Scientific Deep Learning Applications,” he says.
Devarajan discovered that deep-learning applications use scientific data formats that are not well-supported by deep-learning frameworks. However, DLIO can increase efficiency by six times on existing applications by allowing scientists to fine tune their I/O access.
“The DLIO benchmark enables rapid prototyping for researchers to optimize I/O for deep-learning applications,” Devarajan says.
Devarajan began the research as a summer intern under the supervision of Huihuo Zheng at Argonne National Laboratory’s Data Science Group, headed by Venkatram Vishwanath at Argonne Leadership Computing Facility. By interacting with domain scientists there, Devarajan says he began to understand new perspectives of deep-learning applications and enhancing collective knowledge through active collaboration. He completed the research under Sun’s supervision after the conclusion of his internship.
“Profiling the I/O behavior for these applications is very challenging because of the lack of tools to produce I/O profiling and analyzing them,” Devarajan says. “As a part of this work, we built a methodology to effectively profile scientific deep-learning applications on production supercomputers. Also, we developed several tools to analyze these profiling logs and generate a holistic view of the I/O behavior.”
Devarajan presented his findings at the IEEE/ACM CCGrid conference virtually.
Photo: Hariharan Devarajan [second from right] works with fellow students in February 2020
