Dynamically Optimizing Research Data Workflow

Dynamically Optimizing Research Data Workflow

Intelligent management of campus research networks has become a major challenge for Yale and many other institutions, as their networks grow rapidly in size and complexity in order to meet the demands of on-campus researchers who are conducting research, collaborating with peers, and fulfilling the institution’s educational mission. Traditional, static approaches to network management are no longer adequate, since they often result in low efficiency, poor usability, and unpredictable network application performance. Yale was recently awarded an NSF grant to design and deploy a novel intelligent network cyberinfrastructure that will greatly expand the ability of researchers to rapidly and efficiently move the large quantities of data required for their computation- and data-intensive research activities both on- and off-campus. Led by Professor Richard Yang, Andrew Sherman, David Galassi, and Robert Bjornson, the project is a joint effort of faculty and staff from the Department of Computer Science, Yale’s High Performance Computing Center, and ITS. It will focus initially on a broad range of applications in the sciences, including astrophysics, climatology, and genomics. However, the new network cyberinfrastructure developed in the project will benefit the full breadth of research at Yale, including the social sciences and digital humanities. The project is based on several prior network research and development efforts at Yale. It will unify the recently deployed high-speed Science and Laboratory Networks and will make it possible to intelligently manage and optimize these networks. To achieve this, the researchers will leverage, validate, and extend Maple, a novel Software Defined Networking (SDN) programming framework developed in the Department of Computer Science, as well as an Application Layer Traffic Optimization (ALTO) protocol and framework pioneered at Yale and now incorporated in a proposed standard for the Internet by the Internet Engineering Task Force. Maple simplifies network programming for end-to-end, complex, dynamically constructed network services, while the ALTO protocol enables network applications to adapt dynamically, according to the state of the network, to deliver high levels of network efficiency and application quality of service.