home · mobile · calendar · colloquia · 1997-1998 · 

Colloquium - Wang

Surviving the I/O Blind Spots
University of California, Berkeley

I start this talk by asking two simple questions: 1) What is the minimum latency of transferring a file block over the network? 2) What is the minimum latency of writing a file block to disk?

These questions arise naturally as network attached storage systems are converging towards a common architecture that seeks to exploit the fast switched networks and the aggregate resources of a cluster. Although deceptively simple, these questions represent two important I/O blind spots that are not well understood. In the process of answering these questions, I first present a general pipeline theory for minimizing communication latency. Then I describe a new file system paradigm based on a programmable embedded disk processor (IDISK).

Although pipelining is a well known latency reduction technique, little research exists to guide the construction of optimal fragmentation algorithms. In the first half of my talk, I present a framework that can lead to a complete theory of optimal communication pipelines. I demonstrate several important optimality criteria. I provide a methodology that systematically uncovers communication pipeline parameters and constructs customized pipeline algorithms. I demonstrate significant performance improvement by applying the model to the Myrinet-GAM system.

Existing file systems suffer from two common fundamental assumptions that handicap their designs: that a small disk write is intrinsically expensive and that reorganizing the disk layout is a heavy-weight operation. By dedicating part of the storage management to the intelligent embedded disk processor, we are faced with the unique opportunity of rethinking a new file system paradigm based on low-latency persistent writes and light-weight reorganization. In the second half of the talk, I present the design and performance results of an IDISK-based file system.

Together, these two pieces form a synergy that allows one to craft a well-balanced I/O path and fully realize the performance potential of modern network file systems.

Department of Computer Science
University of Colorado Boulder
Boulder, CO 80309-0430 USA
May 5, 2012 (14:13)