Circus: Opportunistic block reordering for scalable content servers

Published

Conference Paper

© 2004 by The USENIX Association All Rights Reserved. Whole-file transfer is a basic primitive for Internet content dissemination. Content servers are increasingly limited by disk arm movement given the rapid growth in disk density, disk transfer rates, server network bandwidth, and content size. Individual file transfers are sequential, but the block access sequence on a content server is effectively random when many slow clients access large files concurrently. Although larger blocks can help improve disk throughput, buffering requirements increase linearly with block size. This paper explores a novel block reordering technique that can reduce server disk traffic significantly when large content files are shared. The idea is to transfer blocks to each client in any order that is convenient for the server. The server sends blocks to each client opportunistically in order to maximize the advantage from the disk reads it issues to serve other clients accessing the same file. We first illustrate the motivation and potential impact of opportunistic block reordering using a simple analytical model. Then we describe a file transfer system using a simple block reordering algorithm, called Circus. Experimental results with the Circus prototype show that it can improve server throughput by a factor of two or more in workloads with strong file access locality.

Duke Authors

Cited Authors

  • Anastasiadis, SV; Wickremesinghe, RG; Chase, JS

Published Date

  • January 1, 2004

Published In

  • Proceedings of the 3rd Usenix Conference on File and Storage Technologies, Fast 2004

Citation Source

  • Scopus