SE Concerns: Google Sorts a Petabyte

In an example of how software engineering is concerned with issues of scale: here's a blog post by Google that talks about how they used MapReduce (something will talk about later in the semester) to sort 1 petabyte of data. They first report on how long it takes them to sort 1 TB of data: 68 seconds. They then moved on to sort a petabyte (1000 terabytes) of data that was divided into 10 trillion 100-byte records: using 4000 computers they sorted that amount of data in six hours and two minutes. Enjoy!

© Kenneth M. Anderson, 2009