AliCloud Technology Sets World Sorting Records
Company’s FuxiSort solution sorts 100TB of data in 377 seconds
AliCloud’s FuxiSort tops the 2015 Sort Benchmark contests
Singapore, October 30, 2015 －AliCloud, Alibaba Group’s (NYSE: BABA) cloud computing division,
In 2015 results recently released on the Sort Benchmark website, AliCloud’s FuxiSort took less than six-and-a-half minutes (377 seconds) to sort 100 TB of data, crushing a 23.4-minute record set in 2014 by Apache Spark, which had itself replaced a previous Hadoop record of 72 minutes.
The AliCloud team employed a cluster of 3,377 commodity servers* to set the Daytona GraySort record of 15.9TB/min and Daytona MinuteSort record of 7.7TB, an improvement of 3.6x and 2.1x over the previous records respectively.
Sort Benchmark competition
2014 World Records
2015 World Records
Apache Spark: 4.27TB/min.
Source: SortBenchmark.org. The larger the number, the better the performance.
“Making a clean sweep of the 2015 GraySort and MinuteSort categories for both Daytona and Indy categories with FuxiSort in our first year of participation is a clear validation of AliCloud’s performance leadership. We cannot rest on our laurels and will strive to process even higher volumes of data in shorter times going forward. Ultimately, our ultimate goal is to offer our customers the best possible experience at all times,” said Chao LI, team leader, Fuxi
“As more mobile devices and sensors from the Internet of Things put data online, we will be capturing and analyzing ever larger volumes of data in various formats. Gaining accurate, actionable insights affordably and quickly from increasingly large volumes of data will require smarter technologies. AliCloud has proven expertise in this field, and we are committed to pushing the state-of-the-art technologies harder, faster, and further.”
FuxiSort is built on top of Apsara, a general-purpose computing system developed in-house from scratch by AliCloud. Apsara, which debuted in 2011, manages cluster resources within a data center, and schedules parallel execution for a wide range of distributed online and offline applications. Apsera is the foundation for the majority of public cloud services offered by AliCloud, including Open Data Processing Service (ODPS), Open Storage Service (OSS) and Open Table Service (OTS). It supports all data-processing workloads within Alibaba Group as well. Fuxi, named after a god in Chinese mythology, is the framework that handles cluster-resource management and job scheduling within Apsara.
Apsara has been deployed on hundreds of thousands of physical servers in AliCloud data centers. A single Apsara cluster can be scaled up to 5,000 servers with hundreds of petabytes of storage and hundreds of thousands of CPU cores, making this unique computational engine one of the most powerful of its kind in the world. Together, they form the backbone of AliCloud’s comprehensive suite of cloud services.
For further technical details of FuxiSort and Apsara, please refer to the technical report at http://sortbenchmark.org/
FuxiSort2015.pdf. For more information on the Sort Benchmarks and Benchmark Categories, please visit http://sortbenchmark.org.
*3,134 nodes x (dual Xeon E5-2630 2.30Ghz, 96 GB memory, 12x2 TB SATA HD, 10 Gb/s Ethernet) and 243 nodes x (dual Xeon E5-2650v2 2.60Ghz, 128 GB memory, 12x2 TB SATA HD, 10 Gb/s Ethernet)
For the LATEST tech updates,
FOLLOW us on our Twitter
LIKE us on our FaceBook
SUBSCRIBE to us on our YouTube Channel!