BigDataFr recommends: Spark speeds towards the next data processing revolution

 »“A mighty flame followeth a tiny spark.” —Dante Alighieri

If you know anything about Apache Spark, you know that its chief claim to fame is speed. With in-memory processing, Spark can achieve ten to hundredfold improvements or more in data-processing times compared to traditional MapReduce. Spark is also highly flexible, supporting a wider variety of workloads than traditional systems. But faster and highly flexible don’t do justice to the sea change for data processing that is enabled by Spark.

My first experience with the Internet was through a 300-baud modem. When 1200 baud came along, life got better, and when I upgraded to 2400 baud, I thought I had died and gone to heaven. But those changes were incremental. The emails I was sending and files I was transferring went faster than before, but my horizons had not broadened much.

Broadband’s arrival was a different story. The quantum improvement in speed, along with the fact that it was always on, enabled vast new application areas that simply did not exist before. Video, gaming, streaming music, real-time communication, video conferencing and more suddenly took the Internet by storm. These applications in turn enabled and attracted a new class of Internet end user, and the modern Internet era was born. Not until the recent mobile explosion did we see another revolution as significant. ‘

Read more
By Timothy A. Howes, Chief Technology Officer, ClearStory Data
Source: ibmbigdatahub.com