The Apache Spark community has improved support for Python to such a great degree over the past few years that Python is now a “first-class” language, and no longer a “clunky” add-on as it once was, ...

Apache Spark is a project designed to accelerate Hadoop and other big data applications through the use of an in-memory, clustered data engine. The Apache Foundation describes the Spark project ...

Spark also provides many language choices, including Scala, Java, Python, and R. The 2015 Spark Survey that polled the Spark community shows particularly rapid growth in Python and R.

The Python-versus-R-in-Spark discussion also carries over to the production side of the equation. In the olden days of Spark (i.e. 18 months ago), putting a Spark job into production often required ...