Apache SystemML

Apache SystemML is a machine learning system with support for Java 8+, Scala 2.11+, Python 2.7/3.5+, Hadoop 2.6+, and Spark 2.1+. It provides a workplace for machine learning using big data. As It runs on top of Apache Spark, it automatically scales data, line by line, determining whether the code should be run on the driver or an Apache Spark cluster. Future releases may include additional deep learning with GPU capabilities such as importing and running neural network architectures and pre-trained models for training.

Apache SystemML is customizable with R-like and Python-like languages. It also supports multiple execution modes, including Spark MLContext API, Spark Batch, Hadoop Batch, Standalone, and JMLC and automatic optimization based on data and cluster characteristics to ensure both efficiency and scalability.

Documentation | Mailing lists