A Distributed Evolutionary Multivariate Discretizer (DEMD)

Distributed multivariate discretizer for Apache Spark  based on a evolutionary points selection scheme, called Distributed Evolutionary Multivariate Discretizer (DEMD). It has been inspired by EMD evolutionary discretizer, so both algorithm have things in common as a binary representation and a fitness function, however many improvements have been introduced in DEMD to suit a distributed environment, for example that partial solutions are generated locally and fused eventually to produce the final discretization scheme. 

Status

Use

To Include this package in your Spark application via spark-shell or pySpark, you must use it like:

$SPARK_HOME/bin/spark-shell\ --packages sramirez:spark-DEMD-discretizer:1.0

where $SPARK_HOME is your Spark path.

Release

The latest version is :  1.0.0       Date: 2016-02-04 / Scala version: 2.10

Reference

S. Ramírez-Gallego, S. García, J.M. Benítez, F. Herrera. A distributed evolutionary multivariate discretizer for Big Data processing on Apache Spark. Swarm and Evolutionary Computation (2017), in press. doi: 10.1016/j.swevo.2017.08.005