This article applies to Sparkling Water for h2o versions 3.28.0.1 and later.


After setting up Sparkling Water for your environment follow these steps:


1. Start sparkling-shell from the Sparkling Water folder:

bin/sparkling-shell


2. Import the parquet file:

val sqlContext = spark.sqlContext
val parquetFile = sqlContext.read.parquet("file:///path/to/file/")


   To preview the imported file:

parquetFile.show(false)


3. Flatten the parquet file:

import ai.h2o.sparkling.ml.utils.SchemaUtils

val flattenDF = H2OSchemaUtils.flattenDataFrame(parquetFile)


   To preview the flattened data frame:

flattenDF.show(false)


4. Save the flattened file to disk:

flattenDF.write.parquet("file:///tmp/flattened.parquet")