This article applies to Sparkling Water for h2o versions 3.24.0.5 and later.


After setting up Sparkling Water for your environment follow these steps:


1. Start sparkling-shell from the Sparkling Water folder:

bin/sparkling-shell


2. Import the parquet file:

import org.apache.spark.sql.SparkSession

val sqlContext = SparkSession.builder().getOrCreate().sqlContext
val parquetFile = sqlContext.read.parquet("/path/to/file/")


   To preview the imported file:

parquetFile.show(false)


3. Flatten the parquet file:

import org.apache.spark.h2o.utils.H2OSchemaUtils

val flattenDF = H2OSchemaUtils.flattenDataFrame(parquetFile)


   To preview the flattened data frame:

flattenDF.show(false)


4. Save the flattened file to disk:

flattenDF.write.parquet("flattened.parquet")