Generate schema from case class
Spark provides an easy way to generate a schema from a Scala case class. For case class A
, use the method ScalaReflection.schemaFor[A].dataType.asInstanceOf[StructType]
. For example:
%scala
import org.apache.spark.sql.types.StructType
import org.apache.spark.sql.catalyst.ScalaReflection
case class A(key: String, time: java.sql.Timestamp, date: java.sql.Date, decimal: java.math.BigDecimal, map: Map[String, Int], nested: Seq[Map[String, Seq[Int]]])
val schema = ScalaReflection.schemaFor[A].dataType.asInstanceOf[StructType]
schema.printTreeString