JSON to Spark

This Node Is Deprecated — This version of the node has been replaced with a new and improved version. The old version is kept for backwards-compatibility, but for all new workflows we suggest to use the version linked below.
Go to Suggested ReplacementJSON to Spark
Creates a Spark DataFrame/RDD from given JSON file. See Jackson JSON Parser documentation for more information.

Notice: This feature requires at least Apache Spark 1.5.


Sampling ratio
Infer the type of a collection of JSON records in three stages:
  1. Sample given amount of records and infer the type.
  2. Merge types by choosing the lowest type necessary to cover equal keys.
  3. Replace any remaining null fields with string, the top type.
Convert primitives into Strings.
Allow comments.
Allow unquoted fieldnames.
Feature that determines whether parser will allow use of single quotes (apostrophe, character '\'') for quoting Strings (names and String values).
  • Allow numeric leading zeros: Feature that determines whether parser will allow JSON integral numbers to start with additional (ignorable) zeroes (like: 000001).
  • Allow non numeric numbers: Feature that allows parser to recognize set of "Not-a-Number" (NaN) tokens as legal floating number values (similar to how many other data formats and programming language source code allows it). Supported tokens: INF, -INF, Infinity, -Infinity and NaN.

Input Ports

Spark compatible connection (HDFS, WebHDFS, HttpFS, S3, Blob Storage, ...)
Optional Spark context. If not connected a context is created based on the settings in the Spark preferences page.

Output Ports

Spark DataFrame/RDD

Popular Predecessors

  • No recommendations found

Popular Successors

  • No recommendations found


This node has no views


  • No workflows found



You want to see the source code for this node? Click the following button and we’ll use our super-powers to find it for you.