JSON to Spark

Options

Settings

Mode

Select if you like to read a file or folder.

File/Folder

Enter the input path. The required syntax of a path depends on the connected file system. The node description of the respective connector node describes the required path format. You can also choose a previously selected file from the drop-down list, or select a location from the "Browse..." dialog. Note that browsing is disabled in some cases:

• Browsing is disabled if the connector node hasn't been executed since the workflow has been opened. (Re)execute the connector node to enable browsing.

The location can be exposed as or automatically set via a path flow variable.

Sampling ratio

Infer the type of a collection of JSON records in three stages:

1. Sample given amount of records and infer the type.
2. Merge types by choosing the lowest type necessary to cover equal keys.
3. Replace any remaining null fields with string, the top type.

Primitives

Convert primitives into Strings.

Comments

Allow comments.

Fieldnames

Allow unquoted fieldnames.

Quotes

Feature that determines whether parser will allow use of single quotes (apostrophe, character '\'') for quoting Strings (names and String values).

Numerics

• Allow numeric leading zeros: Feature that determines whether parser will allow JSON integral numbers to start with additional (ignorable) zeroes (like: 000001).
• Allow non numeric numbers: Feature that allows parser to recognize set of "Not-a-Number" (NaN) tokens as legal floating number values (similar to how many other data formats and programming language source code allows it). Supported tokens: INF, -INF, Infinity, -Infinity and NaN.

Views

This node has no views

Workflows

• No workflows found

Links

• No links available

Developers