Icon

04_​GoogleCloudExample

<p>Working with Google Cloud Services<br><br>This workflow demonstrates how to connect to various Google Cloud Services such as <em>Google BigQuery, Google Dataproc, and Google Cloud </em>Storage from within KNIME Analytics Platform. <br><br>The output of the <strong>Google Authenticator</strong> node can be used as input for the Google BigQuery Connector node. </p><p>The <strong>Google BigQuery Connector</strong> node provides a DB Connection which can be used with the existing DB nodes to visually assemble queries that are executed within your BigQuery cluster. To upload large amounts of data into the BigQuery cluster use the <strong>DB Loader</strong> node since the JDBC based interface has a lot of restrictions.<br><br>The <strong>Google Cloud Storage Connector</strong> node connects KNIME Analytics Platform with your <em>Google Cloud Storage</em> and allows you to work with your files using the file handling nodes.<br><br>Finally, the <strong>Create Spark Context (Livy)</strong> node can be used to set up a Spark context in your <em>Google Cloud Dataproc</em>. In order to use the node, you need to execute the Apache Livy Initialization Action during cluster creation. For more details see link to the documentation. Once a context is created you can use all the existing Spark nodes to visually assemble your Spark analysis flow.</p>

URL: Google Cloud Dataproc https://cloud.google.com/dataproc/
URL: Apache Livy Initialization Action https://github.com/GoogleCloudPlatform/dataproc-initialization-actions/tree/master/livy
URL: Google BigQuery https://cloud.google.com/bigquery/
URL: Google Cloud Storage https://cloud.google.com/storage/

Nodes

Extensions

Links