Icon

kn_​example_​python_​date_​time

Check out Date and Datetime variable types between KNIME and Python

Check out Date and Datetime variable types between KNIME and Python
A Jupyter notebook to toy around with data variable is in the foler /data/kn_example_python_date_time.ipynb
Several date and time columns are exported to a parquet file from withi the Python Script node

Check out Date and Datetime variable types between KNIME and PythonA Jupyter notebook to toy around with data variable is in the foler /data/kn_example_python_date_time.ipynbSeveral date and time columns are exported to a parquet file from withi the Python Script node import knime.scripting.io as knioimport pandas as pdimport datetimeimport pyarrow as paimport pyarrow.parquet as pq# input_table_1 = knio.input_tables[0].to_pandas()input_table_1 = knio.input_tables[0].to_pyarrow()ts = pd.Timestamp('2020-03-14T15:32:52.192548')ts_isoformat = ts.isoformat() ts_tuple = ts.timetuple()ts_pytime = ts.to_pydatetime()ts_localize = ts.tz_localize(tz='Europe/Stockholm')ts_from_utc = pd.Timestamp.fromtimestamp(1584199972, 'UTC')ts_tz_utc = pd.Timestamp('2020-03-14T15:32:52.192548651', tz='UTC')ts_tz_asia = ts_tz_utc.tz_convert(tz='Asia/Tokyo')dtt = datetime.datetime(year=2020, month=5, day=3, hour=6, minute=45, second=10, microsecond=348713)dtt_timestamp = 1545730073dtt_object = dtt.fromtimestamp(dtt_timestamp)# ------------------------------df_time = pd.DataFrame( { "ts" : [ts] , "ts_isoformat" : [ts_isoformat] , "ts_tuple" : [ts_tuple] , "ts_pytime" : [ts_pytime] , "ts_localize" : [ts_pytime] , "ts_from_utc" : [ts_from_utc] , "ts_tz_utc" : [ts_tz_utc] , "ts_tz_asia" : [ts_tz_asia] , "dtt" : [dtt] , "dtt_timestamp" : [dtt_timestamp] , "dtt_object" : [dtt_timestamp] } )v_parquet_file = knio.flow_variables['v_path_parquet_file']# import the local parquet file into Python as 'df_time' dataframedf_time.to_parquet(v_parquet_file, compression='gzip', allow_truncated_timestamps=True, coerce_timestamps='ms')knio.output_tables[0] = knio.Table.from_pyarrow(input_table_1)knio.output_tables[1] = knio.Table.from_pandas(df_time) Fails - or does it .... v_path_txt_filev_path_*test_data_all_typeskeep time variables1st Port = just send thru the input table2nd port = try to see the structure of tthe Pandas dfalso /data/df_info.txt1st Port = input table sent thru as PyArrow table2nd port = try to send a time column file "df_time_pyarrow" (PyArrow)or "df_time" (Pandas Dataframe)export data to parquetv_path_*v_path_parquet_file/data/df_time.parquet=> succesful exportof Python date and timevariable to parquet fileflow_variables['var_py_version_pandas'] = pd.__version__flow_variables['var_py_version_numpy'] = np.__version__flow_variables['var_py_version_pyarrow'] = pa.__version__var_py_* collect meta data Java EditVariable (simple) String to Path(Variable) Test Data Generator Column Filter Python Script Python Script Python Script String to Path(Variable) Java EditVariable (simple) Parquet Reader Python Script Variable toTable Row Check out Date and Datetime variable types between KNIME and PythonA Jupyter notebook to toy around with data variable is in the foler /data/kn_example_python_date_time.ipynbSeveral date and time columns are exported to a parquet file from withi the Python Script node import knime.scripting.io as knioimport pandas as pdimport datetimeimport pyarrow as paimport pyarrow.parquet as pq# input_table_1 = knio.input_tables[0].to_pandas()input_table_1 = knio.input_tables[0].to_pyarrow()ts = pd.Timestamp('2020-03-14T15:32:52.192548')ts_isoformat = ts.isoformat() ts_tuple = ts.timetuple()ts_pytime = ts.to_pydatetime()ts_localize = ts.tz_localize(tz='Europe/Stockholm')ts_from_utc = pd.Timestamp.fromtimestamp(1584199972, 'UTC')ts_tz_utc = pd.Timestamp('2020-03-14T15:32:52.192548651', tz='UTC')ts_tz_asia = ts_tz_utc.tz_convert(tz='Asia/Tokyo')dtt = datetime.datetime(year=2020, month=5, day=3, hour=6, minute=45, second=10, microsecond=348713)dtt_timestamp = 1545730073dtt_object = dtt.fromtimestamp(dtt_timestamp)# ------------------------------df_time = pd.DataFrame( { "ts" : [ts] , "ts_isoformat" : [ts_isoformat] , "ts_tuple" : [ts_tuple] , "ts_pytime" : [ts_pytime] , "ts_localize" : [ts_pytime] , "ts_from_utc" : [ts_from_utc] , "ts_tz_utc" : [ts_tz_utc] , "ts_tz_asia" : [ts_tz_asia] , "dtt" : [dtt] , "dtt_timestamp" : [dtt_timestamp] , "dtt_object" : [dtt_timestamp] } )v_parquet_file = knio.flow_variables['v_path_parquet_file']# import the local parquet file into Python as 'df_time' dataframedf_time.to_parquet(v_parquet_file, compression='gzip', allow_truncated_timestamps=True, coerce_timestamps='ms')knio.output_tables[0] = knio.Table.from_pyarrow(input_table_1)knio.output_tables[1] = knio.Table.from_pandas(df_time) Fails - or does it .... v_path_txt_filev_path_*test_data_all_typeskeep time variables1st Port = just send thru the input table2nd port = try to see the structure of tthe Pandas dfalso /data/df_info.txt1st Port = input table sent thru as PyArrow table2nd port = try to send a time column file "df_time_pyarrow" (PyArrow)or "df_time" (Pandas Dataframe)export data to parquetv_path_*v_path_parquet_file/data/df_time.parquet=> succesful exportof Python date and timevariable to parquet fileflow_variables['var_py_version_pandas'] = pd.__version__flow_variables['var_py_version_numpy'] = np.__version__flow_variables['var_py_version_pyarrow'] = pa.__version__var_py_* collect meta data Java EditVariable (simple) String to Path(Variable) Test Data Generator Column Filter Python Script Python Script Python Script String to Path(Variable) Java EditVariable (simple) Parquet Reader Python Script Variable toTable Row

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: m_lauber
Created at: 2022-04-02
On NodePit since: 2022-04-03
Last update: 2026-01-17
Created with KNIME version: v4.7.8
Tags: pythonknimedatetimedatetimeparquetarrowpyarrow

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!