Icon

m_​001_​write_​csv_​41

Export huge MS sql table to csv without fetching all records together

The example attached uses ROWIDs from SQLite. You could see if you either have a ROWIDs present or you could create one yourself. You can choose the size of the chunk/batch yourself (I use integers you might have to see if with very large numbers you might need big integers or something. Even a workaround with using strings might be possible).

https://forum.knime.com/t/export-huge-ms-sql-table-to-csv-without-fetching-all-records-together/15105/2?u=mlauber71 result.csvv_batch_sizen_batchesn_batches_fulln_batches_overspilln_batches_loopcurrent_batch_sizev_start_rowv_end_rowmake sureCSV is deleteddelete_this_fileresult.csvdb.sqlitedata1data1select by rowid CSV Writer Data Generator Integer Input Math Formula Math Formula Math Formula Math Formula Counting Loop Start Java Snippet(simple) Math Formula Math Formula Variable Loop End Delete Files Table Creator String to URI CSV Reader SQLite Connector DB Writer DB Query Reader DB Query Reader Table Rowto Variable Table Rowto Variable https://forum.knime.com/t/export-huge-ms-sql-table-to-csv-without-fetching-all-records-together/15105/2?u=mlauber71 result.csvv_batch_sizen_batchesn_batches_fulln_batches_overspilln_batches_loopcurrent_batch_sizev_start_rowv_end_rowmake sureCSV is deleteddelete_this_fileresult.csvdb.sqlitedata1data1select by rowidCSV Writer Data Generator Integer Input Math Formula Math Formula Math Formula Math Formula Counting Loop Start Java Snippet(simple) Math Formula Math Formula Variable Loop End Delete Files Table Creator String to URI CSV Reader SQLite Connector DB Writer DB Query Reader DB Query Reader Table Rowto Variable Table Rowto Variable

Nodes

Extensions

Links