Icon

m_​001_​write_​csv

Export huge MS sql table to csv without fetching all records together

The example attached uses ROWIDs from SQLite. You could see if you either have a ROWIDs present or you could create one yourself. You can choose the size of the chunk/batch yourself (I use integers you might have to see if with very large numbers you might need big integers or something. Even a workaround with using strings might be possible).

https://forum.knime.com/t/export-huge-ms-sql-table-to-csv-without-fetching-all-records-together/15105/2?u=mlauber71 result.csvdb.sqlitedata1data1v_batch_sizen_batchesn_batches_fulln_batches_overspilln_batches_loopcurrent_batch_sizev_start_rowv_end_rowselect by rowidmake sureCSV is deleteddelete_this_fileresult.csv CSV Writer Data Generator SQLite Connector(legacy) Database Writer(legacy) Database Reader(legacy) Integer Input Math Formula Math Formula Math Formula Math Formula Table Row to Variable(deprecated) Counting Loop Start Java Snippet(simple) Table Row to Variable(deprecated) Math Formula Math Formula Variable Loop End Database Reader(legacy) Delete Files Table Creator String to URI CSV Reader https://forum.knime.com/t/export-huge-ms-sql-table-to-csv-without-fetching-all-records-together/15105/2?u=mlauber71 result.csvdb.sqlitedata1data1v_batch_sizen_batchesn_batches_fulln_batches_overspilln_batches_loopcurrent_batch_sizev_start_rowv_end_rowselect by rowidmake sureCSV is deleteddelete_this_fileresult.csvCSV Writer Data Generator SQLite Connector(legacy) Database Writer(legacy) Database Reader(legacy) Integer Input Math Formula Math Formula Math Formula Math Formula Table Row to Variable(deprecated) Counting Loop Start Java Snippet(simple) Table Row to Variable(deprecated) Math Formula Math Formula Variable Loop End Database Reader(legacy) Delete Files Table Creator String to URI CSV Reader

Nodes

Extensions

Links