Icon

kn_​forum_​70179_​10m_​rows_​filter

Issue: Row Filter with many rows never finishes.

Issue: Row Filter with many rows never finishes.

Forum Posthttps://forum.knime.com/t/ways-to-speed-this-up-10m-rows/70179Issue: Row Filter with many rows never finishes. Neither CPU, nor Memory nor Discsare utilized properly 10k RowsOnlySrtingColNode 1910k perChunkNode 21Node 2216 Chunksone for each core10k perChunkNode 25Node 26Node 27Node 30cache to diskNode 34Date&Time1Date&Time2durationRow FilterNode 3803Date&Time1Date&Time2String onlydurationNode 3808Date&Time1Date&Time210k per ChunkdurationNode 3813Date&Time1Date&Time210k per ChunkdurationNode 3818Node 3819spped_result.xlsxNode 3821Node 3822big_file.table Test Data Generator Concat with itself is fasterthen generating 10 million rows Column Filter Row Filter Chunk Loop Start Row Filter Loop End ParallelChunk Start Chunk Loop Start Row Filter Loop End Parallel Chunk End Row Filter Cache Column Filter Create Date&TimeRange Create Date&TimeRange Date&TimeDifference ConstantValue Column Joiner Create Date&TimeRange Create Date&TimeRange ConstantValue Column Date&TimeDifference Joiner Create Date&TimeRange Create Date&TimeRange ConstantValue Column Date&TimeDifference Joiner Create Date&TimeRange Create Date&TimeRange ConstantValue Column Date&TimeDifference Joiner Concatenate Excel Writer Don't Save Start Don't Save End Table Writer Forum Posthttps://forum.knime.com/t/ways-to-speed-this-up-10m-rows/70179Issue: Row Filter with many rows never finishes. Neither CPU, nor Memory nor Discsare utilized properly 10k RowsOnlySrtingColNode 1910k perChunkNode 21Node 2216 Chunksone for each core10k perChunkNode 25Node 26Node 27Node 30cache to diskNode 34Date&Time1Date&Time2durationRow FilterNode 3803Date&Time1Date&Time2String onlydurationNode 3808Date&Time1Date&Time210k per ChunkdurationNode 3813Date&Time1Date&Time210k per ChunkdurationNode 3818Node 3819spped_result.xlsxNode 3821Node 3822big_file.tableTest Data Generator Concat with itself is fasterthen generating 10 million rows Column Filter Row Filter Chunk Loop Start Row Filter Loop End ParallelChunk Start Chunk Loop Start Row Filter Loop End Parallel Chunk End Row Filter Cache Column Filter Create Date&TimeRange Create Date&TimeRange Date&TimeDifference ConstantValue Column Joiner Create Date&TimeRange Create Date&TimeRange ConstantValue Column Date&TimeDifference Joiner Create Date&TimeRange Create Date&TimeRange ConstantValue Column Date&TimeDifference Joiner Create Date&TimeRange Create Date&TimeRange ConstantValue Column Date&TimeDifference Joiner Concatenate Excel Writer Don't Save Start Don't Save End Table Writer

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: info
Created at: 2023-07-07
On NodePit since: 2023-07-14
Last update: 2024-04-26
Created with KNIME version: v4.7.5

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!