Icon

Impossible to filter Large Data Sets

Forum Posthttps://forum.knime.com/t/ways-to-speed-this-up-10m-rows/70179Issue: Row Filter with many rows never finishes. Neither CPU, norMemory nor Discs are utilized properly 10k RowsOnlySrtingCol10k perChunk16 Chunksone for each core10k perChunkTest Data Generator Concat with itself is fasterthen generating 10 million rows Column Filter Row Filter Chunk Loop Start Row Filter Loop End ParallelChunk Start Chunk Loop Start Row Filter Loop End Parallel Chunk End Row Filter Forum Posthttps://forum.knime.com/t/ways-to-speed-this-up-10m-rows/70179Issue: Row Filter with many rows never finishes. Neither CPU, norMemory nor Discs are utilized properly 10k RowsOnlySrtingCol10k perChunk16 Chunksone for each core10k perChunkTest Data Generator Concat with itself is fasterthen generating 10 million rows Column Filter Row Filter Chunk Loop Start Row Filter Loop End ParallelChunk Start Chunk Loop Start Row Filter Loop End Parallel Chunk End Row Filter

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: info
Created at: 2023-07-07
On NodePit since: 2023-07-07
Last update: 2024-04-29
Created with KNIME version: v4.7.5

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!