Icon

exorbyte_​Intra_​Duplicate_​Discovery

This workflow models a duplicate discovery process within a person data set using exorbyte MatchMaker.

The MM sample index is located in a central exorbyte server and the workflow can be executed directly after download. There is no need for further configuration.

This workflow uses the exorbyte matchmaker server extension, which can be installed from the KNIME Partner Extensions Update Site.

exorbyte Intra-Duplicate DiscoveryWorkflow descriptionThis workflow models a duplicate discovery process within adataset using exorbyte MatchMaker.Workflow requirementsexorbyte matchmaker server extension 2.0.xKNIME Analytics Platform >= 4.6, recommended 4.7 Convert relative paths of the rule files to absolute file paths. Step 1. Read the indexRetieve the index from MatchMaker. Step 2. MatchEach retireved record is used as a query tobe matched against the MM index. Similarrecords are returned as the match results. Step 3. AnnotateAssign the Match Relation, known as thereason why two records are considered amatch. The assignation is done based onthe hit quality of certain fields (Single FieldQuality) and the rule definitions in the rulefiles. Step 4. ClusterFilter the match results based on the Match Relation to include only goodrelations. The records are then grouped into clusters using the MatchRelations. Each cluster has a unique ID, which is added to each record toidentify the cluster they belong to. Read Index from MM serverperform fuzzy matchagainst the indexfrom the MM serverannotate recordswith Match RelationsUsed variables: Relation Rules Definition File:relation_rule_file_pathRelation Levels Definition File:relation_level_file_pathfind intra-duplicatesinclude onlygood Match Relationsrule files' relative pathsto the current workflowrelation_ruleto absolute pathrelation_levelto absolute pathadd CLUSTERIDto retrieved dataIndexReader MM7.x Matcher MM7.x MatchAnnotator DedupClusterer Nominal ValueRow Filter Table Creator Table Rowto Variable URL to FilePath (Variable) URL to FilePath (Variable) Joiner exorbyte Intra-Duplicate DiscoveryWorkflow descriptionThis workflow models a duplicate discovery process within adataset using exorbyte MatchMaker.Workflow requirementsexorbyte matchmaker server extension 2.0.xKNIME Analytics Platform >= 4.6, recommended 4.7 Convert relative paths of the rule files to absolute file paths. Step 1. Read the indexRetieve the index from MatchMaker. Step 2. MatchEach retireved record is used as a query tobe matched against the MM index. Similarrecords are returned as the match results. Step 3. AnnotateAssign the Match Relation, known as thereason why two records are considered amatch. The assignation is done based onthe hit quality of certain fields (Single FieldQuality) and the rule definitions in the rulefiles. Step 4. ClusterFilter the match results based on the Match Relation to include only goodrelations. The records are then grouped into clusters using the MatchRelations. Each cluster has a unique ID, which is added to each record toidentify the cluster they belong to. Read Index from MM serverperform fuzzy matchagainst the indexfrom the MM serverannotate recordswith Match RelationsUsed variables: Relation Rules Definition File:relation_rule_file_pathRelation Levels Definition File:relation_level_file_pathfind intra-duplicatesinclude onlygood Match Relationsrule files' relative pathsto the current workflowrelation_ruleto absolute pathrelation_levelto absolute pathadd CLUSTERIDto retrieved dataIndexReader MM7.x Matcher MM7.x MatchAnnotator DedupClusterer Nominal ValueRow Filter Table Creator Table Rowto Variable URL to FilePath (Variable) URL to FilePath (Variable) Joiner

Nodes

Extensions

Links