Icon

02_​Train_​A_​NER_​Model

Train a NER Model

This workflow describes the model training process. The first part reads the text corpus created in the first workflow and preprocesses and filters some articles. The middle part shows the model training while the third part is for evaluation.

Preparing dictionary and documents for model training. Validating the model - Opening the view shows three tables:1) Table of entities labeled differently by dictionary tagging and model2) List of new entities3) Table with basic statistics from StanfordNLP Scorer TRAINING: Split data in training and test data. Using 10% of the data as training set.Train the model. Extracting entities Model training process to tag drug names in documents. Split into training and test setTrain modelRemove annotationsTag basedon learned modelUngroup queryGroup train docsGroup test docsRemove docs used for trainingPort 1: Documents and related queriesPort 2: Used dictionary (case insensitive) Documentsfrom01_Creating_A_CorpuslowerCaseWrite list of drugsused to train the modelto data folderWrite extractedentities to data folderWrite BoW for TermCo.-Oc.-Counter to data folderValidation on training setValidation on test set Partitioning StanfordNLPNE Learner Tag Stripper StanfordNLPNE Tagger ModifiableTerm Filter Bag Of WordsCreator Ungroup GroupBy GroupBy ReferenceRow Filter Prepare andfilter documents Table Reader Term To String String Manipulation GroupBy Table Writer Table Writer Table Writer Column Filter Validation Validation Preparing dictionary and documents for model training. Validating the model - Opening the view shows three tables:1) Table of entities labeled differently by dictionary tagging and model2) List of new entities3) Table with basic statistics from StanfordNLP Scorer TRAINING: Split data in training and test data. Using 10% of the data as training set.Train the model. Extracting entities Model training process to tag drug names in documents. Split into training and test setTrain modelRemove annotationsTag basedon learned modelUngroup queryGroup train docsGroup test docsRemove docs used for trainingPort 1: Documents and related queriesPort 2: Used dictionary (case insensitive) Documentsfrom01_Creating_A_CorpuslowerCaseWrite list of drugsused to train the modelto data folderWrite extractedentities to data folderWrite BoW for TermCo.-Oc.-Counter to data folderValidation on training setValidation on test set Partitioning StanfordNLPNE Learner Tag Stripper StanfordNLPNE Tagger ModifiableTerm Filter Bag Of WordsCreator Ungroup GroupBy GroupBy ReferenceRow Filter Prepare andfilter documents Table Reader Term To String String Manipulation GroupBy Table Writer Table Writer Table Writer Column Filter Validation Validation

Nodes

Extensions

Links

Download

To use this workflow in KNIME, download it from the below URL and open it in KNIME:

Download Workflow
Created by: julian
Created at: 2019-01-18
On NodePit since: 2023-10-13
Last update: 2024-11-18
Created with KNIME version: v4.1.0
Tags: preprocessingfiltermodel trainingdrug predictionPubMedLife sciences

Deploy, schedule, execute, and monitor your KNIME workflows locally, in the cloud or on-premises – with our brand new NodePit Runner.

Try NodePit Runner!