Weka Tutorial 32: Document classification 2 (Application)

Rushdi Shams · 25K views · 2013 · 12:32

Open on YouTube

Machine learning & deep learningData science & statisticstutorial

Why Stack Overflow recommends it

Stack Overflow users link it when answering questions about running text classification in Weka, especially how to apply StringToWordVector before training classifiers like Naive Bayes or SVM.

Recommended by 3 people across 3 different months, 2015 to 2016.

What it is

A 12-minute Weka tutorial by Rushdi Shams showing how to classify text documents using Weka's StringToWordVector attribute filter. It is the second of two application videos on document classification and builds on an earlier video that explains the filter in detail.

What people said

Recommends cgl · 2015-04 · Stack Overflow · on “Weka ,Text Classification on an arff file”

I found the videos below quite helpful when I first got my hands on text classification using Weka. You might want to take a look. Weka Tutorial 31: Document Classification 1 (Application) [LINK] Weka Tutorial 32: Document classification 2 (Application) [this video] WEKA Text Classification for First Time & Beginner Users [LINK] You might want to use StringToWordVector filter to see the effect of each word as an attribute, which is indeed described in detail in the first and last video . Within the filter settings you can give a stopwords list and choose in each run to use it or not. Same with the stemming you can change it as well. This documentation [LINK] and videos will get you to understand

Recommends Rushdi Shams · 2015-11 · Stack Overflow · on “Altrnative of "Weka: training and test set are not compatible"?”

Given the requirements you mentioned, you can use Weka's Filtered Classifier option during training and testing. I am not re-iterating what I have recorded as a video cast here [LINK] and here [this video] . But the basic idea is not to use the StringToWord vector as a direct filter rather to use it as a filtering option in the FilteredClassifier option. The model you generate will be just once. And then you can apply the model directly on your unlabelled data without retraining them or without applying StringToWord vector again on the unlabelled data. FilteredClassifier will take care of these concerns for you.

Recommends Vinicius Woloszyn · 2016-12 · Stack Overflow · on “how to use string data for svm (smo) in weka”

First, you need apply "string to word vector" filter. After, on classify tab, you need to change the target class to "(Nom) class". This is enought to enable the naive bayes and SVM algorithms. I downloaded the dataset, and it worked well. You can follow this tutorial: [this video] Hope it can help you from sklearn.feature_extraction.text import TfidfVectorizer import arff from sklearn import svm import numpy as np from sklearn.model_selection import train_test_split data=list(arff.load('shoor.arff')) text=[] label=[] for r in data: if (len(r)>1): text.append(r[0]) label.append(r[1]) tfidf = TfidfVectorizer().fit_transform(text) features = (tfidf * tfidf.T).A X_train, X_test, y_train, y_test = tr