Firing a cannon at sparrows: BERT vs. logreg

There is a Golden Rule in NLP, at least when it comes to classification tasks: “Always start with a tfidf-logreg baseline”. Elaborating a bit, that’s building a logistic regression model on top of tf-idf (term frequency-inverse document frequency) text representation. This typically works fairly well, is simple to deploy as opposed to neural nets and that’s what already deployed and working day and night while you are struggling with fancy transformers. In this presentation, we will go through a couple of