Interpretable and Explainable NER With LIME

While a lot of progress has been made to develop the latest greatest, state-of-the art, deep learning models with a gazillion parameters, very little effort has been given to explain the output of these models.

During a workshop in December 2020, Abubakar Abid, CEO of Gradio, examined the way GPT-3 generates text about religions by using the prompt, “Two _ walk into a.” Upon observing the first 10 responses for various religions, he found that GPT-3 mentioned violence once each for Jews, Buddhists, and Sikhs, twice for Christians, but nine out of ten times for Muslims”.

November 14, 2021

Deploying Serverless NER Transformer Model with AWS Lambda

Introduction

With transformers becoming essential for many NLP tasks thanks to their unmatched performance, various useful and impactful NLP models are created every day. However, many NLP practitioners find it challenging to deploy models into production. According to this report, 90% of machine learning models never make it into production.

Model deployment enables you to host your model in a server environment so it can be used to output prediction when called by an API, for example.

September 5, 2021

How to Train a Joint Entities and Relation Extraction Classifier Using BERT Transformer With spaCy 3

Photo by JJ Ying on Unsplash.

Introduction

One of the most useful applications of NLP technology is information extraction from unstructured texts — contracts, financial documents, healthcare records, etc. — that enables automatic data query to derive new insights. Traditionally, named entity recognition has been widely used to identify entities inside a text and store the data for advanced querying and filtering. However, if we want to semantically understand the unstructured text, NER alone is not enough since we don't know how the entities are related to each other. Performing joint NER and relation extraction will open up a whole new way of information retrieval through knowledge graphs where you can navigate across different nodes to discover hidden relationships. Therefore, performing these tasks jointly will be beneficial.

August 9, 2021

How to Fine-Tune BERT Transformer With spaCy v3.0

Since the seminal paper “Attention Is All You Need” of Vaswani et al, transformer models have become by far the state of the art in NLP technology. With applications ranging from NER, text classification, question answering, or text generation, the applications of this amazing technology are limitless.

More specifically, BERT — which stands for Bidirectional Encoder Representations from Transformers — leverages the transformer architecture in a novel way. For example, BERT analyses both sides of the sentence with a randomly masked word to make a prediction. In addition to predicting the masked token, BERT predicts the sequence of the sentences by adding a classification token [CLS] at the beginning of the first sentence and tries to predict if the second sentence follows the first one by adding a separation token [SEP] between the two sentences.