A Tour of Python NLP Libraries

NLP, or Pure Language Processing, is a subject inside Synthetic Intelligence that focuses on the interplay between human language and computer systems. It tries to discover and apply textual content information so computer systems can perceive the textual content meaningfully.

Because the NLP subject analysis progresses, how we course of textual content information in computer systems has developed. Fashionable occasions, we’ve got used Python to assist discover and course of information simply.

With Python changing into the go-to language for exploring textual content information, many libraries have been developed particularly for the NLP subject. On this article, we’ll discover numerous unimaginable and helpful NLP libraries.

So, let’s get into it.
 

NLTK

 
NLTK, or Pure Language Instrument Equipment, is an NLP Python library with many text-processing APIs and industrial-grade wrappers. It’s one of many largest NLP Python libraries utilized by researchers, information scientists, engineers, and others. It’s a typical NLP Python library for NLP duties.

Let’s attempt to discover what NLTK may do. First, we would want to put in the library with the next code.

Subsequent, we might see what NLTK may do. First, NLTK can carry out the tokenization course of utilizing the next code:

import nltk from nltk.tokenize
import word_tokenize

# Obtain the mandatory sources
nltk.obtain('punkt')

textual content = "The fruit in the table is a banana"
tokens = word_tokenize(textual content)

print(tokens)

Output>> 
['The', 'fruit', 'in', 'the', 'table', 'is', 'a', 'banana']

Tokenization mainly would divide every phrase in a sentence into particular person information.

With NLTK, we will additionally carry out Half-of-Speech (POS) Tags on the textual content pattern.

from nltk.tag import pos_tag

nltk.obtain('averaged_perceptron_tagger')

textual content = "The fruit in the table is a banana"
pos_tags = pos_tag(tokens)

print(pos_tags)

Output>>
[('The', 'DT'), ('fruit', 'NN'), ('in', 'IN'), ('the', 'DT'), ('table', 'NN'), ('is', 'VBZ'), ('a', 'DT'), ('banana', 'NN')]

The output of the POS tagger with NLTK is every token and its meant POS tags. For instance, the phrase Fruit is Noun (NN), and the phrase ‘a’ is Determinant (DT).

It’s additionally attainable to carry out Stemming and Lemmatization with NLTK. Stemming is decreasing a phrase to its base kind by slicing its prefixes and suffixes, whereas Lemmatization additionally transforms to the bottom kind by contemplating the phrases’ POS and morphological evaluation.

from nltk.stem import PorterStemmer, WordNetLemmatizer
nltk.obtain('wordnet')
nltk.obtain('punkt')

textual content = "The striped bats are hanging on their feet for best"
tokens = word_tokenize(textual content)

# Stemming
stemmer = PorterStemmer()
stems = [stemmer.stem(token) for token in tokens]
print("Stems:", stems)

# Lemmatization
lemmatizer = WordNetLemmatizer()
lemmas = [lemmatizer.lemmatize(token) for token in tokens]
print("Lemmas:", lemmas)

Output>> 
Stems: ['the', 'stripe', 'bat', 'are', 'hang', 'on', 'their', 'feet', 'for', 'best']
Lemmas: ['The', 'striped', 'bat', 'are', 'hanging', 'on', 'their', 'foot', 'for', 'best']

You may see that the stemming and lentmatization processes have barely totally different outcomes from the phrases.

That’s the straightforward utilization of NLTK. You may nonetheless do many issues with them, however the above APIs are probably the most generally used.
 

SpaCy

 
SpaCy is an NLP Python library that’s designed particularly for manufacturing use. It’s a complicated library, and SpaCy is thought for its efficiency and skill to deal with massive quantities of textual content information. It’s a preferable library for trade use in lots of NLP instances.

To put in SpaCy, you’ll be able to take a look at their utilization web page. Relying in your necessities, there are a lot of combos to select from.

Let’s attempt utilizing SpaCy for the NLP activity. First, we might attempt performing Named Entity Recognition (NER) with the library. NER is a technique of figuring out and classifying named entities in textual content into predefined classes, reminiscent of individual, deal with, location, and extra.

import spacy

nlp = spacy.load("en_core_web_sm")

textual content = "Brad is working in the U.K. Startup called AIForLife for 7 Months."
doc = nlp(textual content)
#Carry out the NER
for ent in doc.ents:
    print(ent.textual content, ent.label_)

Output>>
Brad PERSON
the U.Okay. Startup ORG
7 Months DATE

As you’ll be able to see, the SpaCy pre-trained mannequin understands which phrase throughout the doc will be labeled.

Subsequent, we will use SpaCy to carry out Dependency Parsing and visualize them. Dependency Parsing is a technique of understanding how every phrase pertains to the opposite by forming a tree construction.

import spacy
from spacy import displacy

nlp = spacy.load("en_core_web_sm")

textual content = "SpaCy excels at dependency parsing."
doc = nlp(textual content)
for token in doc:
    print(f"{token.text}: {token.dep_}, {token.head.text}")

displacy.render(doc, jupyter=True)

Output>> 
Brad: nsubj, working
is: aux, working
working: ROOT, working
in: prep, working
the: det, Startup
U.Okay.: compound, Startup
Startup: pobj, in
referred to as: advcl, working
AIForLife: oprd, referred to as
for: prep, referred to as
7: nummod, Months
Months: pobj, for
.: punct, working

The output ought to embody all of the phrases with their POS and the place they’re associated. The code above would additionally present tree visualization in your Jupyter Pocket book.

Lastly, let’s attempt performing textual content similarity with SpaCy. Textual content similarity measures how comparable or associated two items of textual content are. It has many methods and measurements, however we’ll attempt the only one.

import spacy

nlp = spacy.load("en_core_web_sm")

doc1 = nlp("I like pizza")
doc2 = nlp("I love hamburger")

# Calculate similarity
similarity = doc1.similarity(doc2)
print("Similarity:", similarity)

Output>>
Similarity: 0.6159097609586724

The similarity measure measures the similarity between texts by offering an output rating, normally between 0 and 1. The nearer the rating is to 1, the extra comparable each texts are.

There are nonetheless many issues you are able to do with SpaCy. Discover the documentation to seek out one thing helpful to your work.
 

TextBlob

 
TextBlob is an NLP Python library for processing textual information constructed on prime of NLTK. It simplifies lots of NLTK’s utilization and may streamline textual content processing duties.

You may set up TextBlob utilizing the next code:

pip set up -U textblob
python -m textblob.download_corpora

First, let’s attempt to use TextBlob for NLP duties. The primary one we might attempt is to do sentiment evaluation with TextBlob. We will do this with the code under.

from textblob import TextBlob

textual content = "I am in the top of the world"
blob = TextBlob(textual content)
sentiment = blob.sentiment

print(sentiment)

Output>>
Sentiment(polarity=0.5, subjectivity=0.5)

The output is a polarity and subjectivity rating. Polarity is the sentiment of the textual content the place the rating ranges from -1 (unfavorable) to 1 (optimistic). On the similar time, the subjectivity rating ranges from 0 (goal) to 1 (subjective).

We will additionally use TextBlob for textual content correction duties. You are able to do that with the next code.

from textblob import TextBlob

textual content = "I havv goood speling."
blob = TextBlob(textual content)

# Spelling Correction
corrected_blob = blob.appropriate()
print("Corrected Text:", corrected_blob)

Output>>
Corrected Textual content: I've good spelling.

Attempt to discover the TextBlob packages to seek out the APIs to your textual content duties.
 

Gensim

 
Gensim is an open-source Python NLP library specializing in matter modeling and doc similarity evaluation, particularly for giant and streaming information. It focuses extra on industrial real-time purposes.

Let’s attempt the library. First, we will set up them utilizing the next code:

After the set up is completed, we will attempt the Gensim functionality. Let’s attempt to do matter modeling with LDA utilizing Gensim.

import gensim
from gensim import corpora
from gensim.fashions import LdaModel

# Pattern paperwork
paperwork = [
    "Tennis is my favorite sport to play.",
    "Football is a popular competition in certain country.",
    "There are many athletes currently training for the olympic."
]

# Preprocess paperwork
texts = [[word for word in document.lower().split()] for doc in paperwork]

dictionary = corpora.Dictionary(texts)
corpus = [dictionary.doc2bow(text) for text in texts]


#The LDA mannequin
lda_model = LdaModel(corpus, num_topics=2, id2word=dictionary, passes=15)

matters = lda_model.print_topics()
for matter in matters:
    print(matter)

Output>>
(0, '0.073*"there" + 0.073*"currently" + 0.073*"olympic." + 0.073*"the" + 0.073*"athletes" + 0.073*"for" + 0.073*"training" + 0.073*"many" + 0.073*"are" + 0.025*"is"')
(1, '0.094*"is" + 0.057*"football" + 0.057*"certain" + 0.057*"popular" + 0.057*"a" + 0.057*"competition" + 0.057*"country." + 0.057*"in" + 0.057*"favorite" + 0.057*"tennis"')

The output is a mixture of phrases from the doc samples that cohesively turn into a subject. You may consider whether or not the consequence is smart or not.

Gensim additionally supplies a method for customers to embed content material. For instance, we use Word2Vec to create embedding from phrases.

import gensim
from gensim.fashions import Word2Vec

# Pattern sentences
sentences = [
    ['machine', 'learning'],
    ['deep', 'learning', 'models'],
    ['natural', 'language', 'processing']
]

# Prepare Word2Vec mannequin
mannequin = Word2Vec(sentences, vector_size=20, window=5, min_count=1, employees=4)

vector = mannequin.wv['machine']
print(vector)

Output>>
[ 0.01174188 -0.02259516  0.04194366 -0.04929082  0.0338232   0.01457208
 -0.02466416  0.02199094 -0.00869787  0.03355692  0.04982425 -0.02181222
 -0.00299669 -0.02847819  0.01925411  0.01393313  0.03445538  0.03050548
  0.04769249  0.04636709]

There are nonetheless many purposes you need to use with Gensim. Attempt to see the documentation and consider your wants.
 

Conclusion

On this article, we explored a number of Python NLP libraries important for a lot of textual content duties. All of those libraries could be helpful to your work, from Textual content Tokenization to Phrase Embedding. The libraries we’re discussing are:

1. NLTK
2. SpaCy
3. TextBlob
4. Gensim

I hope it helps
 
 

Cornellius Yudha Wijaya is a knowledge science assistant supervisor and information author. Whereas working full-time at Allianz Indonesia, he likes to share Python and information suggestions by way of social media and writing media. Cornellius writes on a wide range of AI and machine studying matters.