What is the function of stemmer in the nltk library of python




In the below python code what is the function of the stemmer:

from nltk.stem.snowball import SnowballStemmer

stemmer = SnowballStemmer('english')

num_train = df_train.shape[0]

    def str_stemmer(s):
    	return " ".join([stemmer.stem(word) for word in s.lower().split()])

#Finding total common words:
def str_common_word(str1, str2):
	return sum(int(str2.find(word)>=0) for word in str1.split())

Also I could not understand what the function str_stemmer is doing,so can someone please help me with it??


Hello @hackers,

For understanding function str_stemmer you need to first know about the stemming in text. Which is nothing but convert a words into their root words, For example: playing, play and played are sharing same root word which is play.

So before doing stemming it lowers the all words in s followed by split and then apply stemming in str_stemmer function.