Back ArrowExplore all kits

Lemmatization using Spacy

share link

by vigneshchennai74 dot icon Updated: Jan 5, 2023

technology logo
technology logo

Solution Kit Solution Kit  

In this solution we are going to learn how to lemmatize words using Spacy . Lemmatization is the process of grouping together the different inflected forms of a word so they can be analyzed as a single item. Lemmatization is similar to stemming but it brings context to the words. So it links words with similar meanings to one word. In this solution kit, I am sharing the code snippet and library that Lemmatize words using Spacy in Python which can be executed directly in the IDE.


Preview of the output that you will get on running this code from your IDE

Code

In this solution we have used Spacy library.

How can I solve an attribute error when using spacy?

PythondotLines of Code : 25dotLicense : Strong Copyleft (CC BY-SA 4.0)

Dependent Libraries : 
import re
import spacy

# using the web English model for practicality here
nlp = spacy.load('en_core_web_sm')

tex = ['I\'m going to get a cat tomorrow',
        'I don\'t know if I\'ll be able to get him a cat house though!']

data = [re.sub(r"\"", "", i) for i in tex]
data1 = [re.sub(r"\“", "", i) for i in data]
data2 = [re.sub(r"\„", "", i) for i in data1]

spacy_doc1 = []

for line in data2:
    spac = nlp(line)
    lem = [tok.lemma_ for tok in spac]
    no_punct = [tok for tok in lem if re.match('\w+', tok) or tok in ["-PRON-"]]
    no_numbers = [tok for tok in no_punct if not re.match('\d+', tok)]
    print(no_numbers)

# > ['-PRON-', 'be', 'go', 'to', 'get', 'a', 'cat', 'tomorrow']
# > ['-PRON-', 'do', 'not', 'know', 'if', '-PRON-', 'will', 'be', 'able', 'to', 'get', '-PRON-', 'a', 'cat', 'house', 'though']
  1. Copy the code using the "Copy" button above, and paste it in a Python file in your IDE.
  2. Run the program to get the text to lemmatize


I hope you found this useful. I have added the link to dependent libraries, version information in the following sections.


I found this code snippet by searching for "How can i solve an attribute error when using SpaCy " in kandi. You can try any such use case!

Dependent Library

spaCyby explosion

Python doticonstar image 26383 doticonVersion:v3.2.6doticon
License: Permissive (MIT)

💫 Industrial-strength Natural Language Processing (NLP) in Python

Support
    Quality
      Security
        License
          Reuse

            spaCyby explosion

            Python doticon star image 26383 doticonVersion:v3.2.6doticon License: Permissive (MIT)

            💫 Industrial-strength Natural Language Processing (NLP) in Python
            Support
              Quality
                Security
                  License
                    Reuse

                      If you do not have SpaCy that is required to run this code, you can install it by clicking on the above link and copying the pip Install command from the Spacy page in kandi.

                      You can search for any dependent library on kandi like Spacy

                      Environment Test

                      I tested this solution in the following versions. Be mindful of changes when working with other versions.


                      1. The solution is created in Python 3.7.15 Version
                      2. The solution is tested on Spacy 3.4.3 Version


                      Using this solution, we are going to Lemmatize the words with help of Spacy library. This process also facilities an easy to use, hassle free method to create a hands-on working version of code which would help us Lemmatize the words in Python.

                      Support

                      • For any support on kandi solution kits, please use the chat
                      • For further learning resources, visit the Open Weaver Community learning page.


                      See similar Kits and Libraries

                      Open Weaver – Develop Applications Faster with Open Source

                      kandi

                      Community and Support

                      Company

                      Follow

                      • © 2023 Open Weaver Inc.