Back ArrowExplore all topics

How to Remove Stop Words with NLTK in Python

share link

by vigneshchennai74 dot icon Updated: Feb 24, 2023

technology logo
technology logo

Solution Kit Solution Kit  

Stop words are words that do not contain essential meanings and are usually removed from texts. They are words filtered out before or after processing natural language data. Stop words are commonly used words in any language, not just English. Examples of stop words include: a, an, and, the, of, or, in, on, at, etc. 


To remove Stopwords using python: 

  • Firstly, you need to have a list of stop words.  
  • Then you can use the nltk library to tokenize the text and filter out the stop words. 
  • nltk: The Natural Language Toolkit (nltk) is a library in Python that provides tools to work with human language data (text). 
  • Alternatively, you can use the sklearn library to create a list of stop words and filter them out of the text. 
  • Sklearn: This is a machine learning library for Python. It provides a wide range of tools for data preprocessing, classification, regression, clustering, model selection, and dimensionality reduction via a consistent interface. 


Here is how you can remove stop words with NLTK in Python;

Preview of the output that you will get on running this code from your IDE

CODE

In this code we using NLTK library from python to Remove Stop word.

spam filtering: remove stopwords

PythondotLines of Code : 22dotLicense : Strong Copyleft (CC BY-SA 4.0)

Dependent Libraries : 
import nltk
data = [['ham', 'And how you will do that, princess? :)'], ['spam', 'Urgent! Please call 09061213237 from landline. £5000 cash or a luxury 4* Canary Islands Holiday await collection']]

for text in (label_text[1] for label_text in data): 
    filtered_tokens = [token for token in nltk.word_tokenize(text) if token.lower() not in nltk.corpus.stopwords.words('english')]
    print(filtered_tokens)

>>> [',', 'princess', '?', ':', ')']
>>> ['Urgent', '!', 'Please', 'call', '09061213237', 'landline', '.', '£5000', 'cash', 'luxury', '4*', 'Canary', 'Islands', 'Holiday', 'await', 'collection']
  1. Copy the code using "Copy" and paste it in your python ide,
  2. check whether nltk library is added.
  3. Enter the data that need to remove Stopwords
  4. Run the code and get the Output


I hope you found this useful. I have added the link to dependent libraries, version information in the following sections.


I found this code snippet by searching for "Spam filtering: remove Stopwords" in kandi. You can try any such use case!


Note:-



Use this command line by running the following command in your terminal to download punk and stopwords:

  • python -m nltk.downloader punkt
  • python -m nltk.downloader stopwords


or


Use this command line in your ide to dowload punk and stopwords

  • nltk.download('punkt')
  • nltk.download('stopwords')

Environment Tested


I tested this solution in the following versions. Be mindful of changes when working with other versions


  • The solution is created and tested in Vscode version 1.75.1
  • The solution is created and executed in Python version 3.7.15


Using this solution, we are able to remove the stopwords in Python with simple steps. This process also facilities an easy to use, hassle free method to create a hands-on working version of code which would help us remove words in Python..

Dependent Libraries

nltkby nltk

Python doticonstar image 12020 doticonVersion:Currentdoticon
License: Permissive (Apache-2.0)

NLTK Source

Support
    Quality
      Security
        License
          Reuse

            nltkby nltk

            Python doticon star image 12020 doticonVersion:Currentdoticon License: Permissive (Apache-2.0)

            NLTK Source
            Support
              Quality
                Security
                  License
                    Reuse

                      Support

                      1. For any support on kandi solution kits, please use the chat
                      2. For further learning resources, visit the Open Weaver Community learning page.

                      See similar Kits and Libraries

                      Open Weaver – Develop Applications Faster with Open Source

                      kandi

                      Community and Support

                      Company

                      Follow

                      • © 2023 Open Weaver Inc.