How to Remove a word in a Span from SpaCy 

by vigneshchennai74 Updated: Jan 31, 2023

We will locate a specific group of words in a text using the SpaCy library, then replace those words with an empty string to remove them from the text. 

Using SpaCy, it is possible to exclude words within a specific span from a text in the following ways: 

Text pre-processing: Removing specific words or phrases from text can be a useful step in pre-processing text data for NLP tasks such as text classification, sentiment analysis, and language translation. 
Document summarization: Maintaining only the most crucial information, specific words or phrases will serve to construct a summary of a lengthy text. 
Data cleaning: Anonymization and data cleaning can both benefit from removing sensitive or useless text information, such as names and addresses. 
Text generation: Adding context or meaning to the generated content might help create new text by deleting specific words or phrases. 
Text augmentation: Text can be used for text augmentation techniques in NLP by removing specific words or phrases and replacing them with new text variations.

Here is how you can remove words in span using SpaCy: 

Preview of the output that you will get on running this code from your IDE

Code

In this solution we have used spacy library of python

Remove a word in a span from SpaCy?

PythonLines of Code : 18License : CC BY-SA 4.0

Dependent Libraries :

import spacy
from spacy.attrs import LOWER, POS, ENT_TYPE, IS_ALPHA
from spacy.tokens import Doc
import numpy

def remove_span(doc, index):
    np_array = doc.to_array([LOWER, POS, ENT_TYPE, IS_ALPHA])
    np_array_2 = numpy.delete(np_array, (index), axis = 0)
    doc2 = Doc(doc.vocab, words=[t.text for i, t in enumerate(doc) if i!=index])
    doc2.from_array([LOWER, POS, ENT_TYPE, IS_ALPHA], np_array_2)
    return doc2

# load english model
nlp = spacy.load('en')
doc = nlp("This is some text")
new_doc = remove_span(doc, 3)
print(new_doc)

Copy the code using the "Copy" button above, and paste it in a Python file in your IDE.
Enter the Text
Run the code that Remove Specific words in the text

I hope you found this useful. I have added the link to dependent libraries, version information in the following sections.

I found this code snippet by searching for "Remove words in span from spacy" in kandi. You can try any such use case!

Note

In this snippet we are using a Language model (en_core_web_sm)

Download the model using the command python -m spacy download en_core_web_sm .
paste it in your terminal and download it.

Check the user's spacy version using pip show spacy command in users terminal.

if its version 3.0, you will need to load it using nlp = spacy.load("en_core_web_sm")
if its version is less than 3.0 you will need to load it using nlp = spacy.load("en")

Environment Tested

I tested this solution in the following versions. Be mindful of changes when working with other versions.

The solution is created in Python 3.7.15 Version
The solution is tested on Spacy 3.4.3 Version

Using this solution, we can collect nouns that ends with s-t-l with the help of function in spacy . This process also facilities an easy to use, hassle free method to create a hands-on working version of code which would help us use full stop whenever the user needs in the sentence in python.

Dependent Library

spaCyby explosion

Python 25129 Version:3.4.4

License: Permissive (MIT)

💫 Industrial-strength Natural Language Processing (NLP) in Python

Support

Quality

Security

License

Reuse

spaCyby explosion

Python 25129 Version:3.4.4 License: Permissive (MIT)

💫 Industrial-strength Natural Language Processing (NLP) in Python

Support

Quality

Security

License

Reuse

numpyby numpy

Python 22606 Version:1.24.1

License: Permissive (BSD-3-Clause)

The fundamental package for scientific computing with Python.

Support

Quality

Security

License

Reuse

numpyby numpy

Python 22606 Version:1.24.1 License: Permissive (BSD-3-Clause)

The fundamental package for scientific computing with Python.

Support

Quality

Security

License

Reuse

If you do not have SpaCy and numpy that is required to run this code, you can install it by clicking on the above link and copying the pip Install command from the Spacy page in kandi.

You can search for any dependent library on kandi like SpaCy and numpy