spaCy | 💫 Industrial-strength Natural Language Processing | Natural Language Processing library

 by   explosion Python Version: 4.0.0.dev1 License: MIT

kandi X-RAY | spaCy Summary

kandi X-RAY | spaCy Summary

spaCy is a Python library typically used in Artificial Intelligence, Natural Language Processing, Deep Learning, Pytorch, Bert applications. spaCy has no vulnerabilities, it has build file available, it has a Permissive License and it has medium support. However spaCy has 8 bugs. You can install using 'pip install spaCy' or download it from GitHub, PyPI.

spaCy is a library for advanced Natural Language Processing in Python and Cython. It's built on the very latest research, and was designed from day one to be used in real products. spaCy comes with pretrained pipelines and currently supports tokenization and training for 60+ languages. It features state-of-the-art speed and neural network models for tagging, parsing, named entity recognition, text classification and more, multi-task learning with pretrained transformers like BERT, as well as a production-ready training system and easy model packaging, deployment and workflow management. spaCy is commercial open-source software, released under the MIT license.

            kandi-support Support

              spaCy has a medium active ecosystem.
              It has 26383 star(s) with 4147 fork(s). There are 553 watchers for this library.
              There were 9 major release(s) in the last 6 months.
              There are 75 open issues and 5348 have been closed. On average issues are closed in 56 days. There are 39 open pull requests and 0 closed requests.
              It has a neutral sentiment in the developer community.
              The latest version of spaCy is 4.0.0.dev1

            kandi-Quality Quality

              spaCy has 8 bugs (4 blocker, 0 critical, 3 major, 1 minor) and 1001 code smells.

            kandi-Security Security

              spaCy has no vulnerabilities reported, and its dependent libraries have no vulnerabilities reported.
              spaCy code analysis shows 0 unresolved vulnerabilities.
              There are 93 security hotspots that need review.

            kandi-License License

              spaCy is licensed under the MIT License. This license is Permissive.
              Permissive licenses have the least restrictions, and you can use them in most projects.

            kandi-Reuse Reuse

              spaCy releases are available to install and integrate.
              Deployable package is available in PyPI.
              Build file is available. You can build the component from source.
              Installation instructions, examples and code snippets are available.

            Top functions reviewed by kandi - BETA

            kandi has reviewed spaCy and discovered the below as its top functions. This is intended to give you an instant insight into spaCy implemented functionality, and help decide if they suit your requirements.
            • Defines a factory
            • Returns a fully qualified name for the given language
            • Sets the factory meta
            • Register a factory function
            • Compute the PRF score for the given examples
            • Calculate the tp score
            • Embed a character embedding
            • Construct a model of static vectors
            • Lemmatize a token
            • Get a table by name
            • Command line interface for debugging
            • Parse dependencies
            • Process if node
            • Create a model of static vectors
            • Lemmatize a word
            • Parse command line interface
            • Lemmatize rule
            • Setup package
            • Forward layer computation
            • Lemmatize a specific word
            • Update the model with the given examples
            • Builds a token embedding model
            • Command line interface for pretraining
            • Extract the words from the wiktionary
            • Rehearse the language
            • Process a for loop
            • Lemmatize a rule
            Get all kandi verified functions for this library.

            spaCy Key Features

            No Key Features are available at this moment for spaCy.

            spaCy Examples and Code Snippets

            Download the dataset and install spaCy and pandas-Import pre-annotated data
            Pythondot img1Lines of Code : 69dot img1License : Permissive (Apache-2.0)
            copy iconCopy
            import spacy
            import pandas as pd
            import json
            from itertools import groupby
            # Download spaCy models:
            models = {
                'en_core_web_sm': spacy.load("en_core_web_sm"),
                'en_core_web_lg': spacy.load("en_core_web_lg")
            # This function converts spaCy d  
            Compare the spaCy model with the gold standard dataset
            Pythondot img2Lines of Code : 19dot img2License : Permissive (Apache-2.0)
            copy iconCopy
            import json
            from collections import defaultdict
            tasks = json.load(open('annotations.json'))
            model_hits = defaultdict(int)
            for task in tasks:
                annotation_result = task['annotations'][0]['result']
                for r in annotation_result:
            Download the dataset and install spaCy and pandas-Install spaCy and pandas
            Pythondot img3Lines of Code : 3dot img3License : Permissive (Apache-2.0)
            copy iconCopy
            python -m pip install -U pip
            pip install -U spacy
            pip install pandas
            Get a word's function in a sentence PY
            Pythondot img4Lines of Code : 15dot img4License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            import spacy
            nlp = spacy.load('en_core_web_sm')
            # Process the document
            doc = nlp('God loves apples.')
            for tok in doc:
                print(tok, tok.dep_, sep='\t')
            God nsubj
            loves   ROOT
            apples  dobj
            .   punct
            How to solve the spacy latin language import error
            Pythondot img5Lines of Code : 2dot img5License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            nlp = spacy_stanza.load_pipeline("xx", lang="la")
            copy iconCopy
            import spacy
            from spacy.scorer import Scorer
            from spacy.tokens import Doc
            from import Example
            examples = [
                ('Who is Talha Tayyab?',
                 {(7, 19, 'PERSON')}),
                ('I like London and Berlin.',
                 {(7, 13, 'LOC
            Tweek spacy spans
            Pythondot img7Lines of Code : 30dot img7License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            import spacy
            from spacy.matcher import Matcher
            nlp = spacy.load("en_core_web_sm")
            matcher = Matcher(nlp.vocab)
            # Add match ID "HelloWorld" with no callback and one pattern
            pattern = [{"LOWER": "hello"}, {"IS_PUNCT": True}, {"LOWER": "worl
            How to label multi-word entities?
            Pythondot img8Lines of Code : 19dot img8License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            FINANCE = ["Frontwave Credit Union",
            "St. Mary's Bank",
            "Center for Financial Services Innovation"]
            SPORT = [
                "Christiano Ronaldo",
                "Lewis Hamilton",
            FINANCE = '|'.join(FINANCE)
            sent = pd.DataFrame({'sent': ["Dear members of Fr
            Value Error when trying to train a spacy model
            Pythondot img9Lines of Code : 3dot img9License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
                        for batch in batches:
                            nlp.update(batch, sgd=optimizer, losses=losses)
            AWS Lambda function not able to find other packages in same directory
            Pythondot img10Lines of Code : 4dot img10License : Strong Copyleft (CC BY-SA 4.0)
            copy iconCopy
            COPY core ${LAMBDA_TASK_ROOT}
            COPY core ${LAMBDA_TASK_ROOT}/core

            Community Discussions


            Error while loading vector from Glove in Spacy
            Asked 2022-Mar-17 at 16:39

            I am facing the following attribute error when loading glove model:

            Code used to load model:



            Answered 2022-Mar-17 at 14:08

            spacy version: 3.1.4 does not have the feature from_glove.

            I was able to use nlp.vocab.vectors.from_glove() in spacy version: 2.2.4.

            If you want, you can change your spacy version by using:

            !pip install spacy==2.2.4 on your Jupyter cell.



            Save and load nlp results in spacy
            Asked 2022-Mar-10 at 20:36

            I want to use SpaCy to analyze many small texts and I want to store the nlp results for further use to save processing time. I found code at Storing and Loading spaCy Documents Containing Word Vectors but I get an error and I cannot find how to fix it. I am fairly new to python.

            In the following code, I store the nlp results to a file and try to read it again. I can write the first file but I do not find the second file (vocab). I also get two errors: that Doc and Vocab are not defined.

            Any idea to fix this or another method to achieve the same result is more than welcomed.




            Answered 2022-Mar-10 at 18:06

            I tried your code and I had a few minor issues wgich I fixed on the code below.

            Note that SaveTest.nlp is a binary file with your doc info and
            SaveTest.voc is a folder with all the spacy model vocab information (vectors, strings among other).

            Changes I made:

            1. Import Doc class from spacy.tokens
            2. Import Vocab class from spacy.vocab
            3. Download en_core_web_md model using the following command:



            How to match repeating patterns in spacy?
            Asked 2022-Mar-09 at 04:14

            I have a similar question as the one asked in this post: How to define a repeating pattern consisting of multiple tokens in spacy? The difference in my case compared to the linked post is that my pattern is defined by POS and dependency tags. As a consequence I don't think I could easily use regex to solve my problem (as is suggested in the accepted answer of the linked post).

            For example, let's assume we analyze the following sentence:

            "She told me that her dog was big, black and strong."

            The following code would allow me to match the list of adjectives at the end of the sentence:



            Answered 2022-Mar-09 at 04:14

            The solution / issue isn't fundamentally different from the question linked to, there's no facility for repeating multi-token patterns in a match like that. You can use a for loop to build multiple patterns to capture what you want.



            Spacy adds words automatically to vocab?
            Asked 2022-Feb-28 at 04:26

            I loaded regular spacy language, and tries the following code:



            Answered 2022-Feb-28 at 04:26

            The spaCy Vocab is mainly an internal implementation detail to interface with a memory-efficient method of storing strings. It is definitely not a list of "real words" or any other thing that you are likely to find useful.

            The main thing a Vocab stores by default is strings that are used internally, such as POS and dependency labels. In pipelines with vectors, words in the vectors are also included. You can read more about the implementation details here.

            All words an nlp object has seen need storage for their strings, and so will be present in the Vocab. That's what you're seeing with your nonsense string in the example above.



            Spacy tokenization add extra white space for dates with hyphen separator when I manually build the Doc
            Asked 2022-Feb-15 at 12:39

            I've been trying to solve a problem with the spacy Tokenizer for a while, without any success. Also, I'm not sure if it's a problem with the tokenizer or some other part of the pipeline.

            Any help is welcome!


            I have an application that for reasons besides the point, creates a spacy Doc from the spacy vocab and the list of tokens from a string (see code below). Note that while this is not the simplest and most common way to do this, according to spacy doc this can be done.

            However, when I create a Doc for a text that contains compound words or dates with hyphen as a separator, the behavior I am getting is not what I expected.



            Answered 2022-Feb-14 at 21:06


            After installing scrubadub_spacy package, spacy.load("en_core_web_sm") not working OSError: [E053] Could not read config.cfg
            Asked 2022-Feb-06 at 04:46

            I am getting the below error when I'm trying to run the following line of code to load en_core_web_sm in the Azure Machine Learning instance.

            I debugged the issue and found out that once I install scrubadub_spacy, that seems is the issue causing the error.



            Answered 2022-Feb-06 at 04:46

            Taking the path from your error message:



            Show NER Spacy Data in dataframe
            Asked 2022-Jan-25 at 21:27

            I am doing some web scraping to export text info from an html and using a NER (Spacy) to identify information such as Assets Under Management, Addresses, and founding dates of companies. Once the information is extracted, I would like to place it in a dataframe.

            I am working with the following script:



            Answered 2022-Jan-25 at 21:27

            After you obtained the body with plain text, you can parse the text into a document and get a list of all entities with their labels and texts, and then instantiate a Pandas dataframe with those data:



            How to get a description for each Spacy NER entity?
            Asked 2022-Jan-24 at 16:01

            I am using Spacy NER model to extract from a text, some named entities relevant to my problem, such us DATE, TIME, GPE among others.

            For example, I need to recognize the Time Zone in the following sentence:



            Answered 2022-Jan-24 at 16:01

            Most labels have definitions you can access using spacy.explain(label).

            For NORP: "Nationalities or religious or political groups"

            For more details you would need to look into the annotation guidelines for the resources listed in the model documentation under



            Do I need to do any text cleaning for Spacy NER?
            Asked 2021-Dec-28 at 11:42

            I am new to NER and Spacy. Trying to figure out what, if any, text cleaning needs to be done. Seems like some examples I've found trim the leading and trailing whitespace and then muck with the start/stop indexes. I saw one example where the guy did a bunch of cleaning and his accuracy was really bad because all the indexes were messed up.

            Just to clarify, the dataset was annotated with DataTurks, so you get json like this:



            Answered 2021-Dec-28 at 05:19

            First, spaCy does no transformation of the input - it takes it literally as-is and preserves the format. So you don't lose any information when you provide text to spaCy.

            That said, input to spaCy with the pretrained pipelines will work best if it is in natural sentences with no weird punctuation, like a newspaper article, because that's what spaCy's training data looks like.

            To that end, you should remove meaningless white space (like newlines, leading and trailing spaces) or formatting characters (maybe a line of ----?), but that's about all the cleanup you have to do. The spaCy training data won't have bullets, so they might get some weird results, but I would leave them in to start. (Also, bullets are obviously printable characters - maybe you mean non-ASCII?)

            I have no idea what you mean by "muck with the indexes", but for some older NLP methods it was common to do more extensive preprocessing, like removing stop words and lowercasing everything. Doing that will make things worse with spaCy because it uses the information you are removing for clues, just like a human reader would.

            Note that you can train your own models, in which case they'll learn about the kind of text you show them. In that case you can get rid of preprocessing entirely, though for actually meaningless things like newlines / leading and following spaces you might as well remove them anyway.

            To address your new info briefly...

            Yes, character indexes for NER labels must be updated if you do preprocessing. If they aren't updated they aren't usable.

            It looks like you're trying to extract "skills" from a resume. That has many bullet point lists. The spaCy training data is newspaper articles, which don't contain any lists like that, so it's hard to say what the right thing to do is. I don't think the bullets matter much, but you can try removing or not removing them.

            What about stuff like lowercasing, stop words, lemmatizing, etc?

            I already addressed this, but do not do this. This was historically common practice for NLP models, but for modern neural models, including spaCy, it is actively unhelpful.



            How to use japanese engine in Spacy
            Asked 2021-Dec-20 at 04:54

            I am building a NLP App using python. I heard the Spacy is proper to NLP and installed it. How should I use the Japanese engine from Spacy?



            Answered 2021-Dec-15 at 21:39

            You should download and install the language package.


            Community Discussions, Code Snippets contain sources that include Stack Exchange Network


            No vulnerabilities reported

            Install spaCy

            For detailed installation instructions, see the documentation.
            Operating system: macOS / OS X · Linux · Windows (Cygwin, MinGW, Visual Studio)
            Python version: Python 3.6+ (only 64 bit)
            Package managers: pip · conda (via conda-forge)
            Trained pipelines for spaCy can be installed as Python packages. This means that they're a component of your application, just like any other module. Models can be installed using spaCy's download command, or manually by pointing pip to a path or URL.


            New to spaCy? Here's everything you need to know!. How to use spaCy and its features. 🚀 New in v3.0. New features, backwards incompatibilities and migration guide. End-to-end workflows you can clone, modify and run. The detailed reference for spaCy's API. Download trained pipelines for spaCy. Plugins, extensions, demos and books from the spaCy ecosystem. Learn spaCy in this free and interactive online course. Our YouTube channel with video tutorials, talks and more. Changes and version history. How to contribute to the spaCy project and code base. Get a custom spaCy pipeline, tailor-made for your NLP problem by spaCy's core developers. Streamlined, production-ready, predictable and maintainable. Start by completing our 5-minute questionnaire to tell us what you need and we'll be in touch! Learn more →.
            Find more information at:

            Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items

            Find more libraries
          • PyPI

            pip install spacy

          • CLONE
          • HTTPS


          • CLI

            gh repo clone explosion/spaCy

          • sshUrl


          • Stay Updated

            Subscribe to our newsletter for trending solutions and developer bootcamps

            Agree to Sign up and Terms & Conditions

            Share this Page

            share link