Updated: Oct 20, 2022
1-Click Kit 
NLP text summarizer, is a Python package that summarizes texts and extracts the most important sentences from a given text. Text summarizer is commonly used in news feeding websites to summarize long articles. Summarizer shortens long texts such that the summarized text preserves all the essential points of the actual text. It uses spaCy, nltk, and NumPy to do the job. This solution is also used to summarize texts (in Extractive and abstractive techniques), extract key sentences and find their TF-IDF values. You can use this package for your own projects; we are sure you'll find it useful!
Extraction-based summarization involves selecting sentences from an original document and organizing them into a cohesive summary. In contrast to extraction-based summarization, abstraction-based summaries are created by using algorithms to produce abstracts that can be used as templates.
spaCy is a library for Natural Language Processing (NLP). It provides functions for tokenization, part of speech tagging, and parsing. The library also includes pre-trained models for some languages. NLTK (Natural Language Toolkit) is another popular toolkit for NLP tasks. It is used in many research papers to solve different problems related to NLP.
Deployment Information
Please find the kit solution in this group.
- Download, extract and double-click the kit installer file to install the kit.
- After the successful installation of the kit, press 'Y' to run the kit.
- To run the kit manually, press 'N' and locate the zip file 'Text_Summarizer.zip'
- Extract the zip file and navigate to the directory 'bert-extractive-summarizer-master'
- Open command prompt in the extracted directory 'bert-extractive-summarizer-master' and run the command 'jupyter notebook'
- Locate and open the 'Text_Summarizer.ipynb' notebook from the Jupyter Notebook browser window.
- Execute cells in the notebook
Click on the button below to download the solution and follow the deployment instructions to begin set-up. This 1-click kit has all the required dependencies and resources you may need to build your Text Summarizer in Python.
For a detailed tutorial on installing & executing the solution as well as learning resources including training & certification opportunities, please visit the OpenWeaver Community
Development Environment
VSCode and Jupyter Notebook are used for development and debugging. Jupyter Notebook is a web based interactive environment often used for experiments, whereas VSCode is used to get a typical experience of IDE for developers.
notebookby jupyter
10204 Version:v7.0.0b4Jupyter Interactive Notebook
notebookby jupyter
Jupyter Notebook 10204 Version:v7.0.0b4 License: Permissive (BSD-3-Clause)
Exploratory Data Analysis
For extensive analysis and exploration of data, and to deal with arrays, these libraries are used. They are also used for performing scientific computation and data manipulation.
numpyby numpy
23755 Version:v1.25.0rc1The fundamental package for scientific computing with Python.
numpyby numpy
Python 23755 Version:v1.25.0rc1 License: Permissive (BSD-3-Clause)
pandasby pandas-dev
38689 Version:v2.0.2Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more
pandasby pandas-dev
Python 38689 Version:v2.0.2 License: Permissive (BSD-3-Clause)
Text Mining
Libraries in this group are used for analysis and processing of unstructured natural language.
spaCyby explosion
26383 Version:v3.2.6💫 Industrial-strength Natural Language Processing (NLP) in Python
spaCyby explosion
Python 26383 Version:v3.2.6 License: Permissive (MIT)
sentencepieceby google
7616 Version:v0.1.99Unsupervised text tokenizer for Neural Network-based text generation.
sentencepieceby google
C++ 7616 Version:v0.1.99 License: Permissive (Apache-2.0)
Machine Learning & Natural Language Processing
The library offers state-of-the-art pre-trained models for Natural Language Processing (NLP).
scikit-learnby scikit-learn
54584 Version:1.2.2scikit-learn: machine learning in Python
scikit-learnby scikit-learn
Python 54584 Version:1.2.2 License: Permissive (BSD-3-Clause)
pytorchby pytorch
67874 Version:v2.0.1Tensors and Dynamic neural networks in Python with strong GPU acceleration
pytorchby pytorch
Python 67874 Version:v2.0.1 License: Others (Non-SPDX)
transformersby huggingface
104111 Version:v4.30.2🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
transformersby huggingface
Python 104111 Version:v4.30.2 License: Permissive (Apache-2.0)
sentence-transformersby UKPLab
10938 Version:v2.2.2Multilingual Sentence & Image Embeddings with BERT
sentence-transformersby UKPLab
Python 10938 Version:v2.2.2 License: Permissive (Apache-2.0)
Utilities
library tqdm can be used to show progress bar for any long running process step in the code
tqdmby tqdm
25025 Version:v4.65.0A Fast, Extensible Progress Bar for Python and CLI
tqdmby tqdm
Python 25025 Version:v4.65.0 License: Others (Non-SPDX)
Testing
The libraries listed here can be used for unit testing as well as integration testing
pytestby pytest-dev
10300 Version:7.3.2The pytest framework makes it easy to write small tests, yet scales to support complex functional testing
pytestby pytest-dev
Python 10300 Version:7.3.2 License: Permissive (MIT)
Kit Solution Source
bert-extractive-summarizerby dmmiller612
1206 Version:0.10.1Easy to use extractive text summarization with BERT
bert-extractive-summarizerby dmmiller612
Python 1206 Version:0.10.1 License: Permissive (MIT)
Support
If you need help using this kit, you may reach us at the OpenWeaver Community.