stop-words | A collection of stop words from around the web | Awesome List library

I am following this tutorial to make a chatbot with the following code.

I am implementing a simple doc2vec with gensim, not a word2vec

I need to remove stopwords without losing the correct order to a list of list.

Each list is a document and, as I understood for doc2vec, the model will have as input a list of TaggedDocuments

model = Doc2Vec(lst_tag_documents, vector_size=5, window=2, min_count=1, workers=4)

I would like to extract a specific group of words from a list of comments scraped from one website to count them and use the most common of them in my TextBlob dictionary, that will be used in a simple sentiment analysis. To simplify: I would like to get all the adjectives, that might have positive or negative sentiment. komentarze is a huge list of strings, every string is a sentence, which sentiment I would like to check. I want to create a list of words from this list of strings and then check which adjectives, that are not punctuation nor stopwords and are before a verb, are the most frequent. When I run my code, I get an error: IndexError: [E040] Attempt to access token at 18, max length 18. This error stands for Attempt to access token at {i}, max length {max_length}. I tried different codes, but none of them works.

Here is an example of a code that wants to proceed, but gives an E040 Error:

I am new in NLP and it is a bit confusing me. I am trying to do a text classification with SVC on my dataset. I have an imbalanced dataset of 6 classes. The text is news for classes of health, sport, culture, economy, science and web. I am using TF-IDF for vectorization.

the preprocessing steps: lower-case all the texts and to remove the stop-words. since my text is in German I did not use lemmatization

my first try:

I'm trying to extract top words by date as follows:

I have list of words like words = ['a', 'spam', 'an', 'eggs', 'the', 'foo', 'and', 'bar'].

And I want to exclude some words (stop-words) defined in another list or set stop_words = ['a', 'an', 'the', 'and'].

What is the fastest way to do that and also keeping the order of original list? I tried to use set() or even SortedSet(). But it still doesn't help, words are still different from original order.

I am very new to NLP. Please, don't judge me strictly.

I have got a very big data-frame on customers' feedback, my goal is to analyze feedbacks. I tokenized words in feedbacks, deleted stop-words (SMART). Now, I need to receive a table of most and less frequent used words.

The code looks like this:

I have been running the TF-IDF Vectorizer from SKLearn but am having trouble recreating the values manually (as an aid to understanding what is happening).

To add some context, i have a list of documents that I have extracted named entities from (in my actual data these go up to 5-grams but here I have restricted this to bigrams). I only want to know the TF-IDF scores for these values and thought passing these terms via the vocabulary parameter would do this.

Here is some dummy data similar to what I am working with:

I am trying to implement the LinearSVC model on a data-set containing 25000 movie reviews. 12,500 are positive labelled reviews and the rest are negative. I am trying to vectorize the data using TfidfVectorizer.

This is my code: