DPP_CNN_Summarization | Pytorch code for DivCNN | Machine Learning library
kandi X-RAY | DPP_CNN_Summarization Summary
kandi X-RAY | DPP_CNN_Summarization Summary
Pytorch code for DivCNN
Support
Quality
Security
License
Reuse
Top functions reviewed by kandi - BETA
Currently covering the most popular Java, JavaScript and Python libraries. See a Sample of DPP_CNN_Summarization
DPP_CNN_Summarization Key Features
DPP_CNN_Summarization Examples and Code Snippets
Community Discussions
Trending Discussions on Machine Learning
QUESTION
I have trained an RNN model with pytorch. I need to use the model for prediction in an environment where I'm unable to install pytorch because of some strange dependency issue with glibc. However, I can install numpy and scipy and other libraries. So, I want to use the trained model, with the network definition, without pytorch.
I have the weights of the model as I save the model with its state dict and weights in the standard way, but I can also save it using just json/pickle files or similar.
I also have the network definition, which depends on pytorch in a number of ways. This is my RNN network definition.
...ANSWER
Answered 2022-Feb-17 at 10:47You should try to export the model using torch.onnx. The page gives you an example that you can start with.
An alternative is to use TorchScript, but that requires torch libraries.
Both of these can be run without python. You can load torchscript in a C++ application https://pytorch.org/tutorials/advanced/cpp_export.html
ONNX is much more portable and you can use in languages such as C#, Java, or Javascript https://onnxruntime.ai/ (even on the browser)
A running exampleJust modifying a little your example to go over the errors I found
Notice that via tracing any if/elif/else, for, while will be unrolled
QUESTION
I'm trying to implement a gradient-free optimizer function to train convolutional neural networks with Julia using Flux.jl. The reference paper is this: https://arxiv.org/abs/2005.05955. This paper proposes RSO, a gradient-free optimization algorithm updates single weight at a time on a sampling bases. The pseudocode of this algorithm is depicted in the picture below.
I'm using MNIST dataset.
...ANSWER
Answered 2022-Jan-14 at 23:47Based on the paper you shared, it looks like you need to change the weight arrays per each output neuron per each layer. Unfortunately, this means that the implementation of your optimization routine is going to depend on the layer type, since an "output neuron" for a convolution layer is quite different than a fully-connected layer. In other words, just looping over Flux.params(model) is not going to be sufficient, since this is just a set of all the weight arrays in the model and each weight array is treated differently depending on which layer it comes from.
Fortunately, Julia's multiple dispatch does make this easier to write if you use separate functions instead of a giant loop. I'll summarize the algorithm using the pseudo-code below:
QUESTION
This question is the same with How can I check a confusion_matrix after fine-tuning with custom datasets?, on Data Science Stack Exchange.
BackgroundI would like to check a confusion_matrix, including precision, recall, and f1-score like below after fine-tuning with custom datasets.
Fine tuning process and the task are Sequence Classification with IMDb Reviews on the Fine-tuning with custom datasets tutorial on Hugging face.
After finishing the fine-tune with Trainer, how can I check a confusion_matrix in this case?
An image of confusion_matrix, including precision, recall, and f1-score original site: just for example output image
...ANSWER
Answered 2021-Nov-24 at 13:26What you could do in this situation is to iterate on the validation set(or on the test set for that matter) and manually create a list of y_true and y_pred.
QUESTION
I am trying to train a model using PyTorch. When beginning model training I get the following error message:
RuntimeError: CUDA out of memory. Tried to allocate 5.37 GiB (GPU 0; 7.79 GiB total capacity; 742.54 MiB already allocated; 5.13 GiB free; 792.00 MiB reserved in total by PyTorch)
I am wondering why this error is occurring. From the way I see it, I have 7.79 GiB total capacity. The numbers it is stating (742 MiB + 5.13 GiB + 792 MiB) do not add up to be greater than 7.79 GiB. When I check nvidia-smi I see these processes running
ANSWER
Answered 2021-Nov-23 at 06:13This is more of a comment, but worth pointing out.
The reason in general is indeed what talonmies commented, but you are summing up the numbers incorrectly. Let's see what happens when tensors are moved to GPU (I tried this on my PC with RTX2060 with 5.8G usable GPU memory in total):
Let's run the following python commands interactively:
QUESTION
I am a bit confusing with comparing best GridSearchCV model and baseline.
For example, we have classification problem.
As a baseline, we'll fit a model with default settings (let it be logistic regression):
ANSWER
Answered 2021-Nov-04 at 21:17No, they aren't comparable.
Your baseline model used X_train to fit the model. Then you're using the fitted model to score the X_train sample. This is like cheating because the model is going to already perform the best since you're evaluating it based on data that it has already seen.
The grid searched model is at a disadvantage because:
- It's working with less data since you have split the
X_trainsample. - Compound that with the fact that it's getting trained with even less data due to the 5 folds (it's training with only 4/5 of
X_valper fold).
So your score for the grid search is going to be worse than your baseline.
Now you might ask, "so what's the point of best_model.best_score_? Well, that score is used to compare all the models used when searching for the optimal hyperparameters in your search space, but in no way should be used to compare against a model that was trained outside of the grid search context.
So how should one go about conducting a fair comparison?
- Split your training data for both models.
QUESTION
I am not able to access jupyter lab created on google cloud
I created one notebook using Google AI platform. I was able to start it and work but suddenly it stopped and I am not able to start it now. I tried building and restarting the jupyterlab, but of no use. I have checked my disk usages as well, which is only 12%.
I tried the diagnostic tool, which gave the following result:
but didn't fix it.
Thanks in advance.
...ANSWER
Answered 2021-Aug-20 at 14:00You should try this Google Notebook trouble shooting section about 524 errors : https://cloud.google.com/notebooks/docs/troubleshooting?hl=ja#opening_a_notebook_results_in_a_524_a_timeout_occurred_error
QUESTION
I am new to Machine Learning.
Having followed the steps in this simple Maching Learning using the Brain.js library, it beats my understanding why I keep getting the error message below:
I have double-checked my code multiple times. This is particularly frustrating as this is the very first exercise!
Kindly point out what I am missing here!
Find below my code:
...ANSWER
Answered 2021-Sep-29 at 22:47Turns out its just documented incorrectly.
In reality the export from brain.js is this:
QUESTION
IF we are not sure about the nature of categorical features like whether they are nominal or ordinal, which encoding should we use? Ordinal-Encoding or One-Hot-Encoding? Is there a clearly defined rule on this topic?
I see a lot of people using Ordinal-Encoding on Categorical Data that doesn't have a Direction. Suppose a frequency table:
...ANSWER
Answered 2021-Sep-04 at 06:43You're right. Just one thing to consider for choosing OrdinalEncoder or OneHotEncoder is that does the order of data matter?
Most ML algorithms will assume that two nearby values are more similar than two distant values. This may be fine in some cases e.g., for ordered categories such as:
quality = ["bad", "average", "good", "excellent"]orshirt_size = ["large", "medium", "small"]
but it is obviously not the case for the:
color = ["white","orange","black","green"]
column (except for the cases you need to consider a spectrum, say from white to black. Note that in this case, white category should be encoded as 0 and black should be encoded as the highest number in your categories), or if you have some cases for example, say, categories 0 and 4 may be more similar than categories 0 and 1. To fix this issue, a common solution is to create one binary attribute per category (One-Hot encoding)
QUESTION
I am using sentence-transformers for semantic search but sometimes it does not understand the contextual meaning and returns wrong result eg. BERT problem with context/semantic search in italian language
by default the vector side of embedding of the sentence is 78 columns, so how do I increase that dimension so that it can understand the contextual meaning in deep.
code:
...ANSWER
Answered 2021-Aug-10 at 07:39Increasing the dimension of a trained model is not possible (without many difficulties and re-training the model). The model you are using was pre-trained with dimension 768, i.e., all weight matrices of the model have a corresponding number of trained parameters. Increasing the dimensionality would mean adding parameters which however need to be learned.
Also, the dimension of the model does not reflect the amount of semantic or context information in the sentence representation. The choice of the model dimension reflects more a trade-off between model capacity, the amount of training data, and reasonable inference speed.
If the model that you are using does not provide representation that is semantically rich enough, you might want to search for better models, such as RoBERTa or T5.
QUESTION
I have a table with features that were used to build some model to predict whether user will buy a new insurance or not. In the same table I have probability of belonging to the class 1 (will buy) and class 0 (will not buy) predicted by this model. I don't know what kind of algorithm was used to build this model. I only have its predicted probabilities.
Question: how to identify what features affect these prediction results? Do I need to build correlation matrix or conduct any tests?
Table example:
...ANSWER
Answered 2021-Aug-11 at 15:55You could build a model like this.
x = features you have. y = true_lable
from that you can extract features importance. also, if you want to go the extra mile,you can do Bootstrapping, so that the features importance would be more stable (statistical).
Community Discussions, Code Snippets contain sources that include Stack Exchange Network
Vulnerabilities
No vulnerabilities reported
Install DPP_CNN_Summarization
Fairseq Install: clone this repo and run cd fairseq pip install --editable .
Dataset Format: download cleaned dataset of CNNDM and extract it under /fairseq/raw_dataset/cnndm/raw/(make a new cnndm folder) , you should get 7 files here: train.src test.src valid.src train.tgt test.tgt valid.tgt corpus_total.txt (which is an combination of train.src and train.tgt) Other datasets should be preprocessed in the same way(7 files)
BPEncoding and Truncate: run cd /raw_dataset/cnndm mkdir bpe-output bpe-truncate cd .. && bash ./bpe-summarization.sh cnndm to generate BPE code list(located in "./raw/code") and apply bpe on 6 dataset files. You will get six Byte Pair Encoded files in "./bpe-output" python truncate.py -sl 600 -tl 70 -d cnndm . after Byte Pair Encoding the length of sentence may be longer than before. What's more the next fairseq-preprocess step do not truncate sentence(but it may make sentences longer than before) in fixed length, so we should truncate the original text and summaries. In our paper we truncate original text for 600 words and summary for 70 words.
Binarizing Data: run cd .. && mkdir -p data-bin/cnndm TEXT=raw_dataset/cnndm/bpe-truncate fairseq-preprocess --source-lang src --target-lang tgt --trainpref $TEXT/train --validpref $TEXT/valid --testpref $TEXT/test --destdir data_bin/cnndm --nwordssrc 50000 --nwordstgt 20000 --memory-efficient-fp16 --task summ_cnndm and should output the following information | [src] Dictionary: 49999 types | [src] raw_dataset/cnndm/bpe-truncate/train.src: 287227 sents, 154092057 tokens, 1.19% replaced by <unk> | [src] Dictionary: 49999 types | [src] raw_dataset/cnndm/bpe-truncate/valid.src: 13368 sents, 7048430 tokens, 1.16% replaced by <unk> | [src] Dictionary: 49999 types | [src] raw_dataset/cnndm/bpe-truncate/test.src: 11490 sents, 6078757 tokens, 1.21% replaced by <unk> | [tgt] Dictionary: 19999 types | [tgt] raw_dataset/cnndm/bpe-truncate/train.tgt: 287227 sents, 15092804 tokens, 4.55% replaced by <unk> | [tgt] Dictionary: 19999 types | [tgt] raw_dataset/cnndm/bpe-truncate/valid.tgt: 13368 sents, 758025 tokens, 4.53% replaced by <unk> | [tgt] Dictionary: 19999 types | [tgt] raw_dataset/cnndm/bpe-truncate/test.tgt: 11490 sents, 632533 tokens, 4.7% replaced by <unk> | Wrote preprocessed data to data_bin/cnndm Pretrained Embeddings: If you want to use pretrained embedding, check the parse_embedding functions in utils.py. In our paper we use fasttext to train the embedding and the output .vec file is exactly the format that fairseq needed. Put the .vec file under "./data_bin/cnndm" to use the pretrained embeddings. For reference our settings are: ./fasttext skipgram -input ../DPPs_Conv_Summarization/fairseq/raw_dataset/cnndm/raw/corpus_total.txt -output model_cnndm_256 -loss hs -ws 5 -epoch 5 -lr 0.05 -dim 256 . NOTE we train the embedding on the raw corpus not BPEncoded corpus so there only part of words have pretrained embedding, specifically: | Found 44025/50000 types in embedding file. | Found 18750/20000 types in embedding file. NOTE: you should make --max-source-positions and --max-target-positions larger than actual to ensure that no training samples are skipped because after fairseq-preprocess the sample will get longer than before. In our experiment we set the two values to 620 and 80 Train: start training by running: TEXT = cnndm mkdir -p checkpoints/summarization_vanilla_$TEXT CUDA_VISIBLE_DEVICES=0 fairseq-train data_bin/$TEXT --lr 0.25 --clip-norm 0.1 --dropout 0.2 --max-tokens 10000 --arch summ_vanilla --save-dir checkpoints/summarization_vanilla_$TEXT --task summ_$TEXT --memory-efficient-fp16 --skip-invalid-size-inputs-valid-test --keep-last-epochs 3 NOTE If you want to train the DPP Model you should add --criterion dpp_micro_loss or --criterion dpp_macro_loss it will cost about 6 GB GPU memories We train the model on a single RTX2070 and it takes about 24 minutes for one epoch. After training for 22 epoches the result is as follows: | epoch 022 | loss 4.290 | ppl 19.56 | wps 10476 | ups 6 | wpb 1852.327 | bsz 35.251 | num_updates 179247 | lr 2.5e-05 | gnorm 0.321 | clip 1.000 | oom 0.000 | loss_scale 64.000 | wall 32337 | train_wall 31350 | epoch 022 | valid on 'valid' subset | loss 4.008 | ppl 16.09 | num_updates 179247 | best_loss 4.00763 | saved checkpoint checkpoints/summarization_vanilla/checkpoint22.pt (epoch 22 @ 179247 updates) (writing took 0.3943181037902832 seconds) | done training in 32352.1 seconds Generate run fairseq-generate data_bin/cnndm --path checkpoints/summarization_vanilla_cnndm/checkpoint_best.pt --batch-size 256 --beam 5 --skip-invalid-size-inputs-valid-test --remove-bpe --quiet --task summ_cnndm --memory-efficient-fp16 --results-path recent-output --print-alignment then the systems and models folder under /recent-output-nounk can be directly used to evaluate ROUGE the model generated readable (keep unk) summaries are under /recent-output/models attention alignments are generated under /recent-output/alignments the infer speed on RTX 2070 is as follows: | Summarized 11490 articles (484468 tokens) in 73.3s (156.80 articles/s, 6611.44 tokens/s) NOTE The original fairseq-generate runs BLEU test on each generated sample but we removed it NOTE If the infer speed is too slow try to set --beam option lower. Interactive: run fairseq-interactive data_bin/cnndm --path checkpoints/summarization_vanilla_cnndm/checkpoint_best.pt --beam 5 --remove-bpe --task summ_cnndm --memory-efficient-fp16 and type in truncated testset article to get the results or you can type in any news article shorter than --max-source-positions. If you paste news artcle on Internet instead from dataset you need remove the \n in the article and lowercase all words.
For now you can change the ratio of KL loss or Det Loss in the fairseq_model.py, in which the raio is defined in class FairseqDetModel and class FairseqKLModel. The ratio is connected to the dpp_macro_loss.py and dpp_micro_loss.py in criterions.
Other hyperparameters can be set directly in the fconv_dpp_macro.py and fconv_dpp_micro.py. We will improve the configuration soon.
Support
Reuse Trending Solutions
Find, review, and download reusable Libraries, Code Snippets, Cloud APIs from over 650 million Knowledge Items
Find more librariesStay Updated
Subscribe to our newsletter for trending solutions and developer bootcamps
Share this Page
