CNN-for-text-classification | A simple CNN implementation in Keras | Machine Learning library

I'm novice to deep learning.I use tensorflow to construct my TextCNN model(two categories) referring this tutorial.
This model can predict the categories of the text. But I want a score (continuous value in [0,1]) rather than the discrete value. For example, If the model give 0.77, the text is more likely one of the category; if it gives 1.0, the text is actually that category.
This is the part of my code.

I wrote a module based on this article: http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/

The idea is pass the input into multiple streams then concat together and connect to a FC layer. I divided my source code into 3 custom modules: TextClassifyCnnNet >> FlatCnnLayer >> FilterLayer

FilterLayer:

I misused binary cross-entropy for softmax, changed to categorical cross-entropy. And did some reviewing about details of the problem below in my own answer

I am trying to using open source data: sogou_news_csv(converted to pinyin using jieba from for text classification following https://arxiv.org/abs/1502.01710 "Text understanding from scratch" by Xiang Zhang and Yann LeCun. (mainly follow the idea of using character level CNN, but the structure proposed in the paper).

I did the preprocessing by using one-hot encoding according to a alphabet collection and filling all those not in the alphabet collection with 0s. As a result, I got the training data with the shape of (450000, 1000, 70),(data_size, sequence_length, alphabet_size).

Then I feed the data into a cnn structure following http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/.

Problem is During the training, the loss and acc merely change, I tried preprocessing again for the data, and tried different learning rate settings, but not helpful, So what went wrong?

Below is one-hot encoding:

I am reviewing Denny Britz's tutorial on text classification using CNNs in TensorFlow. Filter and stride shapes make perfect sense in the image domain. However, when it comes to text, I am confused on how to correctly define the stride and filter shapes. Consider the following two layers from Denny's code:

I have written tensorflow code based on:

http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/

but using precomputed word embeddings from the GoogleNews word2vec 300 dimension model.

I created my own data from the UCML News Aggregator Dataset in which I parsed the content of the news articles and have created my own labels.

Due to the size of the articles I use TF-IDF to filter out the top 120 words per article and embed those into 300 dimensions.

When I run the CNN I created regardless of the hyper parameters it converges to a small general accuracy, around 38%.

Hyper parameters changed:

Various filter sizes:

I've tried a single filter of 1,2,3 Combinations of filters [3,4,5], [1,3,4]

Learning Rate:

I've varied this from very low to very high, very low doesn't converge to 38% but anything between 0.0001 and 0.4 does.

Batch Size:

Tried many ranges between 5 and 100.

Weight and Bias Initialization:

Set stddev of weights between 0.4 and 0.01. Set bias initial values between 0 and 0.1. Tried using the xavier initializer for the conv2d weights.

Dataset Size:

I have only tried on two partial data sets, one with 15 000 training data, and the other on the 5000 test data. In total I have 263 000 data to train on. There is no accuracy difference whether trained and evaluated on the 15 000 training data or by using the 5000 test data as the training data (to save testing time).

I've run successful classifications on the 15 000 / 5000 split using a feed forward network with a BoW input (93% accurate), TF-IDF with SVM (92%), and TF-IDF with Native Bayes (91.5%). So I don't think it is the data.

What does this imply? Is the model just a poor model for this task? Is there an error in my work?

I feel like my do_eval function is incorrect to evaluate the accuracy / loss over an epoch of the data:

I am following this tutorial in order to understand CNNs in NLP. There are a few things which I don't understand despite having the code in front of me. I hope somebody can clear a few things up here.

The first rather minor thing is the sequence_lengthparameter of the TextCNN object. In the example on github this is just 56 which I think is the max-length of all sentences in the training data. This means that self.input_x is a 56-dimensional vector which will contain just the indices from the dictionary of a sentence for each word.

This list goes into tf.nn.embedding_lookup(W, self.intput_x) which will return a matrix consisting of the word embeddings of those words given by self.input_x. According to this answer this operation is similar to using indexing with numpy: