leaf-classification | Deep Reinforcement , Highway , and Convolutional | Machine Learning library

There are multiple differences between both models, your TF model uses ADAM, while your Keras model uses SGD. The learning rates are different as well, and learning rate greatly affects model convergence.

The loss functions also don't match, the Keras model uses categorical cross-entropy, while the TF model is using sigmoid cross-entropy with logits (which usually is used for multilabel classification). Also sigmoid_cross_entropy_with_logits takes logits as input (real numbers), and you are giving it the output of a softmax function.

There are also differences in weight initialization, you are using truncated normal distribution for the TF model, while Keras by default uses glorot_uniform or uniform.

These differences are of course the reason why one model trains, and the other does not.