Handson-ml: Chapeter10: Using plain TensorFlow, how the tensorflow know the batch_size?

[mingjunz]

I got error:

labels must be 1-D, but got shape [50,10]

I guess this is because tensorflow does not know the batch_size is 50, so how can i tell the tensorflow the batch size for this "Using plain TensorFlow"?

The code I am running is:

import tensorflow as tf
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data
n_inputs = 28*28  # MNIST
n_hidden1 = 300
n_hidden2 = 100
n_outputs = 10

mnist = input_data.read_data_sets('MNIST_data', one_hot=True)

X = tf.placeholder(tf.float32, shape=(None, n_inputs), name="X")
y = tf.placeholder(tf.int64, shape=(None), name="y")

def neuron_layer(X, n_neurons, name, activation=None):
    with tf.name_scope(name):
        n_inputs = int(X.get_shape()[1])
        stddev = 2 / np.sqrt(n_inputs)
        init = tf.truncated_normal((n_inputs, n_neurons), stddev=stddev)
        W = tf.Variable(init, name="kernel")
        b = tf.Variable(tf.zeros([n_neurons]), name="bias")
        Z = tf.matmul(X, W) + b
        if activation is not None:
            return activation(Z)
        else:
            return Z

with tf.name_scope("dnn"):
    hidden1 = neuron_layer(X, n_hidden1, name="hidden1",activation=tf.nn.relu)
    hidden2 = neuron_layer(hidden1, n_hidden2, name="hidden2",activation=tf.nn.relu)
    logits = neuron_layer(hidden2, n_outputs, name="outputs")

with tf.name_scope("loss"):
    xentropy = tf.nn.sparse_softmax_cross_entropy_with_logits(labels=y,logits=logits)
    loss = tf.reduce_mean(xentropy, name="loss")

learning_rate = 0.01

with tf.name_scope("train"):
    optimizer = tf.train.GradientDescentOptimizer(learning_rate)
    training_op = optimizer.minimize(loss)

with tf.name_scope("eval"):
    correct = tf.nn.in_top_k(logits, y, 1)
    accuracy = tf.reduce_mean(tf.cast(correct, tf.float32))

init = tf.global_variables_initializer()
saver = tf.train.Saver()

n_epochs = 40
batch_size = 50

with tf.Session() as sess:
    init.run()
    for epoch in range(n_epochs):
        for iteration in range(mnist.train.num_examples // batch_size):
            X_batch, y_batch = mnist.train.next_batch(batch_size)
            print(y_batch)
            print(y_batch.shape)
            sess.run(training_op, feed_dict={X: X_batch, y: y_batch})
        acc_train = accuracy.eval(feed_dict={X: X_batch, y: y_batch})
        acc_val = accuracy.eval(feed_dict={X: mnist.validation.images,
                                            y: mnist.validation.labels})
        print(epoch, "Train accuracy:", acc_train, "Val accuracy:", acc_val)

    save_path = saver.save(sess, "./my_model_final.ckpt")

---

[ageron]

Hi @mingjunz ,

Just remove one_hot=True when you load MNIST, and it will work fine.

When you use sparse_softmax_cross_entropy_with_logits(), TensorFlow expects the targets to be class indices (since there are 10 classes, each label must be a number from 0 to 9). So if the batch has 50 instances, then the labels should be a 1D array with 50 integers (from 0 to 9). The shape should be [50], and dtype should be integer.

When you use softmax_cross_entropy_with_logits(), TensorFlow expects the targets to be a probability distribution: for each instance, it expects 10 probabilities (one per class). So if the batch has 50 instances, then the labels should be a 2D array (a matrix) with 50 rows and 10 columns. Its shape should be [50, 10], and dtype should be float.

When you specify one_hot=True, TensorFlow gives you labels in the form of one-hot vectors, so for example the class number 3 is represented as [0,0,1,0,0,0,0,0,0]. So if you really want to use one_hot=True, then you must use softmax_cross_entropy_with_logits() instead of sparse_softmax_cross_entropy_with_logits().

Hope this helps.

[mingjunz]

works, thank you @ageron