Keras: How keras behave when step_per_epoch != NUM_OF_SAMPLES / batch_size?

[ziyigogogo]

Hi there,
I am confused with the document here:

steps_per_epoch: Total number of steps (batches of samples) to yield from generator before declaring one epoch finished and starting the next epoch. It should typically be equal to the number of unique samples of your dataset divided by the batch size.

In my poor understanding, the steps_per_epoch should be "EXACTLY THE SAME" as "NUM_OF_SAMPLES / batch_size", so all the samples can be iterated once.

To understand more about this param, I really want to know what keras dose when step_per_epoch is set un-typically .

To be clear, my questions are:

1. when step_per_epoch < NUM_OF_SAMPLES / batch_size

   if some of the samples were skipped? So, the trainer gets the input size < NUM_OF_SAMPLES?

2. when step_per_epoch > NUM_OF_SAMPLES / batch_size,

   if some of the samples imported multiple times to the trainer, so the actual input size > NUM_OF_SAMPLES?

Thank you so much for your time.

[Dref360]

The steps_per_epoch argument is not related to how the samples are fed. It's just a number that is used to define an 'epoch'. Some people want their callbacks to be called more often than once per "real epoch"*. So they can set a lower steps_per_epoch.

1. The unseen samples will be seen in the second epoch
2. Your model will see the same samples multiple times.

*real epoch is a loop over all of your data

[ziyigogogo]

Hi，Dref.

Thanks for you quick response, still not so confidence that if i understand your comment right.
Since, you mentioned that this "steps_per_epoch" is used to define an "epoch",how could it not related to the sample feeding things? Or, in my understand of your description above:

-The unseen samples will be seen in the second epoch.

This epoch you mentioned here is just a "virtual epoch" that is a "subset" of a "real epoch"?

For instance, lets define a "real epoch" with100 samples, batch_size = 10. So, typically the "steps_per_epoch" should be 100 / 10 =10

If we set it to 5, so each virtual epoch has 5 * batch_size = 50 samples. As you said, the unseen 50 samples will be seen in the second epoch. If this mean the number of "virtual epoch" is automatically increased to 2, so that for each "real epcoh" with 100 samples there are 2 "virtual epoch" with 50 samples created by keras?
if so, that would make sense. Looking forward hearing from you again, many thanks !
@Dref360

[Dref360]

You kinda lost me here, sorry.

Let's take your example, let's assume that you use a keras.utils.Sequence, but generators are similars. So you do something like this :
model.fit_generator(sequence, steps_per_epoch=5, epochs = 5, ...)

1. We launch a process to creates the samples.
   i. The process knows that you have 10 batches per epoch from the Sequence itself (__len__ is defined)
2. Keras starts a loop asking for 5 batches, because you asked for an epoch of 5 steps_per_epoch.
   i. It computes the loss on those 5 batches, calls you callbacks etc.
3. Now Keras starts its second epoch and it asked for 5 batches.
   i. The process is still creating the samples for batches 6..10 so it return those. Then it restarts to the sample 0.

As I said, the steps_per_epoch arguments allows callbacks to be called more frequently or compute your val_loss every 1000 batches instead of waiting for an actual epochs to be done.

[ziyigogogo]

Oh i see, that's make sense then, thanks for you patience and wish you all the best!

[cl2227619761]

BUT when you AUGMENTED your data, the steps_per_epoch is unknow, so the steps_per_epoch should be set great than the sample_num / batch_size?