Keras: Keras fit generator - ValueError: Failed to find data adapter that can handle input

[castilho25]

Hey, I'm trying to fit my deep learning model with a custom generator.

When i fit the model, it shows me this error:
`Traceback (most recent call last):
File "/home/castilho/PycharmProjects/Chargrid/Chargrid/main.py", line 201, in
target_path=target_path, prefix=prefix_splits, make_new_representation=False, train=True)
File "/home/castilho/PycharmProjects/Chargrid/Chargrid/main.py", line 117, in main
nn.train(representations_path=representations_path, target_path=target_path, training_filenames = data['train_imgs'], validation_filenames=data['val_imgs'])
File "/home/castilho/PycharmProjects/Chargrid/Chargrid/neural_network.py", line 150, in train
validation_steps=int(len(validation_filenames) // batch_size))
File "/home/castilho/PycharmProjects/Chargrid/venv/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training.py", line 819, in fit
use_multiprocessing=use_multiprocessing)
File "/home/castilho/PycharmProjects/Chargrid/venv/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 235, in fit
use_multiprocessing=use_multiprocessing)
File "/home/castilho/PycharmProjects/Chargrid/venv/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/training_v2.py", line 533, in _process_training_inputs
adapter_cls = data_adapter.select_data_adapter(x, y)
File "/home/castilho/PycharmProjects/Chargrid/venv/lib/python3.6/site-packages/tensorflow_core/python/keras/engine/data_adapter.py", line 998, in select_data_adapter
_type_name(x), _type_name(y)))
ValueError: Failed to find data adapter that can handle input: ,

Process finished with exit code 1
`

I this this a bug in Keras. Below, is a part of the code when i fit the model.
` def train(self, representations_path: str, target_path: str, training_filenames: list, validation_filenames: list,
batch_size: int = 7):

    train_generator = RepresentationGenerator(representation_path=representations_path, target_path=target_path,
                                              filenames=training_filenames, batch_size=batch_size)
    val_generator = RepresentationGenerator(representation_path=representations_path, target_path=target_path,
                                            filenames=validation_filenames, batch_size=batch_size)
    self.model_semantic.fit(x=train_generator, batch_size=7,
                            steps_per_epoch=int(len(training_filenames) // batch_size),
                            epochs=10,
                            verbose=1,
                            validation_data=val_generator,
                            validation_steps=int(len(validation_filenames) // batch_size))
    return 0`

My class RepresentationGenerator is:

`from tensorflow_core.python.keras.utils.data_utils import Sequence

class RepresentationGenerator(Sequence):

def __init__(self, representation_path, target_path, filenames, batch_size):
    self.filenames = np.array(filenames)
    self.batch_size = batch_size
    self.representation_path = representation_path
    self.target_path = target_path

def __len__(self):
    return (np.ceil(len(self.filenames) / float(self.batch_size))).astype(np.int)

def __getitem__(self, idx):
    files_to_batch = self.filenames[idx * self.batch_size: (idx + 1) * self.batch_size]
    batch_x, batch_y = [], []
    for file in files_to_batch:
        batch_x.append(np.load(self.representation_path + file + ".npy", allow_pickle=True))
        batch_y.append(np.load(self.target_path + file + ".npy", allow_pickle=True))

    return np.array(batch_x), np.array(batch_y)

`

When i fit the model with a generator, the variable Y assume as None and it shows the error above. How can i fix that?

System information

• Have I written custom code (as opposed to using example directory): Yes
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Linux Ubuntu 18.04
• TensorFlow backend (yes / no): Yes
• TensorFlow version: 2.1.0
• Keras version: 2.3.1
• Python version: 3.6
• GPU model and memory: NVIDIA Corporation GP108M [GeForce MX250]

[Erik-BM]

Have you found any solution to this?

[aphera]

I solved this problem by following the exact style of imports that data_adapter.py uses when defining my generator:

from tensorflow.python.keras.utils import data_utils

class STFTGenerator(data_utils.Sequence):