Ludwig: Training using training data and validation CSV caused errors

[allenkao]

When using "train" command with training and validation data will cause inappropriate attribute access violation for Dataframe.
I was running ludwig on Windows 10 and Python 3.6.8.

ludwig train --data_train_csv data_train.csv --data_validation_csv data_val.csv -mdf model.yaml

Using training raw csv, no hdf5 and json file with the same name have been found
Building dataset (it may take a while)
Loading training csv...
done
Loading validation csv..
done
Loading test csv..
done
Concatenating csvs..
done
Traceback (most recent call last):
File "d:\python\python36\lib\runpy.py", line 193, in _run_module_as_main
"__main__", mod_spec)
File "d:\python\python36\lib\runpy.py", line 85, in _run_code
exec(code, run_globals)
File "D:\Python\python36\Scripts\ludwig.exe__main__.py", line 9, in
File "d:\python\python36\lib\site-packages\ludwig\cli.py", line 86, in main
CLI()
File "d:\python\python36\lib\site-packages\ludwig\cli.py", line 64, in __init__
getattr(self, args.command)()
File "d:\python\python36\lib\site-packages\ludwig\cli.py", line 70, in train
train.cli(sys.argv[2:])
File "d:\python\python36\lib\site-packages\ludwig\train.py", line 663, in cli
full_train(**vars(args))
File "d:\python\python36\lib\site-packages\ludwig\train.py", line 224, in full_train
random_seed=random_seed
File "d:\python\python36\lib\site-packages\ludwig\data\preprocessing.py", line 487, in preprocess_for_training
random_seed=random_seed
File "d:\python\python36\lib\site-packages\ludwig\data\preprocessing.py", line 90, in build_dataset_df
global_preprocessing_parameters
File "d:\python\python36\lib\site-packages\ludwig\data\preprocessing.py", line 165, in build_data
preprocessing_parameters
File "d:\python\python36\lib\site-packages\ludwig\features\image_feature.py", line 57, in add_feature_data
csv_path = os.path.dirname(os.path.abspath(dataset_df.csv))
File "d:\python\python36\lib\site-packages\pandas\core\generic.py", line 5067, in __getattr__
return object.__getattribute__(self, name)
AttributeError: 'DataFrame' object has no attribute 'csv'

[w4nderlust]

Thanks for posting this issue. This may be a bug, @ydudin3 can you please look into this?
In the meantime, as a workaround I would suggest you to concatenate your CSVs and adding a column named split contaning a value 0 for the training datapoints, 1 for validation and 2 for test if you have any. This script does it for you: https://github.com/uber/ludwig/blob/master/ludwig/data/concatenate_datasets.py

[w4nderlust]

Hi @allenkao , we pushed a pull request #125 that should solve your issue. Can you please test with the current code on master and confirm that that's the case?

[w4nderlust]

Closing this as it is likely solved, please reopen if that's not the case.