Deeplabcut: Labels of the markers are mixed when evaluating/tracking (after adding new labels to config.yaml)

[AntoineCribel]

Describe the bug
After training the network (resnet_50), evaluating it and analyzing videos, I get markers that seems to be correctly placed but with wrong labels (they seems to be mixed randomly). This happened after that I added new label names in the config file and (re)labeled previous and new videos.

I tried a number of things:

• I made sure every one of the "old" videos were updated with the new labels
• I checked every labeled images with deeplabcut.check_labels (everything looks good)
• making a new dlc project from scratch and copy pasting inside only the labeled-data folder
• deeplabcut.dropannotationfileentriesduetodeletedimages
• I checked if coordinates/labels were mixed in the CollectedData_.csv in the training dataset, but I couldn't find anything wrong, however I noticed that it is using the same labels order than the CollectedData_.csv of the first labeled video which is different than the order in my config.yaml

Indeed I also noticed that the order of the label names (in the column) in the .csv files of each labeled video is not always the same. This is because I added new labels and changed the order of the labels in my config file. Is it possible that deeplabcut get confused by that and use the wrong order for the labels?

To Reproduce
Steps to reproduce the behavior:

1. Start labeling some videos (at this point the training and evaluating worked well)
2. Change the order of the labels in the config.yaml
3. Label more videos (I thinks it was still working at this point, but I am not sure)
4. Add new label names in the config.yaml
5. Relabel the previously labeled videos (to add the new labels)
6. Label more videos
7. Train the network + evaluate
8. The marker's labels should be mixed

Expected behavior
Not getting mixed labels ^^

Desktop:

• Ubuntu Mate 18.04
• DeepLabCut 2.1.7

[MMathisLab]

we will look into it, thanks!

quick Q: "Train the network + evaluate" <-- did you reload the previous weights or use "fresh weights"? Just asking to help us debug.

[AntoineCribel]

I used fresh weights (I manually deleted the trained networks in the dlc_model folder).

[AntoineCribel]

Another thing, I tried removing the folders in labeled-data that had a CollectedData_.csv with the wrong order of label names. That solved the issue of the mixed labels after training and evaluating.

[AlexEMG]

This can indeed, in certain situation be wrong. We have a fix now. Will be released soon.

[AntoineCribel]

I tried reordering the label names in the CollectedData_.csv of all labeled videos (with following script), it solved my problem.

import pandas as pd
from glob import glob
import deeplabcut as dlc

path_labels = "path_labeled-data_folder"
path_cfg = "path_config.yaml"

path_labels = glob(path_labels + '*')

label_names_commas = 'copy/past 2nd ligne of a correct CollectedData_.csv (should be something like: 'bodyparts,mouth,mouth,tail,tail)'
coord_names_commas = 'copy/past 3nd ligne of a correct CollectedData_.csv (should be something like: coords,x,y,x,y)'
label_names = label_names_commas.split(',')
coord_names = coord_names_commas.split(',')

label_keys = pd.MultiIndex.from_tuples(zip(label_names, coord_names))

for p in path_labels:

path_csv = glob(p + '/CollectedData_*.csv')
for c in path_csv:
    header = list(pd.read_csv(c, header=None).loc[0])

    df = pd.read_csv(c, header=[1, 2])
    df_reorder = df[label_keys]  # rearrange column here
    df_reorder.columns = pd.MultiIndex.from_tuples(zip(header, label_names, coord_names))
    df_reorder.to_csv(c, index=False)

dlc.convertcsv2h5(path_cfg, userfeedback=False)

[AlexEMG]

That makes sense, in the future this should no longer be necessary with DLC 2.1.8 :)

Just did more testing, while shuffling wildly. Now, I can no longer get such errors!

[MMathisLab]

fixed; pip install deeplabcut==2.1.8

[AntoineCribel]

Great!! Thank you very much!