Ignite: Parameter setting of run function in engine

[InstantWindy]

Hi. I need two datasets for training at the same time. How can I rewrite the run function in engine to support reading two datasets. Thanks.

[vfdev-5]

@InstantWindy you can do this like that for example

small_img_data_loader = ...
large_img_data_loader = ...

def cycle(iterable):
    while True:
        for i in iterable:
            yield i

small_img_data_loader_iter = cycle(small_img_data_loader)
large_img_data_loader_iter = cycle(large_img_data_loader)

def update(engine, _):

    batch_1 = next(small_img_data_loader_iter)
    batch_2 = next(large_img_data_loader_iter)
    ...

trainer = Engine(update)

num_iters = 10000  
data = list(range(num_iters))
trainer.run(data, num_epochs)

large_img_data_loader_iter = small_img_data_loader_iter = None

HTH

[InstantWindy]

num_iters = 10000 data = list(range(num_iters)) trainer.run(data, num_epochs)
But, it seems that there is no data input for this operation. What does "data" mean here? You mean I just need to pass two dataset in the update function?

[vfdev-5]

data is just a counter of a single epoch for two data-sources. Otherwise you can set one dataloader as a principal data source and fetch another batches as above:

small_img_data_loader = ...
large_img_data_loader = ...

def cycle(iterable):
    while True:
        for i in iterable:
            yield i

large_img_data_loader_iter = cycle(large_img_data_loader)

def update(engine, batch_1):

    batch_2 = next(large_img_data_loader_iter)
    ...

trainer = Engine(update)

trainer.run(small_img_data_loader, num_epochs)

large_img_data_loader_iter = None

Here, single epoch is defined by the size of small_img_data_loader.

[InstantWindy]

I understand. Thanks.