Yarp: Yarpdataplayer playing image data to yarpview, an error due to image size

[simbamford]

Describe the bug
I have dumped a stream of images to a data.log using yarpdatadumper. I can play these back by connecting yarpdataplyer to yarpview. When I modify the data.log file to contain images of a custom size (180x240), the dataplayer plays the data, but yarp view fails with the following error message:

There is a problem reading an image
incoming: width 180, height 240, code 6449010, quantum 8, size 129600
my space: width 180, height 240, code 6449010, quantum 8, size 130560
[FATAL]Assertion failure at /home/sbamford/repos/yarp/src/libYARP_sig/src/Image.cpp:64 (allocatedBytes == (size_t) header.imgSize)
/usr/local/lib/qt5/qml/robotology/yarp/view/../../../../../libYARP_OS.so.3(_ZNK4yarp2os3Log5fatalEPKcz+0x16b) [0x7f37dfd05ebb]
/usr/local/lib/qt5/qml/robotology/yarp/view/../../../../../libYARP_sig.so.3(_ZN4yarp3sig5Image4readERNS_2os16ConnectionReaderE+0xc15) [0x7f37e44b5b95]
/usr/local/lib/qt5/qml/robotology/yarp/view/../../../../../libYARP_OS.so.3(_ZN4yarp2os20PortReaderBufferBase4readERNS0_16ConnectionReaderE+0x1bb) [0x7f37dfd6613b]
/usr/local/lib/qt5/qml/robotology/yarp/view/../../../../../libYARP_OS.so.3(+0xd8f46) [0x7f37dfd58f46]
/usr/local/lib/qt5/qml/robotology/yarp/view/../../../../../libYARP_OS.so.3(_ZN4yarp2os4impl8PortCore9readBlockERNS0_16ConnectionReaderEPvPNS0_12OutputStreamE+0xba) [0x7f37dfd50cfa]
/usr/local/lib/qt5/qml/robotology/yarp/view/../../../../../libYARP_OS.so.3(+0xdd206) [0x7f37dfd5d206]
/usr/local/lib/qt5/qml/robotology/yarp/view/../../../../../libYARP_OS.so.3(+0x128255) [0x7f37dfda8255]
/usr/lib/x86_64-linux-gnu/libstdc++.so.6(+0xbd66f) [0x7f37f29cd66f]
/lib/x86_64-linux-gnu/libpthread.so.0(+0x76db) [0x7f37f20d76db]
/lib/x86_64-linux-gnu/libc.so.6(clone+0x3f) [0x7f37f241188f]

It seems to me that perhaps there is a minimum size that yarpdataplayer can allocate to a blob, and my chosen image size is lower that that.

To Reproduce
Steps to reproduce the behavior: Use the attached info.log and data.log files, put them in an appropriate folder structure, then minimally modify the following xml to match your folder structure, before using it in yarpmanager to set up the appropriate apps and connections,

info.log
data.log

<application>
<name>play </name>

<module>
    <name> yarpdataplayer </name>
    <parameters>  </parameters>
    <node> localhost </node>
</module>

<module>
    <name> yarpview </name>
    <parameters> --name /viewer/ch0frame --w 362 --h 482 --synch --RefreshTime 33 </parameters>
    <dependencies>
        <port timeout="100.0" request="load /home/sbamford/data/rpg/shapes_rotation" reply="[ok]">/yarpdataplayer/rpc:i</port>
    </dependencies>
    <node> localhost </node>
</module>

<connection>
    <from> /file/ch0frame:o </from>
    <to> /viewer/ch0frame </to>
<protocol> udp </protocol>
</connection>

</application>

Expected behavior
For simplicity, the attached data.log file only contains a single image, so if it worked as expected, when you play the data using the player, a single image (a random colour image) should appear on yarpview.

Configuration (please complete the following information):

• OS: ubuntu 18 via WSL.
• yarp version: 3.2.0

[vtikha]

Dear @simbamford, it is not clear to me why you are modifying by hand the data.log. Plus a typical dumping of images is done using yarpdatadumper --name /images --type image which is not what you have done from what I can see. In general I discourage you to modify by hand the files dumped. What I am trying to determine is if what you are doing is actually done in a "correct way" or customised for your project.

[simbamford]

What we are trying to achieve is to be able to import yarp logs to python and to export them again. This gives us the ability to (a) easily visualise and manipulate data created from our sensors (iCub, stefi), (b) import data from third parties and particularly from different dynamic vision sensors and pass them through our yarp-based processing chains, so as to compare performance on established benchmarks, and (c) use python as a gateway to established machine learning frameworks. The data.log I give as an example has been created by a python script and is a surrogate for data which we are trying to import from a third party dataset which uses the DAVIS240C sensor - an established competitor to the sensors on the iCub.

[Nicogene]

As @vtikha said, I suggest to dump the images using the following command:

yarpdatadumper --name /images --type image

In this way the they are stored as a collection of images with the format you prefer (.ppm, .jpeg etc).

Once you have the images it easier to resize them, through a python script for example, respect to change the data.log that it shouldn't be touched.

Maybe I don't understand completely your purpose, but I advise you to not dump the images as bottles.

BTW here you can find the documentation of yarpdatadumper.

[simbamford]

Can yarpdataplayer play a set of images? I need to be able to play back these images with yarpdataplayer, synchronised with other data (which is also dumped in bottles, although it may be in a separate file).

[Nicogene]

Can yarpdataplayer play a set of images?

Sure, once you dump them as images, the yarpdataplayer can replay back as images.
See http://yarp.it/yarpdataplayer_example.html as example.

Closing for now, feel free to re-open it if you have further doubts.