Carla: Hints on creating a client with C++?

[p-schulz]

Hi everyone,

first of all: great work! So far I managed to get everything working using your example python client and the documentation, extended the client further (extract all measured data and dump it), built custom maps using splines, got them up running with RoutePlanners and (I think) understood the design of carla.

What I'm now trying to do is get a simple client up running with C++. For this I'm using protobuf and created the carla_server.pb.h/cc files. I already took a look at the carla_server C API. But I'm having problems at requesting a new episode and reading data, maybe I'm just not familiar enough with the protobuf messaging concept. What I have so far by looking at the python API (mostly client.py):

// create tcp communication (localhost:2000)
boost::asio::ip::tcp::iostream world_client;
world_client.connect(m_server, std::to_string(m_port));

// request new episode
carla_server::EpisodeStart episode_request = carla_server::EpisodeStart();
episode_request.set_player_start_spot_index(5);
string rq_string;
episode_request.SerializeToString(&rq_string);
const char* msg_c = rq_string.c_str();
world_client.write(msg_c, strlen(msg_c));

// read data from carla server (header with message length and actual trailing data)
// to be placed inside a loop later
char* header;
char* carla_msg;
world_client.read(header, 4);
streamsize length = header[1]; 
world_client.read(carla_msg, length);

So what I'm asking:

• Is it going to work this way?
• Does anyone have an advice on where to look at how things are done in python?
• Is it necessary to create a carla_settings object or is it somehow possible to use the default CarlaSettings.ini (e.g. by not specifying any scene description)?

Thank you very much!

[p-schulz]

Update:
I've added the scene description from file to send it with a request:

try {
        // open streams for communication with carla server
        stream_world.connect(m_server, std::to_string(m_port));
        stream_gt.connect(m_server, std::to_string(m_sensor_port));
        stream_control.connect(m_server, std::to_string(m_control_port));

        // open CarlaSettings.ini
        ifstream settings_ini;
        streampos file_size;
        char* settings;
        settings_ini.open(m_settings_dir, ios::in|ios::binary|ios::ate);
        if(settings_ini.is_open()) {
            file_size = settings_ini.tellg();
            settings = new char[file_size];
            settings_ini.seekg (0, ios::beg);
            settings_ini.read (settings, file_size-2); // remove gibberish chars at end
            settings_ini.close();
            cout << "Scene description file found!" << std::endl;
            cout << settings << std::endl;
        }
        else {
            cout << "Scene description not found!" << std::endl;
            return false;
        }

        // send request with scene_description
        carla_server::RequestNewEpisode episode_request = carla_server::RequestNewEpisode();
        string scene_description(settings);
        episode_request.set_ini_file(scene_description);
        string rq_string;
        episode_request.SerializeToString(&rq_string);
        const char* msg_request = rq_string.c_str();
        stream_world.write(msg_request, strlen(msg_request));

        // send episode start with fixed start position (5)
        carla_server::EpisodeStart episode_start = carla_server::EpisodeStart();
        episode_start.set_player_start_spot_index(5);
        string ep_string;
        episode_start.SerializeToString(&ep_string);
        const char* msg_start = ep_string.c_str();
        stream_world.write(msg_start, strlen(msg_start));

        std::cout << "Connected to Carla!" << std::endl;
        return true;
} catch (const std::exception& e) {
        // in case ue4 is not running and tcp connections fail
        std::cerr << e.what() << std::endl;
        return false;
}

But how are the sensor data and the measurements packed, or how do I know the length of the header in the sensor data before I start to read (according to the struct in carla_server.h)?

[nsubiron]

Hi @p-schulz,

First of all, we are about to change all the client-server communication, so if you are not in a hurry you can wait until we have it working. I'm in charge of making those changes, I'm trying to move the core of the client to C++ and probably stop using protobuf, if that's the case we'll have a C++ interface for the client too. This is part of the current code sprint (see #439), I expect to have it implemented by the end of June, but some functionality may be still missing. (At this point looking at how feasible is to use rpclib).

Now if you still want to continue, here are some relevant links

• CarlaServer protocol documentation.
• Python code: client.py implements the protocol, tcp.py the TCP connection.
• The proto definition carla_server.proto (I guess you've already seen this one)
• Each sensor has its own serialization (for performance), they're documented in the comments in Unreal/CarlaUE4/Plugins/Carla/Source/Carla/Sensor

And some things to take into account

• Yes, you can load a _CarlaSettings.ini_ as string, that works.
• Yes, all messages are prepended by an uint32_t header stating the size of the coming message.
• In the world thread, each write has to be followed by a read, otherwise it will block; and the messages must follow the strict order that appears in the CarlaServer documentation.
• The "carla_server.h" is only for the interface with Unreal Engine, does not necessarily match the data sent through the network.

As you see, it's sort of cumbersome protocol. That's why I want to move it to something simpler and more flexible.

Also,

char* header;
char* carla_msg;
world_client.read(header, 4);
streamsize length = header[1]; 
world_client.read(carla_msg, length);

when you read from a boost::asio socket you need to allocate the buffer yourself before hand.
--> e.g. char header[4u]; See boost example.

Good luck ;)

[p-schulz]

Hi @nsubiron,

I overlooked the server documentation... And yes, I followed the issues about redesigning the client-server communications, but wasn't aware you actually try to implement the client in C++. That would be awesome.

Thanks for your help! :)

[p-schulz]

@nsubiron
sorry to bother again, but I'm still encountering errors on the server side when writing to the world stream (error reading message / EOF / operation canceled). The connection remains open, so the problem seems to be in the messages or the way they are sent...

Following the protocol from the server documentation, the first message sent is the size of e.g. RequestEpisode as uint32 little endian (I hope boost::endian works), the second is the actual request follows.

By saying each write has to be followed by a read, you mean reading after sending the request (header + scene description), have I got you right?

write header+ write request -> read header + read scene description
write header+ write start -> read header+ read ready

using

uint32_t header = boost::endian::native_to_little(msg_size);
boost::asio::write(socket_world, boost::asio::buffer( (const char*) header, sizeof(uint32_t)));
boost::asio::write(socket_world, boost::asio::buffer( msg_request, msg_size));

uint32_t re_size;
boost::asio::read(socket_world, boost::asio::buffer( &re_size, sizeof(uint32_t)));
char buffer[boost::endian::little_to_native(re_size)];
boost::asio::read(socket_world, boost::asio::buffer( &buffer, boost::endian::little_to_native(re_size)));

Because reading the header from the server returns 0. On my client side I'm facing no errors, the buffers are already allocated. I can also connect to the sensor port and try to read some but actually receive no data.

Do you have any idea?