Swoole-src: how to write broker (accept and maintain tcp from multiple device) app that let external process to talk to each device via maintained TCP.

[pheonics]

i'm not firing a bug report or request for feature but rather seek for suggestion how to use swoole to solve my problem. i'm not sure of this topic is appropriate here, if not suggest where should i seek for help..

i am writing (kind of) broker app where on one side it listen to a set of appliance where tcp connection must be maintain and use to relay command based on request come from another side of the broker.

I already have a working broker written in pure php using stream_socket and stream_select to archive non-blocking effect. during more research on better way to do non-blocking php i stumble into world of swoole and decide to give it a try.

_i consider myself a swoole toddler so please bear with me while i explore this .. please kindly bear with me._

refer to my existing broker app mainly do open 2 stream_socket_server .. the first one accepting tcp from an appliance. on accepted the broker will query the detail such as the appliance unique id and then keep maintain the tcp connection for subsequence query or control command that triggered httpd request (the second server).

note that i need to facilitate 3 communication pattern to and from the appliance
a. broker itself initiate request to appliance such as sending keep alive to the application)
b, external demand came through the http (second listening port) trigger command from broker to appliance and wait for response which then reformatted and return pack in http response.
c. appliance initiate request to broker such heartbeat and system alert.

1. What did you do?

this is mainly what I start to do based on suggestion from doc ref how to listen to multiple port
i start http server then add second listener for tcp server.

// create http server to handle broker side first
$broker_server=new swoole_http_server($pb_broker_cfg['broker_listen_ip'],
  $pb_broker_cfg['broker_listen_port']);

// then we add tcp server to handler connection from power bank sharing station
$station_port=$broker_server->addListener($pb_broker_cfg['station_listen_ip'],
  $pb_broker_cfg['station_listen_port'],SWOOLE_SOCK_TCP);

// set broker server callback
$broker_server->on('request', function ($request, $response) {
  // this object will interpret the request and locate coresponding TCP connection for its desired
  // target appliance 
  $broker = new PB_Broker($request, $response); 
});

// set station callback
$station_port->on('connect', function ($serv, $fd){
  $station = new PB_Station($serv, $fd);

  // timer to do house keeping
  $server->tick(1000, function() use ($server, $fd) {
    $station = PB_station::get_station($serv, $fd);
    $station->house_keeping();
  });
  echo "PB_Station[$station->station_id]:Connect.\n";
});

$station_port->on('receive', function ($serv, $fd, $from_id, $data) {
  $serv->send($fd, 'Swoole: '.$data);
  $station = PB_station::get_station($serv, $fd);
  $station->handler($data);
});

$station_port->on('close', function ($serv, $fd) {
  $station = PB_station::get_station($serv, $fd);
  unset($station);
  echo "Client: Close.\n";
});

// main event loop
$broker_server->start();

2. What did you expect to see?

i want one process that listen to both side (tcp and http) without blocking. this process maintain multiple TCP connection

but it surprise as once start the command, the first thing i notice is a see a bunch of new process spawned.. I do not expect this.. i know in the diagram how swoole work, it depict multiple reactor, master and multiple worker.. but i though it is all in one process... and i'm not usine task.. why do i need multiple worker to process task? ?? ??

if this a way of swoole, so i try to get alone but i don't know how to or where to stumble forward?
with multiple workers where is my persistence TCP connection from each appliance go?
is it maintained by all worker? or one of the broker?
and upon http connection how can I locate my TCP connection?

3. What version of Swoole are you using (show your php --ri swoole)?

swoole

swoole support => enabled
Version => 4.2.6
Author => Swoole Group[email: [email protected]]
coroutine => enabled
epoll => enabled
eventfd => enabled
signalfd => enabled
cpu_affinity => enabled
spinlock => enabled
rwlock => enabled
sockets => enabled
openssl => OpenSSL 1.0.2k-fips 26 Jan 2017
pcre => enabled
zlib => enabled
mutex_timedlock => enabled
pthread_barrier => enabled
futex => enabled
mysqlnd => enabled
async_redis => enabled

Directive => Local Value => Master Value
swoole.enable_coroutine => On => On
swoole.aio_thread_num => 2 => 2
swoole.display_errors => On => On
swoole.use_namespace => On => On
swoole.use_shortname => On => On
swoole.fast_serialize => Off => Off
swoole.unixsock_buffer_size => 8388608 => 8388608

4. What is your machine environment used (including version of kernel & php & gcc) ?

Centos7

[twose]

Not quite understand what you mean
Regarding Swoole's multi-process mode, you can use the SWOOLE_BASE mode to make the server become a single process server to temporarily resolve your understanding problem, and then continue to understand others.

$server = new swoole_server('127.0.0.1', 9501, SWOOLE_BASE);

maybe @flddr can help you

[ghost]

You wrote a great explanation, but, for me too, it's a little bit hard to follow on my phone. I will have a look when i am back on my laptop.

My english is not quite good as yours, but, feeling so, a broker App should be connected via websocket? What do you think? Where comes the data from and which side triggers the updates? Is this like realtime money charts etc.?

[pheonics]

Not quite understand what you mean
Regarding Swoole's multi-process mode, you can use the SWOOLE_BASE mode to make the server become a single process server to temporarily resolve your understanding problem, and then continue to understand others.

$server = new swoole_server('127.0.0.1', 9501, SWOOLE_BASE);

maybe @flddr can help you

since I create http_server first and then tcp_server and use http one as main server, can and how I apply SWOOLE_BASE?

[pheonics]

You wrote a great explanation, but, for me too, it's a little bit hard to follow on my phone. I will have a look when i am back on my laptop.

My english is not quite good as yours, but, feeling so, a broker App should be connected via websocket? What do you think? Where comes the data from and which side triggers the updates? Is this like realtime money charts etc.?

it come across my mind also that it behave pretty much like web socket.. but I do not have control over the device (the appliance) that connect to my broker. I only have control on the other side where accepting http request with json content.

[ghost]

Principially

$broker_server=new swoole_http_server($pb_broker_cfg['broker_listen_ip'], $pb_broker_cfg['broker_listen_port'], SWOOLE_BASE); as twosee said...

But i do not understand more than before

[ghost]

There are several parts in your code that are hard to read or understand :relaxed:

I am not sure about whats happening here

// set station callback
$station_port->on('connect', function ($serv, $fd){
  $station = new PB_Station($serv, $fd);

  // timer to do house keeping
  $server->tick(1000, function() use ($server, $fd) {
    $station = PB_station::get_station($serv, $fd);
    $station->house_keeping();
  });
  echo "PB_Station[$station->station_id]:Connect.\n";
});

$station_port->on('receive', function ($serv, $fd, $from_id, $data) {
  $serv->send($fd, 'Swoole: '.$data);
  $station = PB_station::get_station($serv, $fd);
  $station->handler($data);
});

$station_port->on('close', function ($serv, $fd) {
  $station = PB_station::get_station($serv, $fd);
  unset($station);
  echo "Client: Close.\n";
});

[pheonics]

@flddr it might be difficult because I did not include all object implementation as it is left to be abstact at this top level code and partly due to obscure vocab that i used to name thing and partly due to i crossing over from object oriented where many detail was left off assuming it is state of the object/class is self contained..... 5555 sorry.. i'm switching my head to pure structure synchronous programming to this brave new world .. note that i have not done deep programming over 20 years... now you know why I may not talk the same technical language .. please bear with me.

[pheonics]

@flddr because I may not full understand swoole architecture so I might architect my solution the wrong way..

[ghost]

because I may not full understand swoole architecture so I might architect my solution the wrong way..

Absolutely no problem - you are welcome :blush:

Swoole is one of those teams which are helping as much as is possible in _issues_

There is a _slack channel_, too, but, maybe its a little bit harder to structure such a question there.

i'm switching my head to pure structure synchronous programming to this brave new world

Swoole has implemented their coroutines - you can write synchronous code. No async is needed in future. Think of coroutines as very lightweight threads; but instead of using some of them (which will block due to IO) you can now handle thousands (and far more) of those _threads_, starting on a one-core-CPU ;)

It's just a little bit hard to give an answer, because your code left questions about some things.

If i would build this in this case, i now would start with a new base, first, connecting to the app and talk between swoole and app. If this is done i would work on the other side and talk between swoole and other side. If both communications work your way, you can connect :)

With those communications on both sides we can better understand which way you can go in swoole. But, the app API/TCP and the other API is unknown this moment. You can give sample pieces of communication for example :)

it come across my mind also that it behave pretty much like web socket.. but I do not have control over the device (the appliance) that connect to my broker. I only have control on the other side where accepting http request with json content.

I have never worked on such a _broker_... So, i guess, there are several kinds of apps and softwarw connecting to you? Where is this communication API/PROTO defined?

[pheonics]

may be a bit of sweet picture can help ...

this is what i'm trying to build.. the broker server is in the middleware between device and api server.
the device is TCP client and it only talk to fixed IP/Port (which my broker is waiting for connection).
The device have it is own network protocol that I can't change.
The tcp must be kept persistent at all time so when mobile app want device service it can instruct device via my broker.
beside command to control device.. the device also have heart beat and broker also have to send keepalive to keep TCP active.

[pheonics]

the command on the device is primitive so the http request may not directly map 1-1 with device command. so from time to time the abstract service exposed through http request mey composed of multiple device commands..

[yellow1912]

https://konghq.com/kong-community-edition/

How about this for your issue?

[pheonics]

this is why I think i need one http server and one tcp server.. i have to architect the broker app so that all TCP connection may be in form of fd (file descriptor) are maintained and can be located and use the correct one when http request demand service from specified device.

SWOOLE_BASE vs SWOOLE_PROCESS

i think i want single thread (hence SWOOLE_BASE) as it is simpler to keep all fd in one memory space with http server.
TCP(device).on.connect -> negotiate/authenticate with device. collect device identity and keep fd into collection ___ set timer so each connected device will have its own timer that will wake up to do housekeeping task.
TCP(device).on.receive -> paring the request command and perform necessary thing.
HTTP(broker).on.request -> parse json and perform corresponding device commands.

this the above approach is correct?

[pheonics]

https://konghq.com/kong-community-edition/

How about this for your issue?

interesting comment sir. as i understand it also a good middleware and its API gateway. basically its http router on steroid .. but can it keep tcp connection on one side and route the request via correct tcp session?

[ghost]

You give very good explanation and great pic.

Lastly, there are many questions which can be relatively easy answered, but all those answers will give new questions.

Personally i think a good working prototype is done in some weeks if you are new to swoole and have to look at some parts of documentation. Lastly, that is very good bare metal stuff you can easily do with swoole as perfect engine for that.

Do you know this documents: https://wiki.swoole.com/wiki/index/prid-1

Here it is easy for you to get all answers you need about it, for writing prototype and optimize.

[pheonics]

@flddr i read doc on https://www.swoole.co.uk/docs/ but it does not give my concept that i need .. this is more like syntax ref to me.. i will definitely checkout wiki that you suggested.
thank you...

[ghost]

@pheonics oh no, this is explaining a lot

I think, if we would post some code here you would probably work this way for longer time to find out this is not your way of doing this.

I feel it would be better if you get a bigger overall overview.

When start reading here, you give very good explanations, but every explanation gives new questions and i think, nearly all of them are answered there in a good sorted manner.

I am sure you will get this working next weeks 👍

[doubaokun]

@pheonics since there is no update for a long time, I am closing this issue now.

If you still have questions, please join the Slack channel for general questions, community members may answer your questions: https://docs.google.com/forms/d/e/1FAIpQLScqpkPJeFG-ss3P-KhFECDSNLyo1iKmU1DToIQX9YRWbuetjw/viewform