Ohara: Add a flush size definition to FTP sink connector just like HDFS

Created on 21 Aug 2019  ·  7Comments  ·  Source: oharastream/ohara

All 7 comments

/cc @eechih @chia7712
Let me apply this feature for our RM 😄

@wu87988622 Does it need to be backport to 0.7?

@wu87988622 Does it need to be backport to 0.7?

according to @wu87988622 replied, it is not required feature for 0.7 since it doesn't affect the normal process :)

the I/O buffer, now, is controlled by csv/text task. And the "flush size" should NOT belong to ftp connector. Instead, the setting should be exposed by super class (csv/text connector).

However, there is no csv/text connector yet (and it is addressed by #2319). for 0.8, we should wait for the #2319 to complete the arch for csv/text connector. By contrast, we can hard-code it for 0.7 if @wu87988622 like this improvement.

FYI @eechih

Thanks for the explanation, In that case, I think there is nothing to do for this issue until #2319 done. I will pending this issue.

@wu87988622 @saivirtue @chia7712 我預計在 v0.8 #2319 處理這個問題

Thanks @eechih !
I will close this issue now.

Was this page helpful?
0 / 5 - 0 ratings