Ohara: add group to all objects

Created on 4 Jul 2019  ·  6Comments  ·  Source: oharastream/ohara

from #1546


    • [x] enable DataStore to save group and name (#1927)


    • [x] rename "FileKey" to "DataKey" (#1928)


    • [x] replace RocksDataStore.Key by DataKey (#1948)


    • [x] add basic group handle to basic route (#1950)


    • [x] add group to ftp route (#1952)


    • [x] add group to hdfs route (#1953)


    • [x] add group to jdbc route (#1962)


    • [x] add group to pipeline route (#1964)


    • [x] add group to stream route (@saivirtue ) (#2071)


    • [x] add group to response of topic (#1965)


    • [x] add group to response of connector (#1966)


    • [x] change the flow of pipeline from "string" to "DataKey" (#2002)


    • [x] the url generator should be cognizant of group and name (#2003)


    • [x] make ClusterInfo be sub class of Data (#2006)


    • [x] enable group for Topic (#2011)


    • [x] add metadata "TOPIC_KEYS" type (#2016)


    • [x] enable group for Connector (#2029)


    • [x] enable default group in parsing "ObjectKey" and "TopicKey" (#2056)


    • [x] move group from query parameter to payload of Creation (#2057)


    • [x] move "default group" to each APIs (#2061)


    • [x] Remove name-related methods from DataStore (#2074)


    • [x] The Client APIs should substitute Key for name (#2077)


    • [x] add group to Object APIs (#2096)

v0.7.0

Most helpful comment

@konekoya @wu87988622 除了cluster相關的服務外(zk, bk, wk, streamapp and shabondi)其他物件的操作都已經上group了,這邊需要前端配合在查詢的時候搭配group來查詢相關物件的資料。

我們後續的PR需要等到前端採用group後才可以繼續進行。此外以下是group例外的總和:

  1. node不支援group
  2. topic的group"目前"是default (我們最終會將其改成跟brokerClusterName相同)
  3. connector的group"目前"是default (我們最終會將其改成跟workerClusterName相同)
  4. zk不支援group
  5. bk不支援group
  6. wk不支援group
  7. streamapp"目前"不支援group
  8. rules (pipeline)只支援使用name 所以要改用flows

All 6 comments

@konekoya @wu87988622 除了cluster相關的服務外(zk, bk, wk, streamapp and shabondi)其他物件的操作都已經上group了,這邊需要前端配合在查詢的時候搭配group來查詢相關物件的資料。

我們後續的PR需要等到前端採用group後才可以繼續進行。此外以下是group例外的總和:

  1. node不支援group
  2. topic的group"目前"是default (我們最終會將其改成跟brokerClusterName相同)
  3. connector的group"目前"是default (我們最終會將其改成跟workerClusterName相同)
  4. zk不支援group
  5. bk不支援group
  6. wk不支援group
  7. streamapp"目前"不支援group
  8. rules (pipeline)只支援使用name 所以要改用flows

I plan to change the location of group in Create request.

before

POST /v0/$rs?group=${g0}
{
  "name": "n0"
}

after

POST /v0/$rs
{
  "group": "g0",
  "name": "n0"
}

(the group is ignorable)

The merit of later is that the URL of Create differs from URL of GET.

@oharastream/developer any feedbacks ?

I plan to change the location of group in Create request.

before

POST /v0/$rs?group=${g0}
{
  "name": "n0"
}

after

POST /v0/$rs
{
  "group": "g0",
  "name": "n0"
}

(the group is ignorable)

The merit of later is that the URL of Create differs from URL of GET.

@oharastream/developer any feedbacks ?

Since our ohara "dataKey" current is the combination of
The later is imply "oh, you should use these two parameters to create your object, or we will generate for you"
it's more reasonable, so I agree the change :)

by the way, I think except the "POST" method, the rest url should be the same, right?

GET /v0/$rs/{name}?group=xxx
PUT /v0/$rs/{name}?group=xxx
DELETE /v0/$rs/{name}?group=xxx
PUT /v0/$rs/{name}/start?group=xxx
PUT /v0/$rs/{name}/stop?group=xxx
PUT /v0/$rs/{name}/{nodeName}?group=xxx

by the way, I think except the "POST" method, the rest url should be the same, right?

In fact, I prefer to make the “write” request have different url from “read” request because we are going to add more “query parameters” to read request. And it will mislead users about the usage of query parameters on write request :(

For example, the URL of GET request should be /v0/$rs?name=a&x=a. Also, getting single object should be deprecated.

Furthermore, group is not always “default” in some objects. For example, connector group must be equal to worker cluster name. And hence group is not optional to PUT and DELETE. It it worth of moving the group from query parameter to url.

Finally, all tasks are completed!!!

reopen since I neglect Object APIs

Was this page helpful?
0 / 5 - 0 ratings