Skywalking: Clarify the `Observe service mesh through ALS` steps in README

[XuejiaoZhang]

Please answer these questions before submitting your issue.

• Why do you submit this issue?
• [ ] Question or discussion
• [ ] Bug
• [ - ] Requirement
• [ ] Feature or performance improvement

---

Requirement or improvement

• Please describe your requirements or improvement suggestions.
  Clarify the Observe service mesh through ALS steps in README

The current description is confusing, would you please clarify the steps.
It mentionns

2. Open SkyWalking envoy receiver.

3. Active ALS k8s-mesh analysis

envoy-metric:
  default:
    alsHTTPAnalysis: "k8s-mesh"

a) please clarify how to open envoy receiver, I saw envoy-metric section is already in application.yaml
b) please clarify how to Active ALS k8s-mesh analysis, where should the following be put?

envoy-metric:
  default:
    alsHTTPAnalysis: "k8s-mesh"

c) from skywalking helm, I saw configuration named oap.envoy.als.enabled, is this required to set to true in values.yaml

Thanks in advance. This will definitely help a lot to integrate Skywalking with Istio.

[wu-sheng]

You are mixing two things together. For application.yml, this is the standard and official way when you deploy SkyWalking from the tarball.
And helm config is providing a quick way. Basically, they are the same.
If you read the documents about how to understand the application.yml, https://github.com/apache/skywalking/blob/v8.1.0/docs/en/setup/backend/backend-setup.md#applicationyml, this gives you the explanation.

[wu-sheng]

Helm is just a quick way, it just covers 70%~ features which you could do through the real application.yml. That is also the same we(Apache) only call the source release is the official release. The binary and docker image are never the release.

[XuejiaoZhang]

Thanks for answering. I totally understand the Helm config just exposes part of parameters of application.yaml.
Currently I am trying to find a way to deploy skywalking above k8s to integrate with Istio, I believe this would not be an unusual use case. Do you mean that the following steps of Observe service mesh through ALS need to be done through application.yaml not via the helm way?

2. Open SkyWalking envoy receiver.
3. Active ALS k8s-mesh analysis

[wu-sheng]

I mean, helm should have done these two steps.

[XuejiaoZhang]

Probably it's not the case, or I may miss something.
when I run docker container locally with image apache/skywalking-oap-server:8.1.0-es7 which is used in the helm chart, I would see the settings of 2. Open SkyWalking envoy receiver. as following in the config/application.yaml

envoy-metric:
  selector: ${SW_ENVOY_METRIC:default}
  default:
    acceptMetricsService: ${SW_ENVOY_METRIC_SERVICE:true}
    alsHTTPAnalysis: ${SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS:""}

But didn't see the settings of Active ALS k8s-mesh analysis

envoy-metric:
  default:
    alsHTTPAnalysis: "k8s-mesh"

Do you have any clue about this? Thanks.

[wu-sheng]

Didn't helm set SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS = k8s-mesh? @innerpeacez

[wu-sheng]

@XuejiaoZhang If you are using docker compose only, you need to set the system env value by yourself.

[XuejiaoZhang]

@wu-sheng I run docker locally just to check whether helm meets the requirements of integration with Istio or not.
Then SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS env variable needs to be set in skywalking-kubernetes/chart/skywalking/templates/oap-deployment.yaml.

[wu-sheng]

Send your pull request if this is missed. We could talk more there.

[XuejiaoZhang]

Sure, thanks a lot.

[hanahmily]

@wu-sheng @XuejiaoZhang ALS relevant parameters are not in the core. In order to support them, helm chart has introduced https://github.com/apache/skywalking-kubernetes/blob/d383403a58fefa901b03bcdb925db402a17e8718/chart/skywalking/values.yaml#L61 to set them up.

[innerpeacez]

HELM CHARTS:
oap-deployment.yaml

You may also need to pay attention to the templates/istio-adapter/* files

[wu-sheng]

@innerpeacez From my understanding, SW_ENVOY_ALS_ENABLED system env has not been required anymore. Could you check the application.yml inside the image?

envoy-metric:
  selector: ${SW_ENVOY_METRIC:default}
  default:
    acceptMetricsService: ${SW_ENVOY_METRIC_SERVICE:true}
    alsHTTPAnalysis: ${SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS:""}

This has been included, right?

[innerpeacez]

Checked that this configuration exists @wu-sheng

[wu-sheng]

Then, I think SW_ENVOY_ALS_ENABLED should be removed. Because the docker entry script doesn't use this anymore.

It used to be like this, https://github.com/apache/skywalking/blob/v6.6.0/docker/oap/docker-entrypoint.sh#L380-L382, but with the 8.x enhancement, it is simply like this, https://github.com/apache/skywalking/blob/v8.1.0/docker/oap/docker-entrypoint.sh

[innerpeacez]

Ok i will remove it. @wu-sheng

[XuejiaoZhang]

Could anyone update either skywalking or skywalking-kubernetes document to include the steps of integrate Skywalking with Istio through ALS via the helm install way?
This would definitely help lots of people who are interested to use Skywalking and Istio together. Thanks.

[wu-sheng]

Corrently, only set SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS = k8s-mesh.

[XuejiaoZhang]

No luck with the approach we discussed, would anyone please demo/validate it?

My testing in the following is based on Istio 1.7 and latest skywalking-kubenetes.

1. EnvoyAccessLog is enabled.
   
   Also, access logs could be seen in the testing app called client here, access logs are also output to stout, we could see the following running kubectl logs ...

2. helm install skywalking with env variable set SW_ENVOY_METRIC_ALS_HTTP_ANALYSIS = k8s-mesh.
   Run kubectl exec to login skywalking-oap pod
   

Issue: nothing showed on skywalking UI

Here are logs of skywalking-oap

Any clue? Thanks.

[wu-sheng]

This error shows, you can't read the pod inform from the API server. cc @hanahmily

[hanahmily]

@XuejiaoZhang Please follow https://gist.github.com/hanahmily/b4acf4ae0bbd9d742357279a668bd924

[XuejiaoZhang]

@hanahmily Thanks. I followed the instructions. The "APM" Dashboard of SkyWalking was showed correctly, but nothing on the Tracing page, while the tracing data could be seen on.Jaeger. With the current method, nothing is needed to be done on the application code side, right?

[wu-sheng]

I think that is another thing. Tracing must have something related to the codes, even Javaagent(auto-instrumentation) is doing the thing in the runtime only.

ALS solution is not for tracing, tracing is for the language agent case. You could install agent/SDK in your application, and close the analysis option in the OAP. Then you have the trace on the page.

[XuejiaoZhang]

Oh, I was expecting Observe service mesh including tracing info with agent auto installed and configured. Ok, got it.
From GO2Sky, we configure GO2Sky to export to OAP server, which is listening on oap-skywalking port 11800, does that mean, the agent method is not Envoy based tracing, instead it instruments and exports tracing info to OAP server all by the application itself? With the ENVOY-BASED tracing, Envoy will generates request IDs and trace headers and sends the generated trace spans to the tracing backends.

[wu-sheng]

Oh, I was expecting Observe service mesh including tracing info with agent auto installed and configured. Ok, got it.

Technically impossible, sorry :)

we configure GO2Sky to export to OAP server, which is listening on oap-skywalking port 11800, does that mean, the agent method is not Envoy based tracing, instead it instruments and exports tracing info to OAP server all by the application itself? With the ENVOY-BASED tracing, Envoy will generates request IDs and trace headers and sends the generated trace spans to the tracing backends.

Envoy tracing is tracing the things inside the envoy. It doesn't replace the code instrumentation.

Then envoy tracing progress is lead by Envoy core maintainers, https://github.com/envoyproxy/envoy/issues/12486. You could show you interests there, which should accelerate the progress

[XuejiaoZhang]

Yup, thanks, will followup there.