Fluentd-kubernetes-daemonset: Windows support

[AndrewBoklashko]

FluentD supports Windows as of version 0.14, they also provide windows Docker images.
Kubernetes supports Windows nodes as of version 1.14.

Is that possible to add a DaemonSet setup for windows nodes?

[k1nger]

+1. Looking into this at the moment right now. Doesn't seem like it would be that difficult and could maybe contribute back to it if it's not in the pipeline of things to get done?

[k1nger]

I created a local branch with some changes that at least has it working. Removed some stuff that was local to what i needed and made comments in the yaml file around what specifically would be needed:

https://github.com/k1nger/fluentd-windows-daemon

This is pretty specific to pushing to cloudwatch but it is working on windows nodes across eks.

Also something to note: the windows base image the daemonset pulls from is specific to the version of the windows server host. So that might need to be potentially changed depending on which windows server version you are running. I removed an edit to that to get the proper build and not sure if i put that back to what it was originally.

[leandroandrade-hotmart]

Thanks @k1nger !!!
In wich repository is stored the fluentd-windows image?

[k1nger]

https://github.com/fluent/fluentd-docker-image

You can see in https://github.com/fluent/fluentd-docker-image/blob/master/v1.9/windows/Dockerfile

That fluentd is pulling from mcr.microsoft.com/windows/servercore:1903 which is what I was speaking to earlier. We had to change it to 1809 because that's what our host was running.

[leandroandrade-hotmart]

I a similar situation, but i need create in the servercore:ltsc2019.
I pulled the code and tryed change only the FROM step, and after i tryed build a image:
docker build -t fluend-windows . but the docker stucks on *** Removed specs cache *** :(

[k1nger]

I've had that issue on _one_ of my build machines. If you have access to another machine maybe try it on that?

You could potentially remove line 22 in the dockerfile cause I think that might be the offending line. I don't know the exact cause of that issue though.

[bgsilvait]

Same here...

Remove the line 22 and the "\" on line 21. But now stuck on
"Bundled gems are installed into ./vendor/bundle"

WIndows version:
OS Name: Microsoft Windows Server 2019 Datacenter
OS Version: 10.0.17763 N/A Build 17763

docker version:
Client: Docker Engine - Enterprise
Version: 19.03.5
API version: 1.40
Go version: go1.12.12
Git commit: 2ee0c57608
Built: 11/13/2019 08:00:16
OS/Arch: windows/amd64
Experimental: false

Server: Docker Engine - Enterprise
Engine:
Version: 19.03.5
API version: 1.40 (minimum version 1.24)
Go version: go1.12.12
Git commit: 2ee0c57608
Built: 11/13/2019 07:58:51
OS/Arch: windows/amd64
Experimental: false

@k1nger Can you confirm your versions ?

[k1nger]

The windows base image I am using is:
FROM mcr.microsoft.com/windows/servercore:1809

Building the image on:
OS Name: Microsoft Windows Server 2019 Standard
OS Version: 10.0.17763 N/A Build 17763

docker version
Client: Docker Engine - Enterprise
Version: 19.03.1
API version: 1.40
Go version: go1.12.5
Git commit: f660560464
Built: 07/25/2019 20:59:52
OS/Arch: windows/amd64
Experimental: false

Server: Docker Engine - Enterprise
Engine:
Version: 19.03.1
API version: 1.40 (minimum version 1.24)
Go version: go1.12.5
Git commit: f660560464
Built: 07/25/2019 20:57:41
OS/Arch: windows/amd64
Experimental: false

[bgsilvait]

I create a repo (https://github.com/bgsilvait/k8s-fluentd-windows) for using Fluentd as daemonset and stream logs to CloudWatch using IRSA to avoid Access Key/Secrect key or get Roles from Nodes.

I recently adjusted the repo to have for 2 EKS Supported Images WS2019 and WS1909.

[aneeshcts]

I created a local branch with some changes that at least has it working. Removed some stuff that was local to what i needed and made comments in the yaml file around what specifically would be needed:

https://github.com/k1nger/fluentd-windows-daemon

This is pretty specific to pushing to cloudwatch but it is working on windows nodes across eks.

Also something to note: the windows base image the daemonset pulls from is specific to the version of the windows server host. So that might need to be potentially changed depending on which windows server version you are running. I removed an edit to that to get the proper build and not sure if i put that back to what it was originally.

can we use this to send the logs to an external Elasticsearch ?