Goaccess: Docker-compose : Goaccess cannot find log

[rachid-debu]

Hi there,

Can't find why this is not working.

If i uncomment the command part for goaccess in my docker-compose file i get this error below :

Parsing... [719821] [31296/s][0/s]
goaccess    | GoAccess - version 1.3 - Jul  3 2019 12:51:29
goaccess    | Config file: /srv/data/goaccess.conf
goaccess    | 
goaccess    | Fatal error has occurred
goaccess    | Error occurred at: src/parser.c - read_log - 2728
goaccess    | Unable to open the specified log file. No such file or directory
goaccess    | 

If I keep it commented (like I saw @coldfix was doing here https://github.com/allinurl/goaccess/issues/1042#issuecomment-371940111), I get the man goaccess displayed ^^.

Here's my docker-compose (simplified, removed completely unrelated stuff) :

version: '3.5'
services:

  nginx:
    container_name: nginx
    image: nginx:alpine
    ports:
      - 80:80
      - 443:443
    volumes:
      - ${PWD}/nginx/nginx.conf:/etc/nginx/conf.d/default.conf
      - ${PWD}/nginx/logs:/var/log/nginx
    depends_on:
      - doyouno-api
      - goaccess
    networks:
      - nginx-network

  doyouno-api:
    image: ${DOYOUNO_REGISTRY_URL}/doyouno/api:${DOYOUNO_API_VERSION}
    hostname: api
    container_name: doyouno-api
    ports:
      - 2000:2000
    volumes:
      - ${PWD}/doyouno-api/logs:/api/src/logs
      - ${PWD}/doyouno-api/creds:/api/src/creds
    env_file: .env.api
    restart: always
    networks:
      - nginx-network

  goaccess:
    image: ${DOYOUNO_REGISTRY_URL}/allinurl/goaccess:${GOACCESS_VERSION}
    hostname: goaccess
    container_name: goaccess
    ports:
      - 7890:7890
    volumes:
      - ${PWD}/goaccess/html:/srv/report:rw
      - ${PWD}/nginx/logs:/srv/logs:ro
      - ${PWD}/goaccess/data:/srv/data:ro
    depends_on:
      - doyouno-api
    restart: always
    #command: goaccess /srv/logs/goaccess.log --config-file=/srv/data/goaccess.conf --output=/srv/report/index.html --log-file=/srv/logs/goaccess.log --real-time-html
    networks:
      - nginx-network

networks:
  nginx-network:
    driver: bridge
    name: nginx-network

I don't really know how to show you my folder structure, here's a screen eventhough we don't see everything. Let me know if need more infos (folder/files permissions for example).

Best regards
(and thanks for this awesome project)

[rachid-debu]

While I was posting this, I realized the nginx/logs folder was owned by root. So I changed it to my user (I kept the -rw-r--r-- rights though). still not working, sadly

[rachid-debu]

Oooow... I've just read the Dockerfile of the goaccess repo I've cloned (according to instructions here : https://hub.docker.com/r/allinurl/goaccess/)

Figures out the command of the Dockerfile is --help, here's the end of the file :

# Container
FROM busybox:musl
COPY --from=builds /dist /
VOLUME /var/www/goaccess
EXPOSE 7890
ENTRYPOINT ["/bin/goaccess"]
CMD ["--help"]

From what I read in issues, @coldfix showed a Dockerfile (https://github.com/allinurl/goaccess/pull/795#issuecomment-306211507) in which the command made more sense :
CMD ["goaccess", "--no-global-config", "--config-file=/srv/data/goaccess.conf"]

This would explain why I get the man page displayed when I don't specify any command in my docker-compose file.

Is there something I'm missing ?

[allinurl]

@rachid-debu thanks for reporting this. Let me forward this to @err931 as he wrote the Dockerfile. @err931 Do you know what might be the issue here? Thanks

[rachid-debu]

@rachid-debu thanks for reporting this. Let me forward this to @err931 as he wrote the Dockerfile. @err931 Do you know what might be the issue here? Thanks

Thanks. I still didn't manage to make this work with docker

[allinurl]

Maybe @gam2046 may be able to chime in about this issue?

[gam2046]

@allinurl #1507 It has been merged, but the image of Docker Hub has not been rebuilt.

@rachid-debu Therefore, you can simply replace allinurl/goaccess with gam2046/goaccess to confirm whether the same problem still exists. (This problem has nothing to do with GeoIP, ignore it.)

It seems that this problem is caused by wrong permissions or paths. Therefore, you can enter the container(docker exec -it <container-name-or-id> /bin/sh or docker-compose exec <container-name> /bin/sh) to check whether the files in the /srv/logs directory are correct.

• busybox whoami (in container)
• ls -lah /srv/logs (in container)
• cat /srv/logs/access.log (in container)

[allinurl]

@gam2046 it's built now. Please check again.

[allinurl]

@rachid-debu did you get a chance to look at @gam2046 comments. Any updates? Thanks

[rachid-debu]

looks like an interesting fix thanks a lot.
I will try this out asap, got a lot of work right now. I'll give feedback in 1-2 days

[Nykoshiro]

Not sure if this was fixed or not, but I no longer can docker compose GoAccess. It was working when binding /srv/{data,logs,report}. I noticed the dockerfile was changed (e.g. no more /srv/ volumes?). The previous file dockerfile worked without an issue with the following config:

Docker for Windows (Linux Containers)

`---
version: "3.7"

services:
goaccess:
image: allinurl/goaccess
container_name: goaccess
networks:
- compose_custom
environment:
- TZ=America/Los Angeles
volumes:
- D:/Docker/GoAccess/data:/srv/data
- D:/Docker/GoAccess/report:/srv/report
- D:/Docker/shared/logs:/srv/logs
restart: always`

[allinurl]

@Nykoshiro, @err931 just merged some changes upstream. Can you please try again and let me know how it goes. Thanks.

[Nykoshiro]

@allinurl @err931 I can, give me a bit and I'll let you guys know as soon as I can.

I may just not understand the new changes to the DockerFile but the documentation references /srv/{data,logs,report} volumes, the rewrite of the DockerFile on 5/12 removes those volumes and sets /var/www/goaccess. The latest change is still only referencing the /var/www/goaccess volume, so I'm assuming it still isn't fixed?

I'm learning this stuff as I go so be patient if I don't understand what I'm talking about or the changes... I'll still give the container a shot as soon as I have a few minutes.

[rachid-debu]

@allinurl #1507 It has been merged, but the image of Docker Hub has not been rebuilt.

~@rachid-debu Therefore, you can simply replace allinurl/goaccess with gam2046/goaccess to confirm whether the same problem still exists.~ (This problem has nothing to do with GeoIP, ignore it.)

It seems that this problem is caused by wrong permissions or paths. Therefore, you can enter the container(docker exec -it <container-name-or-id> /bin/sh or docker-compose exec <container-name> /bin/sh) to check whether the files in the /srv/logs directory are correct.

• busybox whoami (in container)
• ls -lah /srv/logs (in container)
• cat /srv/logs/access.log (in container)

I tried again with the latest image of goaccess.
Still not working.
Problem is, I wish I could /bin/sh into the container but it dies instantly. So i cannot sh to it :/..

Regarding permissions I doubled checked but I might be wrong.
My user is admin, the folder containing all my configuration is located in /home/admin/docker. Here are some screens of file permissions, and part of my docker-compose.yml file :

nginx:
    container_name: nginx
    image: nginx:alpine
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./nginx/letsencrypt/etc:/etc/letsencrypt
      - ./nginx/letsencrypt/var/lib:/var/lib/letsencrypt
      - ./nginx/letsencrypt/var/log:/var/log/letsencrypt
      - ./nginx/letsencrypt/dhparam.pem:/etc/letsencrypt/dhparam.pem
      - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf
      - ./nginx/webapp:/nginx/webapp
      - ./nginx/passwd:/nginx/passwd
      - ./goaccess/html:/nginx/goaccess
      - ./nginx/logs:/var/log/nginx
    depends_on:
      - webapp
      - goaccess
    networks:
      - nginx-network
[...]
webapp:
    image: ${REGISTRY_URL}/webapp:${WEBAPP_VERSION}
    container_name: webapp
    ports:
      - 8100:8100
    restart: always
    networks:
      - nginx-network

  goaccess:
    image: allinurl/goaccess:${GOACCESS_VERSION}
    hostname: goaccess
    container_name: goaccess
    ports:
      - 7890:7890
    volumes:
      - ./goaccess/html:/srv/report:rw
      - ./nginx/logs:/srv/logs:ro
      - ./goaccess/data:/srv/data:ro
    depends_on:
      - api
    command: goaccess --config-file=/srv/data/goaccess.conf --output=/srv/report/index.html --log-file=/srv/logs/access.log --real-time-html
    networks:
      - nginx-network

[...]

As you can see, the files and folder I am mapping are ./goaccess/html, ./nginx/logs, ./goaccess/data, here are the file permissions :

And the error that I get (indeed, it looks like file permissions or path, but I don't know.. everything looks fine) :

Parsing... [108260] [27065/s]
GoAccess - version 1.3 - Aug 20 2019 15:58:53
Config file: /srv/data/goaccess.conf

Fatal error has occurred
Error occurred at: src/parser.c - read_log - 2728
Unable to open the specified log file. No such file or directory

Thanks for the help

[rachid-debu]

@allinurl @err931 I can, give me a bit and I'll let you guys know as soon as I can.

I may just not understand the new changes to the DockerFile but the documentation references /srv/{data,logs,report} volumes, the rewrite of the DockerFile on 5/12 removes those volumes and sets /var/www/goaccess. The latest change is still only referencing the /var/www/goaccess volume, so I'm assuming it still isn't fixed?

I'm learning this stuff as I go so be patient if I don't understand what I'm talking about or the changes... I'll still give the container a shot as soon as I have a few minutes.

I think you are right, this change https://github.com/allinurl/goaccess/commit/5a13cfa7e976b060184637e05b4f69a9403186de#diff-3254677a7917c6c01f55212f86c57fbfR60 seems to have broken paths.

[Nykoshiro]

If the Docker Hub build isn't updated at the same time as GitHub then I suspect that the Docker Hub build Docker Hub Commit Source ddf17b0 from 22 days ago update is what finally sync'd the May 12th rewrite. I've been rebuilding containers while trying to learn compose / swarm / kube and that's around the same time when I noticed that my GoAccess container stopped creating the report.html.

Edit: fixed URL for Docker Hub Builds

[allinurl]

Latest build ~3hr ago. Please give it a shot.

[rachid-debu]

Latest build ~3hr ago. Please give it a shot.

For me, the fact that it does build was not the problem. It does build for me. Just it didn't find the files, I thought it was permissions.

But looking at the change in volume paths, I just followed the new /goaccess/<access.log, goaccess.conf, report.html> path, so I just put all 3 needed files in a volume /goaccess, and it works, it prints the report.

Edit: Here's my new docker-compose with proper paths, working.

goaccess:
    image: allinurl/goaccess:latest
    hostname: goaccess
    container_name: goaccess
    ports:
      - 7890:7890
    volumes:
      - ./goaccess:/goaccess:rw
    depends_on:
      - api
    command: goaccess --config-file=/goaccess/goaccess.conf --output=/goaccess/index.html --log-file=/goaccess/access.log --real-time-html
    networks:
      - nginx-network

I consider my issue solved, hopefully my solution will help others. Also maybe documentation could be updated accordingly (it mentions everywhere /srv/{data,report,log}).

[Nykoshiro]

I was able to finally get GoAccess running referencing @rachid-debu compose config. Take away notes:

a) the docker documentation now conflicts with current config of GoAccess (/GoAccess/) volumes; previously it created /srv/{data,logs,report} volumes. Can /GoAccess/{data,logs,report} be added?

b) unless there is a way for Docker Compose to create directories (e.g. /GoAccess/{data,logs,report}) then there is a manual effort to create these paths but only after configuring GA to a temp path? or I'm forced to use a single volume (see new compose config below) and rework all my other containers to point to the new 'shared' GoAccess path

version: "3.7"

#Docker for Windows (Linux Containers)

services:
#5
    goaccess:
        image: allinurl/goaccess
        container_name: goaccess
        networks:
          - mybridge
        environment:
          #- PUID=1000
          #- PGID=1000
          - TZ=America/Los_Angeles
        volumes:
          #- /var/run/docker.sock:/var/run/docker.sock

          #  GoAccess Bind Mounts prior to 5/12
          #- c:/docker/goaccess/data:/srv/data
          #- c:/docker/webserver/sites/goaccess/:/srv/report
          #- c:/docker/shared/variouslogs:/srv/logs

          #  GoAccess Bind Mount after 5/12
          - c:/docker/shared/goaccess:/goaccess:rw
        command:
          #goaccess
          goaccess --config-file=/goaccess/goaccess.conf --output=/goaccess/index.html --log-file=/goaccess/access.log --real-time-html
        #ports:
        #  - 7890:7890
        restart: always
networks:
      mybridge:
        driver: bridge

[Nykoshiro]

@rachid-debu I thought you mentioned something about ws-url: if I recall correctly, the web socket configuration is used for something else, not self hosting the report (e.g. html) file; I believe you have to host this file with your own webserver.

[rachid-debu]

@Nykoshiro I thought I needed it in order to have live report. Goaccess is inside docker and i'm using Nginx inside docker which serves the goaccess and my websites. I wanted to have to access.log regularly refreshed and the report also.

@allinurl isn't it the purpose of the ws-url ?

[allinurl]

By default, goaccess will attempt to connect to the generated report's hostname. If GoAccess is running on a remote server, the host of the remote server should be specified using --ws-url=<hostname>.

@rachid-debu @Nykoshiro can I ask you to please submit a PR to update the README accordingly. I just want to make sure I don't miss anything. Thank you very much both of you!

[Nykoshiro]

@allinurl I can try to do that, I just need to look up what submitting a PR entails...
@rachid-debu I was curious, is your container running GoAccess updating the html file in real-time?

I was able to get the container to run using the config above (using new paths) but it seems that the log file is only parsed one time and that's it. I've tried my original conf file & also using various command lines but for whatever reason, ever since GA was updated I haven't been able to successfully get it going.

[rachid-debu]

@Nykoshiro yes it does work. At first it was not working because the websocket tunnel didn't work but It now works. Detail a bit your configuration, maybe I can help. Are you using a nginx server ? What's your conf ?

[mskec]

I'm also experiencing problems with the new Dockerfile#5a13cfa.

Website docs https://goaccess.io/download#docker and README instructions are not updated to use the new Dockerfile.

docker run --restart=always -d -p 7890:7890 \
  -v "/srv/goaccess/data:/srv/data"         \
  -v "/srv/goaccess/html:/srv/report"       \
  -v "/var/log/apache2:/srv/logs"           \
  --name=goaccess allinurl/goaccess

This command used to work. Now it's not working and only outputs --help. Additionally, volume mounting for data, report and logs will not work because they are removed in Dockerfile.

I managed to get it to work using the old file. I cloned the repo, pasted the old Dockerfile and built the image.

@err931 can you please update README with how to use the current Dockerfile?

[rachid-debu]

@mskec indeed, path has changed and documentation has not been updated yet.

I suggest you try this :

docker run --restart=always -d -p 7890:7890 \
  -v "/srv/goaccess/data/goaccess.conf:/goaccess/goaccess.conf"         \
  -v "/srv/goaccess/html/index.html:/goaccess/index.html"       \
  -v "/var/log/apache2/access.log:/goaccess/access.log"           \
  --name=goaccess allinurl/goaccess:latest goaccess --config-file=/goaccess/goaccess.conf --output=/goaccess/index.html --log-file=/goaccess/access.log --real-time-html

If your html file is named different that index.html feel free to edit.
Also make sure that inside your goaccess.conf you either comment out the log_file and output or update them according to the command goaccess --config-file=/goaccess/goaccess.conf --output=/goaccess/index.html --log-file=/goaccess/access.log --real-time-html. If you edit the goaccess.conf you obviously don't need to specify anymore those 2 options.

[mskec]

@rachid-debu thank you 🎉 that works with the latest Dockerfile

[rachid-debu]

@err931 @allinurl PR #1545 please review

[yi-huan]

The following configuration work:

version: '3'

services:
  goaccess:
    image: allinurl/goaccess
    ports:
      - 7890:7890
    volumes:
      - ./goaccess/config:/srv/config
      - ./goaccess/data:/srv/data
      - ./goaccess/html:/srv/report
      - /var/log/nginx:/srv/logs
    command: ["--no-global-config", "--config-file=/srv/config/goaccess.conf", "--log-file=/srv/logs/access.log"]

My configuration（goaccess.conf） file is in ./goaccess/config.

At least need to be set in goaccess.conf:
eg. my conf (nginx):

time-format %H:%M:%S
date-format %d/%b/%Y
log-format COMBINED
real-time-html true
output /srv/report/index.html

Can input HTML file, but can't preview in real time, access error, http status code is 400, how do you configure it?
I configured ws-url, it still doesn't work
ws-url work

[allinurl]

README has been updated. Please confirm new instructions solve the issue. Thanks!

[allinurl]

Closing this. Feel free to reopen it if needed.