Harbor: Harbor Projects Are Very Slow to Load

[sharkymcdongles]

I wonder if maybe I have a misconfig somewhere or need to allocate more resources or enable a caching mechanism because when I go to the projects view it takes a long time to load. First it takes a long time to load the list of projects at /harbor/projects/ then when I click my projects and go to respositories e.g. /projects/2/repositories/ or helm charts e.g. /projects/2/helm-charts/.

Load times:
23.15s for main projects page: /harbor/projects/
20.96s for clicking through to my project: /projects/2/repositories/
21.33s for clicking through to my chart repo: /projects/2/repositories/

The main project has 335 Repositories and 366 Helm Charts.

Any idea how I can increase performance here?

Here are my chart values:

chartmuseum:
  absoluteUrl: false
  enabled: true
  image:
    repository: goharbor/chartmuseum-photon
    tag: v2.1.0
  nodeSelector:
    agentpool: stateful
  replicas: 1
clair:
  adapter:
    image:
      repository: goharbor/clair-adapter-photon
      tag: v2.1.0
  clair:
    image:
      repository: goharbor/clair-photon
      tag: v2.1.0
  enabled: true
  nodeSelector:
    agentpool: stateful
  replicas: 1
  updatersInterval: 12
core:
  image:
    repository: goharbor/harbor-core
    tag: v2.1.0
  nodeSelector:
    agentpool: stateful
  replicas: 1
  resources:
    requests:
      cpu: 300m
      memory: 8192Mi
  startupProbe:
    initialDelaySeconds: 10
database:
  internal:
    image:
      repository: goharbor/harbor-db
      tag: v2.1.0
    nodeSelector:
      agentpool: stateful
    password: censored
    resources:
      requests:
        cpu: 100m
        memory: 256Mi
    sslmode: disable
  maxIdleConns: 50
  maxOpenConns: 1000
  type: internal
expose:
  clusterIP:
    name: harbor
    ports:
      httpPort: 80
  ingress:
    annotations:
      ingress.kubernetes.io/proxy-body-size: 10000m
      ingress.kubernetes.io/ssl-redirect: "true"
      kubernetes.io/ingress.allow-http: "false"
      kubernetes.io/ingress.class: nginx
      kubernetes.io/tls-acme: "true"
      nginx.ingress.kubernetes.io/proxy-body-size: 10000m
      nginx.ingress.kubernetes.io/ssl-redirect: "true"
    controller: default
    hosts:
      core: harbor.x.com
  tls:
    certSource: secret
    enabled: true
    secret:
      notarySecretName: harbor.x.com
      secretName: harbor.x.com
  type: ingress
externalURL: https://harbor.x.com
harborAdminPassword: censored
imagePullPolicy: IfNotPresent
internalTLS:
  certSource: auto
  chartmuseum:
    crt: ""
    key: ""
    secretName: ""
  clair:
    crt: ""
    key: ""
    secretName: ""
  core:
    crt: ""
    key: ""
    secretName: ""
  enabled: false
  jobservice:
    crt: ""
    key: ""
    secretName: ""
  portal:
    crt: ""
    key: ""
    secretName: ""
  registry:
    crt: ""
    key: ""
    secretName: ""
  trivy:
    crt: ""
    key: ""
    secretName: ""
  trustCa: ""
jobservice:
  image:
    repository: goharbor/harbor-jobservice
    tag: v2.1.0
  jobLogger: stdout
  maxJobWorkers: 10
  replicas: 1
logLevel: debug
notary:
  affinity: {}
  enabled: false
  nodeSelector: {}
  podAnnotations: {}
  secretName: ""
  server:
    image:
      repository: goharbor/notary-server-photon
      tag: v2.1.0
    replicas: 1
    serviceAccountName: ""
  signer:
    image:
      repository: goharbor/notary-signer-photon
      tag: v2.1.0
    replicas: 1
    serviceAccountName: ""
  tolerations: []
persistence:
  enabled: true
  imageChartStorage:
    disableredirect: false
    gcs:
      bucket: harbor
      chunksize: "5242880"
      encodedkey: x
      rootdirectory: /
    type: gcs
  persistentVolumeClaim:
    chartmuseum:
      accessMode: ReadWriteOnce
      existingClaim: '-'
      size: 5Gi
      storageClass: ""
      subPath: ""
    database:
      accessMode: ReadWriteOnce
      existingClaim: ""
      size: 32Gi
      storageClass: fast
      subPath: ""
    jobservice:
      accessMode: ReadWriteOnce
      existingClaim: ""
      size: 8Gi
      storageClass: fast
      subPath: ""
    redis:
      accessMode: ReadWriteOnce
      existingClaim: ""
      size: 32Gi
      storageClass: fast
      subPath: ""
    registry:
      accessMode: ReadWriteOnce
      existingClaim: ""
      size: 5Gi
      storageClass: '-'
      subPath: ""
    trivy:
      accessMode: ReadWriteOnce
      existingClaim: ""
      size: 5Gi
      storageClass: fast
      subPath: ""
  resourcePolicy: keep
portal:
  image:
    repository: goharbor/harbor-portal
    tag: v2.1.0
  nodeSelector:
    agentpool: stateful
  replicas: 1
  resources:
    requests:
      cpu: 300m
      memory: 1Gi
proxy:
  components:
  - core
  - jobservice
  - clair
  - trivy
  httpProxy: null
  httpsProxy: null
  noProxy: 127.0.0.1,localhost,.local,.internal
redis:
  internal:
    image:
      repository: goharbor/redis-photon
      tag: v2.1.0
    nodeSelector:
      agentpool: stateful
    serviceAccountName: ""
  type: internal
registry:
  controller:
    image:
      repository: goharbor/harbor-registryctl
      tag: v2.1.0
  credentials:
    htpasswd: user_harbor:censored
    password: censored
    username: user_harbor
  middleware:
    enabled: false
  registry:
    image:
      repository: goharbor/registry-photon
      tag: v2.1.0
  relativeurls: false
  replicas: 1
secretKey: pass
trivy:
  affinity: {}
  debugMode: false
  enabled: true
  ignoreUnfixed: false
  image:
    repository: goharbor/trivy-adapter-photon
    tag: v2.1.0
  insecure: false
  nodeSelector: {}
  podAnnotations: {}
  replicas: 1
  resources:
    limits:
      cpu: 1
      memory: 1Gi
    requests:
      cpu: 200m
      memory: 512Mi
  severity: UNKNOWN,LOW,MEDIUM,HIGH,CRITICAL
  skipUpdate: false
  tolerations: []
  vulnType: os,library
updateStrategy:
  type: RollingUpdate

[sharkymcdongles]

I did some debug log hunting, and it seems part of the problem is it runs a helm repo update to download the index yaml for chartmuseum for every call to ths project. This seems pretty inefficient. Maybe the index yaml can be cached to redis on each chart push and then read from there?

[ninjadq]

could you provide logs of Harbor?

[ninjadq]

we recommend use oci-registry to storage the chart in the future.

[sharkymcdongles]

@ninjadq How can I enable oci-registry? I don't see an option in the helm chart. As for logs please see below:

2020-10-26T10:10:41Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 4dca566e3956f0b360eb38cf91fa6dd5 to the logger for the request GET /api/v2.0/systeminfo
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/systeminfo
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 93792b9b918c2ba5d34b3c3c85414bd2 to the logger for the request GET /api/v2.0/users/current
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/users/current
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/security/session.go:47][requestID="4dca566e3956f0b360eb38cf91fa6dd5"]: a session security context generated for request GET /api/v2.0/systeminfo
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/security/session.go:47][requestID="93792b9b918c2ba5d34b3c3c85414bd2"]: a session security context generated for request GET /api/v2.0/users/current
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id e8e583a1c7845ef8c92c94eeb1a5714b to the logger for the request GET /api/v2.0/configurations
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/configurations
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/security/session.go:47][requestID="e8e583a1c7845ef8c92c94eeb1a5714b"]: a session security context generated for request GET /api/v2.0/configurations
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 01e24be127f79442ad898d5f666970ef to the logger for the request GET /api/v2.0/statistics
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/statistics
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/security/session.go:47][requestID="01e24be127f79442ad898d5f666970ef"]: a session security context generated for request GET /api/v2.0/statistics
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 75891a35806a8e950e3e16142dbc8f23 to the logger for the request GET /api/v2.0/systeminfo/volumes
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/systeminfo/volumes
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id a7c2da26e572adf7d64cab1e02629538 to the logger for the request GET /api/v2.0/registries
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/registries?q=type=%7Bdocker-hub%20harbor%7D
2020-10-26T10:10:41Z [DEBUG] [/common/dao/project.go:198]: query sql:select count(1)  from project as p where p.deleted=false and p.project_id in ( ?,?,?,? )
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/security/session.go:47][requestID="75891a35806a8e950e3e16142dbc8f23"]: a session security context generated for request GET /api/v2.0/systeminfo/volumes
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/security/session.go:47][requestID="a7c2da26e572adf7d64cab1e02629538"]: a session security context generated for request GET /api/v2.0/registries
2020-10-26T10:10:41Z [DEBUG] [/common/dao/project.go:174]: query sql:select distinct p.project_id, p.name, p.registry_id, p.owner_id,
                                p.creation_time, p.update_time  from project as p where p.deleted=false and p.project_id in ( ?,?,?,? ) order by name
2020-10-26T10:10:41Z [DEBUG] [/common/dao/project.go:198]: query sql:select count(1)  from project as p where p.deleted=false
2020-10-26T10:10:41Z [DEBUG] [/common/dao/project.go:174]: query sql:select distinct p.project_id, p.name, p.registry_id, p.owner_id,
                                p.creation_time, p.update_time  from project as p where p.deleted=false order by name
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id b8ffed150c90de29f8a6b36649f65797 to the logger for the request GET /api/v2.0/projects
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/projects?page=1&page_size=15
2020-10-26T10:10:41Z [DEBUG] [/server/middleware/security/session.go:47][requestID="b8ffed150c90de29f8a6b36649f65797"]: a session security context generated for request GET /api/v2.0/projects
2020-10-26T10:10:41Z [WARNING] [/core/systeminfo/imagestorage/filesystem/driver.go:51]: The path /data is not found, will return zero value of capacity
2020-10-26T10:10:41Z [DEBUG] [/pkg/scan/allowlist/manager.go:71]: No CVE allowlist found for project 1, returning empty list.
2020-10-26T10:10:41Z [DEBUG] [/chartserver/handler_repo.go:166]: Getting index.yaml from 'http://harbor-harbor-chartmuseum/sre/index.yaml'
2020-10-26T10:10:41Z [DEBUG] [/chartserver/handler_repo.go:166]: Getting index.yaml from 'http://harbor-harbor-chartmuseum/cs/index.yaml'
2020-10-26T10:10:41Z [DEBUG] [/chartserver/handler_repo.go:166]: Getting index.yaml from 'http://harbor-harbor-chartmuseum/library/index.yaml'
2020-10-26T10:10:41Z [DEBUG] [/chartserver/handler_repo.go:166]: Getting index.yaml from 'http://harbor-harbor-chartmuseum/CENSORED/index.yaml'
2020-10-26T10:10:56Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 6e331530293c1f45949c7f3d3dc27982 to the logger for the request GET /api/health
2020-10-26T10:10:56Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/health
2020-10-26T10:10:56Z [DEBUG] [/server/middleware/security/unauthorized.go:29][requestID="6e331530293c1f45949c7f3d3dc27982"]: an unauthorized security context generated for request GET /api/health
2020-10-26T10:11:01Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id ded3c576cfbbeca0315553499167c030 to the logger for the request GET /v2/
2020-10-26T10:11:01Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/
2020-10-26T10:11:01Z [DEBUG] [/server/middleware/security/unauthorized.go:29][requestID="ded3c576cfbbeca0315553499167c030"]: an unauthorized security context generated for request GET /v2/
2020-10-26T10:11:01Z [DEBUG] [/lib/http/error.go:59]: {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized: unauthorized"}]}
2020-10-26T10:11:01Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id e67f35d90e065c442906f2fc9bc4f06e to the logger for the request GET /service/token
2020-10-26T10:11:01Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /service/token?account=robot%24gitlab&client_id=docker&offline_token=true&service=harbor-registry
2020-10-26T10:11:01Z [ERROR] [/server/middleware/security/oidc_cli.go:60][requestID="e67f35d90e065c442906f2fc9bc4f06e"]: failed to verify secret: failed to verify the secret: user does not exist, name: robot$gitlab
2020-10-26T10:11:01Z [DEBUG] [/server/middleware/security/robot.go:68][requestID="e67f35d90e065c442906f2fc9bc4f06e"]: a robot security context generated for request GET /service/token
2020-10-26T10:11:01Z [DEBUG] [/core/service/token/token.go:35]: URL for token request: /service/token?account=robot%24gitlab&client_id=docker&offline_token=true&service=harbor-registry
2020-10-26T10:11:01Z [DEBUG] [/core/service/token/creator.go:203]: scopes: []
2020-10-26T10:11:01Z [DEBUG] [/core/service/token/authutils.go:48]: scopes: []
2020-10-26T10:11:01Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id d933beaed4898a38aadac0e3e57b481f to the logger for the request GET /v2/
2020-10-26T10:11:01Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/
2020-10-26T10:11:02Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id da948a3a4e371b1291899602d2d123c1 to the logger for the request HEAD /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:11:02Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:11:02Z [DEBUG] [/server/middleware/security/unauthorized.go:29][requestID="da948a3a4e371b1291899602d2d123c1"]: an unauthorized security context generated for request HEAD /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:11:02Z [DEBUG] [/lib/http/error.go:59]: {"errors":[{"code":"UNAUTHORIZED","message":"authorize header needed to send HEAD to repository: authorize header needed to send HEAD to repository"}]}
2020-10-26T10:11:02Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id e1e55256104bfffeddbc4c746053881f to the logger for the request POST /service/token
2020-10-26T10:11:02Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /service/token
2020-10-26T10:11:02Z [DEBUG] [/server/middleware/security/unauthorized.go:29][requestID="e1e55256104bfffeddbc4c746053881f"]: an unauthorized security context generated for request POST /service/token
2020-10-26T10:11:02Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 2884c4d0ae4e43e4541677275e9b6a64 to the logger for the request GET /service/token
2020-10-26T10:11:02Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /service/token?scope=repository%3ACENSORED%2Fcluster-status%3Apull&service=harbor-registry
2020-10-26T10:11:02Z [ERROR] [/server/middleware/security/oidc_cli.go:60][requestID="2884c4d0ae4e43e4541677275e9b6a64"]: failed to verify secret: failed to verify the secret: user does not exist, name: robot$gitlab
2020-10-26T10:11:02Z [DEBUG] [/server/middleware/security/robot.go:68][requestID="2884c4d0ae4e43e4541677275e9b6a64"]: a robot security context generated for request GET /service/token
2020-10-26T10:11:02Z [DEBUG] [/core/service/token/token.go:35]: URL for token request: /service/token?scope=repository%3ACENSORED%2Fcluster-status%3Apull&service=harbor-registry
2020-10-26T10:11:02Z [DEBUG] [/core/service/token/creator.go:203]: scopes: [repository:CENSORED/cluster-status:pull]
2020-10-26T10:11:02Z [DEBUG] [/core/service/token/authutils.go:48]: scopes: [repository:CENSORED/cluster-status:pull]
2020-10-26T10:11:02Z [DEBUG] [/core/service/token/authutils.go:95]: user: robot$gitlab, access: &{repository  CENSORED/cluster-status [push * pull]}
2020-10-26T10:11:02Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 2a87b9fa784db1bb2c4337adffbc0720 to the logger for the request HEAD /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:11:02Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:11:03Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 4eceb08d6212270c35ecd5bda66bb234 to the logger for the request HEAD /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:11:03Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:11:13Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 0e54a7ec084ef429f63c54111ac364d1 to the logger for the request GET /v2/
2020-10-26T10:11:13Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/
2020-10-26T10:11:13Z [DEBUG] [/server/middleware/security/unauthorized.go:29][requestID="0e54a7ec084ef429f63c54111ac364d1"]: an unauthorized security context generated for request GET /v2/
2020-10-26T10:11:13Z [DEBUG] [/lib/http/error.go:59]: {"errors":[{"code":"UNAUTHORIZED","message":"unauthorized: unauthorized"}]}
2020-10-26T10:11:13Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id b1031678452d42d5de7cd75b4906e3dc to the logger for the request GET /service/token
2020-10-26T10:11:13Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /service/token?account=robot%24gitlab&client_id=docker&offline_token=true&service=harbor-registry
2020-10-26T10:11:13Z [ERROR] [/server/middleware/security/oidc_cli.go:60][requestID="b1031678452d42d5de7cd75b4906e3dc"]: failed to verify secret: failed to verify the secret: user does not exist, name: robot$gitlab
2020-10-26T10:11:13Z [DEBUG] [/server/middleware/security/robot.go:68][requestID="b1031678452d42d5de7cd75b4906e3dc"]: a robot security context generated for request GET /service/token
2020-10-26T10:11:13Z [DEBUG] [/core/service/token/token.go:35]: URL for token request: /service/token?account=robot%24gitlab&client_id=docker&offline_token=true&service=harbor-registry
2020-10-26T10:11:13Z [DEBUG] [/core/service/token/creator.go:203]: scopes: []
2020-10-26T10:11:13Z [DEBUG] [/core/service/token/authutils.go:48]: scopes: []
2020-10-26T10:11:13Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id edef7d3e7d60ec9c88f30e25bdc0de8b to the logger for the request GET /v2/
2020-10-26T10:11:13Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/
2020-10-26T10:11:13Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 2a0613f37e8fbb7200582d0e503a54a0 to the logger for the request GET /api/v2.0/projects/2
2020-10-26T10:11:13Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/projects/2
2020-10-26T10:11:13Z [DEBUG] [/server/middleware/security/session.go:47][requestID="2a0613f37e8fbb7200582d0e503a54a0"]: a session security context generated for request GET /api/v2.0/projects/2
2020-10-26T10:11:13Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2
2020-10-26T10:11:13Z [DEBUG] [/chartserver/handler_repo.go:166]: Getting index.yaml from 'http://harbor-harbor-chartmuseum/CENSORED/index.yaml'
2020-10-26T10:11:39Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 06341c4566f5e5b8b6ab2f5169ca5456 to the logger for the request GET /api/v2.0/users/current/permissions
2020-10-26T10:11:39Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/users/current/permissions?scope=/project/2&relative=true
2020-10-26T10:11:39Z [DEBUG] [/server/middleware/security/session.go:47][requestID="06341c4566f5e5b8b6ab2f5169ca5456"]: a session security context generated for request GET /api/v2.0/users/current/permissions
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/member
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/member
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/log
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/label
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/label
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/quota
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/repository
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/repository
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require pull action for resource /project/2/repository
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/helm-chart
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/helm-chart
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/helm-chart-version
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/helm-chart-version
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/configuration
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/robot
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/robot
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/scan
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/scanner
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/tag
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/artifact
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/artifact
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/artifact-addition
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require update action for resource /project/2
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/member
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require update action for resource /project/2/member
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/member
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/metadata
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/metadata
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require update action for resource /project/2/metadata
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/metadata
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/label
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require update action for resource /project/2/label
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/label
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/repository
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require update action for resource /project/2/repository
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/repository
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require push action for resource /project/2/repository
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/tag-retention
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/tag-retention
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require update action for resource /project/2/tag-retention
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/tag-retention
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/tag-retention
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require operate action for resource /project/2/tag-retention
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/immutable-tag
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require update action for resource /project/2/immutable-tag
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/immutable-tag
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/immutable-tag
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/helm-chart
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/helm-chart
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/helm-chart-version
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/helm-chart-version
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/helm-chart-version-label
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/helm-chart-version-label
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require update action for resource /project/2/configuration
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/robot
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require update action for resource /project/2/robot
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/robot
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/notification-policy
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require update action for resource /project/2/notification-policy
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/notification-policy
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/notification-policy
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/notification-policy
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/scan
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/scanner
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/artifact
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/artifact
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/tag
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/tag
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/artifact-label
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/artifact-label
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require create action for resource /project/2/preheat-policy
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2/preheat-policy
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require update action for resource /project/2/preheat-policy
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require delete action for resource /project/2/preheat-policy
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/preheat-policy
2020-10-26T10:11:39Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id ad3d8252e7dd625db7a4b5ece05d5ff6 to the logger for the request GET /api/v2.0/projects/2
2020-10-26T10:11:39Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/projects/2
2020-10-26T10:11:39Z [DEBUG] [/server/middleware/security/session.go:47][requestID="ad3d8252e7dd625db7a4b5ece05d5ff6"]: a session security context generated for request GET /api/v2.0/projects/2
2020-10-26T10:11:39Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require read action for resource /project/2
2020-10-26T10:11:39Z [DEBUG] [/chartserver/handler_repo.go:166]: Getting index.yaml from 'http://harbor-harbor-chartmuseum/CENSORED/index.yaml'
2020-10-26T10:11:51Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 845bdd1036bed10c69097a38c844ea45 to the logger for the request HEAD /v2/CENSORED/gira/manifests/1.0.2
2020-10-26T10:11:51Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/CENSORED/gira/manifests/1.0.2
2020-10-26T10:11:51Z [DEBUG] [/server/middleware/security/unauthorized.go:29][requestID="845bdd1036bed10c69097a38c844ea45"]: an unauthorized security context generated for request HEAD /v2/CENSORED/gira/manifests/1.0.2
2020-10-26T10:11:51Z [DEBUG] [/lib/http/error.go:59]: {"errors":[{"code":"UNAUTHORIZED","message":"authorize header needed to send HEAD to repository: authorize header needed to send HEAD to repository"}]}
2020-10-26T10:11:51Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 7215462180589bcb0486607747af604e to the logger for the request GET /service/token
2020-10-26T10:11:51Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /service/token?scope=repository%3ACENSORED%2Fgira%3Apull&service=harbor-registry
2020-10-26T10:11:51Z [DEBUG] [/server/middleware/security/unauthorized.go:29][requestID="7215462180589bcb0486607747af604e"]: an unauthorized security context generated for request GET /service/token
2020-10-26T10:11:51Z [DEBUG] [/core/service/token/token.go:35]: URL for token request: /service/token?scope=repository%3ACENSORED%2Fgira%3Apull&service=harbor-registry
2020-10-26T10:11:51Z [DEBUG] [/core/service/token/creator.go:203]: scopes: [repository:CENSORED/gira:pull]
2020-10-26T10:11:51Z [DEBUG] [/core/service/token/authutils.go:48]: scopes: [repository:CENSORED/gira:pull]
2020-10-26T10:11:51Z [DEBUG] [/core/service/token/authutils.go:95]: user: , access: &{repository  CENSORED/gira [pull]}
2020-10-26T10:11:51Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 77bb5a99a2c69ff1212d7234004a3eaa to the logger for the request HEAD /v2/CENSORED/gira/manifests/1.0.2
2020-10-26T10:11:51Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/CENSORED/gira/manifests/1.0.2
2020-10-26T10:11:51Z [DEBUG] [/lib/http/error.go:59]: {"errors":[{"code":"NOT_FOUND","message":"repository CENSORED/gira not found"}]}
2020-10-26T10:11:56Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 27089b967ff7d1824f634d0f34bcb1a1 to the logger for the request GET /api/health
2020-10-26T10:11:56Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/health
2020-10-26T10:11:56Z [DEBUG] [/server/middleware/security/unauthorized.go:29][requestID="27089b967ff7d1824f634d0f34bcb1a1"]: an unauthorized security context generated for request GET /api/health
2020-10-26T10:12:03Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 1ac3a1e84d07dafd40b32f8845b2f193 to the logger for the request HEAD /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:12:03Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:12:03Z [DEBUG] [/server/middleware/security/unauthorized.go:29][requestID="1ac3a1e84d07dafd40b32f8845b2f193"]: an unauthorized security context generated for request HEAD /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:12:03Z [DEBUG] [/lib/http/error.go:59]: {"errors":[{"code":"UNAUTHORIZED","message":"authorize header needed to send HEAD to repository: authorize header needed to send HEAD to repository"}]}
2020-10-26T10:12:03Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 52cd536cbcc9db7bc28a57abc3f3870f to the logger for the request POST /service/token
2020-10-26T10:12:03Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /service/token
2020-10-26T10:12:03Z [DEBUG] [/server/middleware/security/unauthorized.go:29][requestID="52cd536cbcc9db7bc28a57abc3f3870f"]: an unauthorized security context generated for request POST /service/token
2020-10-26T10:12:03Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id bea624778ddd59c61beb3d704894abd1 to the logger for the request GET /service/token
2020-10-26T10:12:03Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /service/token?scope=repository%3ACENSORED%2Fcluster-status%3Apull&service=harbor-registry
2020-10-26T10:12:03Z [ERROR] [/server/middleware/security/oidc_cli.go:60][requestID="bea624778ddd59c61beb3d704894abd1"]: failed to verify secret: failed to verify the secret: user does not exist, name: robot$gitlab
2020-10-26T10:12:03Z [DEBUG] [/server/middleware/security/robot.go:68][requestID="bea624778ddd59c61beb3d704894abd1"]: a robot security context generated for request GET /service/token
2020-10-26T10:12:03Z [DEBUG] [/core/service/token/token.go:35]: URL for token request: /service/token?scope=repository%3ACENSORED%2Fcluster-status%3Apull&service=harbor-registry
2020-10-26T10:12:03Z [DEBUG] [/core/service/token/creator.go:203]: scopes: [repository:CENSORED/cluster-status:pull]
2020-10-26T10:12:03Z [DEBUG] [/core/service/token/authutils.go:48]: scopes: [repository:CENSORED/cluster-status:pull]
2020-10-26T10:12:03Z [DEBUG] [/core/service/token/authutils.go:95]: user: robot$gitlab, access: &{repository  CENSORED/cluster-status [push * pull]}
2020-10-26T10:12:03Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 980c7866a23b5cee6a35ca3a5625fc5e to the logger for the request HEAD /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:12:03Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:12:04Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id ea14e880b29d0f8c0fddbd92a5d90b44 to the logger for the request HEAD /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:12:04Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /v2/CENSORED/cluster-status/manifests/0.0.59
2020-10-26T10:12:05Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id a0801b274171c16dfc8910a531310d73 to the logger for the request GET /api/v2.0/systeminfo
2020-10-26T10:12:05Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/systeminfo
2020-10-26T10:12:05Z [DEBUG] [/server/middleware/log/log.go:30]: attach request id 1f2b788375af2f3c7b748bd0823aafab to the logger for the request GET /api/v2.0/projects/CENSORED/repositories
2020-10-26T10:12:05Z [DEBUG] [/server/middleware/artifactinfo/artifact_info.go:52]: In artifact info middleware, url: /api/v2.0/projects/CENSORED/repositories?page_size=15&page=1
2020-10-26T10:12:05Z [DEBUG] [/server/middleware/security/session.go:47][requestID="a0801b274171c16dfc8910a531310d73"]: a session security context generated for request GET /api/v2.0/systeminfo
2020-10-26T10:12:05Z [DEBUG] [/server/middleware/security/session.go:47][requestID="1f2b788375af2f3c7b748bd0823aafab"]: a session security context generated for request GET /api/v2.0/projects/CENSORED/repositories
2020-10-26T10:12:05Z [DEBUG] [/server/v2.0/handler/util.go:91]: field RepositoryName not found in ListRepositoriesParams
2020-10-26T10:12:05Z [DEBUG] [/pkg/permission/evaluator/admin/admin.go:32]: system administrator CENSORED require list action for resource /project/2/repository

[bitsf]

there's no option needed, just use helm3 to push chart to the project, you will see the artifact type is helm chart which is stored with oci-registry api.
and you don't need install chartmuseum component which is for helm 2, and will be deprecated in future.

[sharkymcdongles]

@bitsf inside of the helm chart I see this:

      # Define which storage backend is used for registry and chartmuseum to store
      # images and charts. Refer to
      # https://github.com/docker/distribution/blob/master/docs/configuration.md#storage
      # for the detail.
      imageChartStorage:
        # Specify whether to disable `redirect` for images and chart storage, for
        # backends which not supported it (such as using minio for `s3` storage type), please disable
        # it. To disable redirects, simply set `disableredirect` to `true` instead.
        # Refer to
        # https://github.com/docker/distribution/blob/master/docs/configuration.md#redirect
        # for the detail.
        disableredirect: true
        # Specify the "caBundleSecretName" if the storage service uses a self-signed certificate.
        # The secret must contain keys named "ca.crt" which will be injected into the trust store
        # of registry's and chartmuseum's containers.
        # caBundleSecretName:

        # Specify the type of storage: "filesystem", "azure", "gcs", "s3", "swift",
        # "oss" and fill the information needed in the corresponding section. The type
        # must be "filesystem" if you want to use persistent volumes for registry
        # and chartmuseum
        #

It says there it uses chartmuseum explicitly with no mention of oci. Are you saying if I set Values.chartmuseum.enabled to false and upgrade the chart it will then pull from the storage container directly with no chartmuseum in between?

[sharkymcdongles]

I have removed all charts from harbor now. I also had to kill the redis index yaml cache. Now projects load instantaneously almost. I will see if I can figure out how to get OCI working as it isn't documented anywhere nor is it documented in the helm repo.

[sharkymcdongles]

I tried OCI and it seems to work better finally, but given how OCI is experimental and not even finalized in Helm itself, I think addressing the underlying problem of chartmuseum making the UI unusable should be addressed. What if there was no chartmuseum update call for project loads instead or this could be toggled to a cronjob style approach where it only updated every so often to prevent the slow loads?

[agill17]

Running into the same issues with just 10 helm charts but several versions of each and 50 repos... this makes the UI unusable ...

[turnham]

Since upgrading to 2.1 we are seeing what appears to be the same issue. We only have a handful of helm charts (we are not using OCI with Helm 3 because it is still considered Experimental and we need to deliver Helm Charts to our end users). In some cases the main Project Page takes >30 seconds to load and once in a while ends with this:

[sharkymcdongles]

So the problem is every single call to the project related endpoints forces a helm repo update call to chartmuseum to get a fresh index yaml instead of caching it. This means no matter how many charts you have it will cause problems. The solution would be to store the index yaml inside of redis and serve the index yaml that way then have a backend cron job or scheduled job with a configurable refresh timer to not have this call on every page load. This will solve everything while also making chartmuseum usable. Because using container registry storage for charts isn't an option while it is still experimental.

[agill17]

Another thing I noticed is that the index-cache.yaml stored in s3 bucket ( because my backend storage is s3 ), harbor never cleans up index.yaml after deleting a chart from harbor. This overtime makes the index.yaml huge and running a helm repo update on each call to the project on a large enough index.yaml slows operation down by a lot !

[sharkymcdongles]

I think this should no longer be tagged as pending more info needed since the problem is clear as are the potential solutions. @ninjadq