Harbor: Enable external security scanners by adding the `scanner-pull` permission to a robot account

[danielpacak]

Problem statement

When a Harbor project is set as private and there's deployment security enabled some security tools, such as external vulnerability scanners, cannot pull images from the Harbor registry.

Only internal vulnerability scanners can bypass deployment security checks with a temporary robot account created by Harbor, which has the scanner-pull permission.

For example, Aqua CSP, allows you to add Harbor registry integration where you're supposed to enter user or robot account credentials:

However, these credentials do not have enough permissions (i.e. scanner-pull) to pull the images for security assessment when the deployment security flag is on, hence the scans fail:

In general, the problem described above, boils down to distinguish a docker pull requests, which are subject to deployment security, from docker pull requests initiated by trusted 3rd parties (e.g. external security scanners) that are supposed to bypass the security checker.

For more detailed explanation of 412 status code returned by Harbor consult https://github.com/danielpacak/danielpacak.io/blob/master/harbor/registry_authentication.adoc

Possible solution

One solution to the problem might be to add a scanner-pull permission to a robot account. This would allow creating dedicated robot accounts for integrating with external security scanners.

We should be able to add robot accounts with the scanner-pull permission via Harbor API as well.
Ideally we could also specify the expiry date for such credentials.

Architecture

There's no change in overall architecture of Harbor and its authorization service. We need to modify the logic of the token service to cater for the scanner-pull permission.

Development plan

I think this problem impacts Harbor 1.10.x deployments and upcoming 2.x releases. We'd like to address the limitation ASAP. We're available to implement the change in code and test the functionality.

/cc @lizrice @jerbia @xaleeks @michmike

[reasonerjt]

exposing scan-pull on UI can be confusing to user as it's internal mechanism.
If we call it scan it is still confusing, because user may think they can use this robot account to trigger a scan.

Can we support creating such robot account in API?
We may also consider adding more attribute to the robot account also solve this one
https://github.com/goharbor/harbor/issues/11175

[rvennam-lbg]

@danielpacak we hit the same issue when using interrogation services with Aqua where the images are not being pulled due to the pre-condition failed on a new image push which was not even scanned.

[steven-zou]

exposing scan-pull on UI can be confusing to user as it's internal mechanism.
If we call it scan it is still confusing, because user may think they can use this robot account to trigger a scan.

Can we support creating such robot account in API?
We may also consider adding more attribute to the robot account also solve this one

11175

If only supporting this requirement in API, it will break the regular operations via management consoles. Meanwhile, the API way is not very friendly to platform operators.

[steven-zou]

Or explicitly add an option for robot account when creation to let admin aware of the case that if they'd like to use this robot account into the scanner platform they need to check that option to let the account acquire the capabilities of bypassing VULN checking.

@xaleeks @danielpacak @reasonerjt

[reasonerjt]

@steven-zou

If you want reasonable UI flow, you should allow users to create account for scanner, it should happen on the scanner registration page.
Currently, the robot account is created under project view it will only be valid for one project.

[danielpacak]

@danielpacak we hit the same issue when using interrogation services with Aqua where the images are not being pulled due to the pre-condition failed on a new image push which was not even scanned.

@rvennam-lbg Yes, I'm aware of the current limitations in Harbor & Aqua. We're trying to find the best way to enhance the robot account generation to cater for such cases. Do you have any specific requirements / preferences whether such robot account should have project / registry scope that you might share with us?

[honeybajaj]

@danielpacak We have multiple projects on Harbor that integrates with an Aqua instance. So, registry scope would be simpler for our usecase, but we can live with Project level robot accounts as well if that is simpler to implement.

[danielpacak]

Taking into account all the feedback that we get so far, I'd sum up the requirements for the external scanner credentials as follows:

1. An external security scanner needs credentials to search tags and pull images stored in the registry

   

   

2. The credentials should have registry scope (not project-level scope as is the case for robot accounts)

3. The credentials should have read-only permissions. We do not expect security scanners to modify registry artifacts
4. The credentials should bypass deployment security checks, i.e. have the scanner-pull permission as is the case for temporary robot accounts generated for internal scanners
5. The credentials do not have a TTL. TTL sounds appealing and secure, but it's problematic and requires external security scanners to implement mechanisms for refreshing access tokens. This would probably require changes in Harbor token service, i.e. adding support for refresh tokens
6. A Harbor admin should be able to disable or revoke the credentials if needed. For example, in case of security breach in the external scanner
7. The credentials should be auditable, i.e. each image pull request with the credentials should be logged in Harbor audit log
8. A Harbor admin should be able to generated the credentials in the Harbor interface as well as with the Harbor API
9. The credentials can be generated for an external scanner that does not have an internal scanner adapter. For example, we see many deployments where Harbor is integrated with Aqua CSP without deploying Harbor Scanner adapter for Aqua

@steven-zou @reasonerjt @michmike @xaleeks @zhill @honeybajaj @rvennam-lbg Taking into account the above listed requirement, one more possibility is to add a new view called External Scanners under the Interrogation Services to manage such credentials:

I'm looking forward to seeing your comments.

[zhill]

@danielpacak This seems to me like the time to consider a 'Service Account' type that is a like a robot account but with explicitly managed credentials. Similar to what you propose, but not tied to scanning, that would just be a specific use-case. I think 'Interrogation Services' is a reasonable place to put it, but would call the section 'Service Accounts' instead of 'External Scanners', that gives flexibility as more interrogations become available.

It would give a place to add more per-credential config like project vs registry scope, check bypasses, etc later on.

[michmike]

thanks for the summary everyone. this is a good feature to have.

Essentially we are:

1. creating a service account concept under system settings (I assume this will be its own item on the left hand side navigation tree)
2. only harbor system admin can view or create them or delete them
3. they will be scoped to all projects
4. they will be read-only privilege for pull operations only
5. they can use pull or pull without policy permissions (a feature toggle)

[danielpacak]

thanks for the summary everyone. this is a good feature to have.

Essentially we are:

1. creating a service account concept under system settings (I assume this will be its own item on the left hand side navigation tree)
2. only harbor system admin can view or create them or delete them
3. they will be scoped to all projects
4. they will be read-only privilege for pull operations only
5. they can use pull or pull without policy permissions (a feature toggle)

Hey @michmike what about access to registry catalog? Would it be possible for such service account to list all repositories / tags in the registry?

[michmike]

@danielpacak makes sense to include that. @xaleeks what do you think?

[xaleeks]

Yep we can do that, this is still leveraging our current robot account in the background, so we can add those scopes to the jwt

[danielpacak]

To keep everyone posted we met with @xaleeks and @steven-zou and our conclusion was as follows:

"...There seems to be little utility in building global robot accounts scoped to all projects (would have been all or nothing right now) with a special set of permissions just for a specific class of scanners including aqua CSP, without a clear picture of how to extend that work to the final goal of being able to dynamically add / remove robot accounts out of projects. So that robots can behave like true service accounts with rbac, and with more differentiated access levels. So we are abandoning this temp solution for 2.1 and aiming to deliver this in full in v2.2..."

Long story short we identified too many corner cases and too many opened questions to deliver such functionality without considering upcoming enhancement to Harbor authorization model and granular access controls to guard system functionality and settings

[xaleeks]

This will be covered in this proposal https://github.com/goharbor/community/pull/148

@danielpacak let me know if you see any issues