Containers-roadmap: ECR [request]: Add support for scheduled scans of container images

[jicowan]

Tell us about your request
Scheduled scans of container images stored in ECR

Which service(s) is this request for?
ECR

Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?
Trying to identify the containers I've deployed that have developed vulnerabilities.

Are you currently working around this issue?
Creating a timed Lambda function that performs an on-demand scan of the image and querying the API the a summary of the vulnerabilities discovered.

[jtoberon]

We expected this to be one of the most common asks! It would be helpful to know how you want to control this:

What frequency and timing do you have in mind? Which image(s) in a repository do you want to scan regularly?

Presumably, frequency and timing can be captured in a cron-like expression. The second question is more complicated. Many customers can make a simplifying assumption that we should rescan the image that was pushed most recently. But, I can imagine a number of scenarios where a user would want to have more control than this; for example, maybe you want to rescan the image that you're actually running in production.

[rlafferty]

@jtoberon - if metrics are available about which metrics have been pulled most recently it may be nice to keep a running tally of scanning the top N images in a repo that have been pulled in the last N days based on number of pulls.

[nunofernandes]

@jtoberon Let me then explain how I'm currently doing things... Scenario: ECR+ECS (fargate+ec2)
I have a daily script job running in my own CI/CD that will:

1- get a list of all services in ECS
2 - get the task definition being used by those services
3 - get all the docker images in those task definitions
4 - send that information to our internal image scanner (not important for this case). If issue is found fail the job and alert in slack.
5 - get the describe-image-scan-findings from AWS to that image (something like)

    assume-role "$AWS_ECR_ROLE"                   \
        aws ecr describe-image-scan-findings      \
            --registry-id "$REGISTRY_ID"          \
            --repository-name "$REPOSITORY_NAME"  \
            --image-id "imageTag=$IMAGE_TAG"

Check if it has a security issue regarding our policies and whitelists.
6 - Request a new scan for that specific image (for tomorrows' run)

If any point any issue is found I send an alert to slack. In this case, I'm assuming if a security issue is found (severity > threshold), then call some lambda...

[mwarkentin]

+1 to the above - native integration with ECS would be nice since it knows which images are in play.

One other use case which wouldn't be captured by this would be scheduled scanning of shared base images. Scanning all of the images running in ECS would catch the vulnerabilities in the base image, but it wouldn't be clear if they were introduced in the base image or during the application's build.

This could be configured by either configuring a list of images to scan via API, or maybe there would be a way to check if any of the images we're using have a FROM declaration whose image also lives in our ECR, if so scan that as well.

[kuntalakrishna]

Having this feature in-built on ECR would prove very effective. We can schedule an automated scan at the fixed intervals on the containers which are currently in use on ECS to know whether any new vulnerabilities have been detected on those images.

[antoinedeschenes]

Is rescanning actually required? Since Clair is able to find new vulnerabilities without rescanning, is ECR also finding new vulnerabilities automatically? (And sending the appropriate event?)

[mwarkentin]

@antoinedeschenes do you have a link to more details on how that works with clair? Is this because clair is a server, rather than run as a one-off?

[antoinedeschenes]

@mwarkentin Exactly, it inventories package versions found in docker image layers in it's database, and then compares package versions to the CVE databases. Once the inventory is done, there's no point in rescanning an image again. Anchore works in a similar way, while Trivy needs to rescan the image on every update.

https://github.com/quay/clair README:

1. In regular intervals, Clair ingests vulnerability metadata from a configured set of sources and stores it in the database.
2. Clients use the Clair API to index their container images; this creates a list of features present in the image and stores them in the database.
3. Clients use the Clair API to query the database for vulnerabilities of a particular image; correlating vulnerabilities and features is done for each request, avoiding the need to rescan images.
4. When updates to vulnerability metadata occur, a notification can be sent to alert systems that a change has occurred.