Sitespeed.io: Store results as json file in S3 Storage

[thebennos]

I have found an extension to send the metric to influxdb or graphite, but what about a more simpler solution like sending / saving results in a s3 storage?

[soulgalore]

Hey @thebennos yes we been talking about it but lets see when we can fix that.

Best
Peter

[soulgalore]

Lets take this on the coming weeks. I can start doing a simple version following what we did on run.sitespeed.io and then @beenanner you can help me when you have time?

[beenanner]

Absolutely

[thebennos]

can help to test it, but not deep enough in the project to help with coding.

[soulgalore]

Got a working version up in the S3 branch that is testable.

• I'm not sure if we are missing any configuration, should we set a cache time?
• The directory structure on S3: If everything is default config we upload "sitespeed-result" as base dir, but maybe we should go down one level and instead have the domain-name as first folder?

[thebennos]

Little confused, because there is no S3 branch, but found it in the master.

"I'm not sure if we are missing any configuration, should we set a cache time?"
S3 has not cache settings, so you can only mean caching on the sitespeed.
Pretty unsure about, the effects if caching is set or not.

"The directory structure on S3: If everything is default config we upload "sitespeed-result" as base dir, but maybe we should go down one level and instead have the domain-name as first folder?"

domainname as folder sounds better, then it is good structured and not confusing.

I try to test your commit in the next days.

[soulgalore]

Yep it's been merged but the CLI options is hidden in master (poor man's way of making sure the feature isn't released :)

It was http://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#putObject-property CacheControl that I was referring to, not sure what it does?

domainname as folder sounds better, then it is good structured and not confusing.

yes I thought so too but it will be little more confusing when making links between Grafana/S3 and increase the risk of somethings goes bad when changing the path.

[kkopachev]

@soulgalore Regarding key/secret passed as runtime parameters. It might be more valuable if parameters not provided through cli, just don't use any hard-coded credentials. That lets AWS to use default CredentialProviderChain which in turn will look for environment variables AWS_* and AMAZON_*, ini file credentials, but more important it'll be able to use ECS or EC2 credentials (if run on AWS infrastructure).

Additionally would be great to add ability to specify ACL: 'ACL' => 'public-read'.

Looks like s3Options object does not have to have any options to work.

CacheControl sets cache-control header for an object when it's served from s3. Useful when you have CDN pointing to S3 and using these cache-control settings to expire internal CDN's cache.

[beenanner]

@kkopachev thanks for the info and recommendations. I tested the CredentialProviderChain and it works as you suggested. Maybe we move the cli parameters to optional? While I agree that the CredentialProviderChain is probably preferred we would want to keep the CLI parameters for easy of use for beginners I think. :-) What do you think @soulgalore?

[soulgalore]

released