Meteor-feature-requests: Fix Atmosphere Data Problems

[msavin]

Currently, Atmosphere has issues displaying trending data and package stats. Additionally, package stats are not very good. They could be more in-depth, allowing you to view downloads by day, week, month, direct vs indirect, etc.

Fastosphere used to be really nice and fast in these regards, but it looks like it had some issues connecting to the Packages API. Either way, to signal a healthy eco-system its important that the package system looks operational.

With a bit of polish, I think it can serve as a really good replacement for Atmosphere.

[mitar]

I think it would be really great if Atmosphere would be open sourced. Then community members could contribute such fixes to it.

[abernix]

@msavin

Currently, Atmosphere has issues displaying trending data and package stats.

Are you only referring to the flat-lined graph, or are there other actually broken aspects?

[msavin]

Well - I'm monitoring app installs and they are totally off.

For example, this package definitely has more than 0 downloads:
https://atmospherejs.com/meteor/bundle-visualizer

Additionally, Meteor can display a lot more data such as:

• direct downloads (via meteor add)
• indirect downloads (as a dependency)
• determine updates vs installs
• break down downloads by day, week, month, year

Bonus points for:

• keep track of unique users vs total downloads
• ability to view analytics from date X to Y

Meanwhile - the line graphs - they really have no meaning as we do not know how they are tracked or what they represent.

There's a lot more potential here.. I guess a lot of it depends on how invested Meteor remains in its own packaging system. Personally, its one of my favorite things. At minimum, I hope its properly maintained.

As for open sourcing Atmosphere, would be nice if only to fix it up. Maybe integrate it with ElasticSearch to improve speeds, query abilities, etc. I do think the UI is a bit over done, but that's another story.

[lorensr]

There are trending packages with 0 downloads:

[hwillson]

@msavin - regarding the line graphs, @tmeasday talked about this a while back on the forums:

As the person who implemented the line chart on Atmosphere I can let everyone in on a secret--it pretty much always looks the same. The line is graphing Atmosphere Score which is a freshness index based on how popular it is (mainly based on downloads) vs how long it's been since the last release.

It probably isn't the best thing to graph (especially over a week), because almost always it just trends downwards.

[msavin]

So, it does sound like its an overall bad design.

On another note, the download counts on the squares are very different than the download counts on the pages.

[abernix]

There was an actual problem with the Atmosphere stats aggregation server. I think things should be working again now (meaning there shouldn't be any trending packages with zero downloads 🙈), but it'll take a couple days for some of the historical data to show back up.

Basically, things should be back to how they were – though I can't say other concerns about the stats have been addressed.

[msavin]

Nice! I just checked stats for a few packages, and it was nice to see the download stats on Meteor Candy and etc.

I think, just making sure that the current atmosphere works as it should would be good. I did notice one discrepancy though - between the download numbers on the boxes and the pages.

Edit: misread "it'll take a couple days for some of the historical data to show back up" - will wait and see

[abernix]

I'm really not sure those numbers will re-align and they weren't matching before either. I haven't had a chance to figure out exactly why they are different though I do think the relative differences (detail page vs. detail page or homepage-box vs homepage-box) are correct.

Let's do give it a week to see how it recovers though.

[msavin]

Sounds good.

On another note - small but perhaps big improvement: have you thought about blocking MDG packages from the trending section? they will always win because they are installed with every Meteor app.. I figure the community might gain more from seeing other packages up there

[msavin]

@abernix I believe we are back at it :( I've been monitoring a few packages for the past few weeks and it looks like the numbers are stale

[arggh]

It's a pity that Meteor devs are pretty much stuck with Atmosphere.

• I released a small Meteor package 11 days ago, but you still can't find it through search.
• In most sections, a visitor has to first browse through N pages of Meteor core packages before finding any relevant content.
• Finding packages for any specific use cases is really hard. For example, searching for "cloudfront" does not give you zodern:cdn as a result, even though I think it really should.
• Kinda same as above, but having keywords or categories would be awesome. Maybe there could be a keywords property in Package.describe section? Would be nice, if the community could add tags, keywords or categories too, so it wouldn't be solely up to the package maintainers.
• Being able to see and use a package's "pulse", when it has last seen any care & love, would be a great factor when deciding what to pick. I know I can see the last releases when I open a specific package's page, but would be awesome if I could filter out all package with no updates in the last 6 months, for example.
• A tag to filter out all packages which merely wrap some lib from npm would be greatly appreciated too.

So, this leads me into thinking there's four options:

1. Leave Atmosphere as it is, not really helpful, actually might be harmful to the community.
2. Given that Percolate Studios is now part of Meteor and a package repository for Meteor packages is a pretty crucial part of Meteor, please fix Atmosphere. Unlikely to happen, I guess?
3. Open source Atmosphere and let community improve it or build upon it.
4. Document and make available a proper API for accessing package data, so the community can build their own. (Maybe I'm just ignorant and there already exists a great resource for this?)

There's a lot of high quality packages hidden in the catacombs of Atmosphere.

[mitar]

API is documented and available. I have seen alternative implementations of package index page (I do not remember now which). I have made a library to use the API here.

So feel free to build a better package index page.

[arggh]

Thanks for the links @mitar !

[abernix]

@arggh Thanks for pointing this out. It seems that when our Elastic hosts changed five months ago (which was something I took notice of and made sure happened in order to make sure that Atmosphere kept working) I failed to update a worker which was responsible for writing updates. 😞

That said, while Meteor has respectable usage numbers, the fact that this is the first I've heard about this in many months shines a bit of light on Atmosphere usage. Even with Meteor packages having some notable benefits over npm packages, with the npm ecosystem as strong as it is, there's obviously much less incentive to be searching Atmosphere.

Let me know if your package isn't showing up in the index, now that it's updated.

[abernix]

I personally just use meteor search <pkg>.

[arggh]

Thanks @abernix , it's now showing up 👍 I'm glad something good came out of this whining!

However, unless I've missed some recent updates, I don't think Meteor packages are going anywhere? Npm sure is thriving, but Meteor's own packages pack more punch and there's value in keeping that ecosystem alive. At least I think so!

Searching with meteor search doesn't really help with the package discoverability issue, right? Searching for cloudfront still gives me only one result, omitting crucial and valid results like zodern:cdn (this scenario is just an example I came up with).

I guess what I'm trying to say (now that the indexing is working again) is: with some relatively minor improvements like keywords, better indication of packages' pulse/status and smarter listings (like exclude core packages from top-lists), Atmosphere could be _so much better_. Now it seems the only path forward is to build an Atmosphere-clone with these features built on top of Meteor's package system and hope it reaches a critical mass of usage to be actually useful.

[merlinpatt]

I think atmosphere (or whatever takes its place) also needs to be cleaned or separated / sorted into Meteor dependent vs npm replaceable. With the last few versions of Meteor, there has been a greater push to switch most packages to their npm counterparts. There are also tons of packages which are repeated as well (look up jquery-ui, i18n, and underscore for a few examples).