Activeadmin: Mongoid support

[seanlinsley]

Inspired from a conversation in IRC with @machonemedia, I actually read the source code of @elia's activeadmin-mongoid gem and was surprised by how little code was actually there. I've never used Mongoid, but it's about time that Active Admin supported a different ORM.

Now in this ticket I'm not looking for +1s; #26 has already made the desire for this clear. Instead I want to identify the hurdles in the way for Active Admin to natively support Mongoid, and what we can do to get from here to there.

There are obviously changes (like #1713) to be made to Active Admin, but I'm more concerned about our dependencies:

Kaminari

Do things like #2638 apply to huge MongoDB data sets?

Ransack

Ransack doesn't currently support Mongoid, and it may never happen. There's an in-progress attempt at adding that functionality, but I don't know how far that's going to go.

I've set the 1.0.0 milestone to February, so the question is can we get a compatibility layer for Ransack 90% done in that time? If that's a dead end, what other options do we have?

CC @johnnyshields, @kristianmandrup

[johnnyshields]

Firstly, let's distinguish between A) _"Gracefully-degraded support"_ (most features work, those that don't are omitted, AR is not forced to load) and B) _"Full support"_ (all features work identically on Mongoid and AR).

I imagine most in Mongo community including are quite happy with having only (A), and if anyone needs a given feature they can submit a PR.

Kaminari - Kaminari gem works as-is for Mongoid. Have used it in production w/ Mongoid for 6 months. Pagination works equally well in both Mongo and SQL.

Count Queries - No issue with using count on mongo. (Full disclosure: there was an issue on mongoDB 2.3 where count was slow, it has now been patched).

Ransack - I propose to omit Ransack search for mongo (just show a placeholder in the UI "Search not yet supported on Mongo"). I myself don't need search in ActiveAdmin, so I won't be working on this.

As I have said (many times before :smile:) my only hard requirement here is that AA and it's dependencies should _not_ force AR to load. Currently this PR is still outstanding: https://github.com/activerecord-hackery/polyamorous/pull/3

[seanlinsley]

Firstly, let's distinguish between A) "Gracefully-degraded support" (most features work, those that don't are omitted, AR is not forced to load) and B) "Full support" (all features work identically on Mongoid and AR).

Yeah I imagine that all features centered around associations wouldn't have a nice analog in the Mongoid world. Is there anything you can think of that would need to be disabled? Related: how would we determine the model's attributes for the default index, show, and edit views?

I propose to omit Ransack search for mongo (just show a placeholder in the UI "Search not yet supported on Mongo"). I myself don't need search in ActiveAdmin, so I won't be working on this.

If that's the course we take, it'd certainly be good to show something like that in development, but in production the filter sidebar should probably be hidden so as not to confuse users.

As I have said (many times before :smile:) my only hard requirement here is that AA and it's dependencies should _not_ force AR to load. Currently this PR is still outstanding: https://github.com/activerecord-hackery/polyamorous/pull/3

Why don't we make Ransack a conditional dependency? #2513 would be useful here.

While I have you here, is there a Mongoid equivalent to this code?

User.reorder('email asc').uniq.pluck 'email'

[johnnyshields]

I imagine that all features centered around associations wouldn't have a nice analog in the Mongoid world.

Actually -- Mongoid supports the same _associations_ which AR does, although mongoDB itself doesn't support _table joins_. Suppose Customer has_and_belongs_to_many Products. If I do Customer.all.first.products, AR I believe achieves this in one query using joins. Mongoid on the other hand supports the _exact same syntax_, but behind the scenes it executes two queries--first the customer which contains an array of product_ids, then the products.

In production the filter sidebar should probably be hidden so as not to confuse users.

Agreed.

Why don't we make Ransack a conditional dependency

Ransack is not the problem--we've fixed it already here: https://github.com/activerecord-hackery/ransack/pull/296. The problem is Polyamorous, namely whenever it appears in your Gemfile, its gem initializer routine calls require 'activerecord', which it should not do since no other gem in my 200-gem long Gemfile does. It's a trivial change, please get @ernie to accept my PR: activerecord-hackery/polyamorous#3

is there a Mongoid equivalent

User.order(email: :asc).uniq.pluck 'email'

[seanlinsley]

Cool, I'll have to look more into using associations with Mongoid.

Ransack is not the problem--we've fixed it already (...) It's a trivial change, please get @ernie to accept my PR

I guess that's been the Rails community's standard approach to work around the fact that Bundler doesn't support conditional dependencies. I would love to see that type of feature in either Bundler or Active Admin, but the community already has a standard and it seems that polyamorous should follow it.

I have no special power over @ernie, so all we can do is pester him :cat: (note that he can't dedicate much time anymore)

[ernie]

@seanlinsley: activerecord-hackery/polyamorous@e80fb45b

[seanlinsley]

Merged because @seanlinsley has special power over me.

Haha, thanks @ernie :-)

[seanlinsley]

I've been working on using Ransack for advanced sorting: #2608

What will we do for Mongoid?

[johnnyshields]

Thanks @ernie!

[machonemedia]

Glad I could inspire something :)

One thing to note is mongoid has two different types of relations:

Embedded, which embeds the child model/document into the parent document, so when called it only needs to do one query and all children are available.

Referenced, which is similar to AR joins, except does two queries like @johnnyshields mentioned.

http://mongoid.org/en/mongoid/docs/relations.html

[johnnyshields]

Ransack for advanced sorting - What will we do for Mongoid?

I do not know the particulars of Ransack, however, one way to do it would be:

• an abstract Ransack::SortOrder class that has attrs like :field, :direction (asc/desc) etc (maybe you have a list of SortOrders to do multi-column sort)
• a Ransack::Sorter whose job is to convert SortOrders to query objects. Since the desired query object can be either an ActiveRecord::Relation or a Mongoid::Criteria, our Sorter class would merely check type of the model and then hand it off to either:
  
  
  
  • a Ransack::ActiveRecord::Sorter
  
  
  • a Ransack::Mongoid::Sorter
  
  

each of which would transform the SortOrders into the respective query object of the ORM.

In general you should assume that Mongoid behaves _like_ ActiveRecord. There will be small differences in syntax, however, for every AR method there is a 1:1 functional equivalent in Mongoid, and like AR the query methods of Mongoid are chainable.

[johnnyshields]

One nice thing about this approach that, since we are dealing purely in chainable query objects, you can partially omit functionality for Mongoid but still forward the query object downstream.

class Ransack::ActiveRecord::Sorter
  # returns the ordered relation
  def apply_sort(scope, order)
    scope.reorder("#{order.field} #{order.direction}")
  end
end

class Ransack::Mongoid::Sorter
  # can't be bothered to implement this function on Mongoid...
  def apply_sort(scope, order)
    scope
  end
end

[seanlinsley]

Here's a spec that I assume Greg Bell wrote up a long time ago, that was in the wiki. I'm going to remove it from the wiki since it's not helping anyone over there.

Resource Registration

When a resource is registered, it should instantiate the correct adapter for the given backend. So if an ActiveRecord subclass is registered, it should instantiate an ActiveAdmin::ResourceAdapter::ActiveRecord. (Note class naming is up for discussion still)

The type could also be explicitly passed in.

For example:

ActiveAdmin.register User, :type => :mongoid

Abstract Adapter

Adapters will have to support the following:

• Retrieving a collection for the index page. Most of the features required for this are in ActiveAdmin::ResourceController::Collection
  
  
  
  • Sorting
  
  
  • Scoping (support for the scope AA dsl method)
  
  
  • Scope to (support for the scope_to AA DSL method)
  
  
  • Pagination
  
  
  • Search / Filtering
  
  
• Instantiate and return a new resource
• Create a new resource from params
• Find a specific resource
  
  
  
  • This could include a scope_to or belongs_to AA relationship
  
  
• Edit a resource

Libraries can register a new AbstractAdapter subclass and should also be able to specify some logic for when it get's used.

For example:

class MyCustomAdapter < ActiveAdmin::ResourceAdpater::Abstract; end

ActiveAdmin::ResourceAdpater.register :my_custom, MyCustomAdapater do |resource_class|
  # This block get's run if the user does not explicitly pass
  # in the :type option. Return true from it if your adapter should be used
  resource_class.ancestors.include?(ActiveRecord::Base)
end

Abstract Adapter Feature Support

Active Admin supports many features which may not be supported by all ORMs. Initially, the sidebar filtering comes to mind. ORMs should have a way of letting the system know if certain features are enabled on them.

For example:

class SimpleAdapter < ActiveAdmin::ResourceAdapater::Abstract
  def supports_filters?
    false
  end
end

ActiveAdmin.register SomeObj, :type => :simple_adapter do
  filter :name #=> raises an exception since this feature is not support
end

[fred]

This is great news to have official mongoid support in AA
I've been using for many months this branch with filter support working. https://github.com/monfresh/activeadmin-mongoid/tree/filter-fix

@johnnyshields, why not this instead?

class Ransack::ActiveRecord::Sorter
  def apply_sort(scope, order)
    scope.reorder("#{order.field} #{order.direction}")
  end
end

class Ransack::Mongoid::Sorter
  def apply_sort(scope, order)
    scope.order("#{order.field} #{order.direction}")
    # or:
    # scope.order(order.field.to_sym => order.direction.to_sym)
  end
end

http://mongoid.org/en/origin/docs/options.html#sorting

[johnnyshields]

@fred agreed, that would be fine. I was just giving an example of the "lazy approach", i.e. implement the logic in AR but have a dummy implementation in Mongoid that would not cause an error.

[timoschilling]

@seanlinsley

class SimpleAdapter < ActiveAdmin::ResourceAdapater::Abstract
  def supports_filters?
    false
  end
end

That should be more granular, not every Adapter support all filters but the most support the basics!
A better approach could be:

class SimpleAdapter < ActiveAdmin::ResourceAdapater::Abstract
  def supported_filters
    {
      relational: [],
      simple: [:contains, :lteq, :gteq, :equal]
    }
  end
end

Or the Adapter has a filter_supported? method:

class SimpleAdapter < ActiveAdmin::ResourceAdapater::Abstract
  def filter_supported? type
    ....
  end
end

[jordansexton]

@johnnyshields, a minor note:

If I do Customer.all.first.products, AR I believe achieves this in one query using joins. Mongoid [...] executes two queries--first the customer which contains an array of product_ids, then the products.

AR actually has the same behavior as described for Mongoid. In AR 3.2, Customer.all immediately executes returning all results as an array, so #first is just Array#first. In AR 4, Customer.all.first executes the query with LIMIT 1.

Either way, #products is always a second query. AR doesn't try to be very clever with joins, you generally have to explicitly declare them.

[johnnyshields]

@jordansexton thanks for correcting me, it's been ages since I've used AR. Switched to Mongoid and never looked back!

[pencilcheck]

It's great to have official support. Where can I follow on this progress? Is there a branch where I can test it from?

[janckerchen]

how about this feature now? I can't find any doc for this.

[chenkirk]

I am still using elia/activeadmin-mongoid with a few patches.

[timoschilling]

ransack has now a open (working) PR for mongoid support activerecord-hackery/ransack#407

[sbounmy]

Ransack PR for mongoid support has been merged, please give a shot everyone :+1:

[pencilcheck]

So how do I try it?

[Zhomart]

@pencilcheck You can try this gemspec:

gem 'mongoid', '~> 4.0.0'

gem 'ransack', github: 'activerecord-hackery/ransack'
gem 'activeadmin', github: 'activeadmin'
gem 'activeadmin-mongoid', github: 'Zhomart/activeadmin-mongoid', branch: 'ransack'

[timoschilling]

@Zhomart can you help us to improve the mongoid support in activeadmin himself?

[Zhomart]

@timoschilling Sure. Is there any active implementation of mongoid support in activeadmin going on?

[timoschilling]

@Zhomart we have no active work on this. We have lib/active_admin/orm and some if defined?(Mongoid). I would be great to get full support for mongoid

[pencilcheck]

@Zhomart doesn't work :(

[Zhomart]

Here is the example https://github.com/Zhomart/ActiveAdminMongoidExample

[pencilcheck]

@Zhomart doesn't work, 401 unauthorized following the steps on the repo (btw, you have a typo, it should be rails s)

[timoschilling]

@Zhomart is that a plain vanilla rails app? If not can you write a How To for the readme? Should we move the BSON hack into ActiveAdmin? Is that hack just to convert the ID into a string?

[Zhomart]

@timoschilling It is just plain rails 4 app, I've updated readme. About the BSON hack, it is actually devise's issue. I've found there more correct solution and implemented/described in the example.

@pencilcheck Try to re-pull and clean your cookies.

[pencilcheck]

@Zhomart try my fork, added a has_many association filter example, there seems to be a ransack bug so you can't filter based on association. (try go to user table and filter on phone)

https://github.com/pencilcheck/ActiveAdminMongoidExample

[Zhomart]

@pencilcheck yup, ransack currently doesn't support mongoid's associations.

[timoschilling]

@Zhomart can we build the filters so that they only provide which is supported?

[Zhomart]

@timoschilling we may add condition here ActiveAdmin::Filters::ResourceExtension#add_filter.

I've impemented basic features and adapted some specs. Here what I've done:

• Mongoid::Document minimally monkeypatched in orm/mongoid/document.rb
• added many defined?(ActiveRecord) and defined?(Mongoid) to the source code, features and specs
• didn't enable comments for mongoid
• skipped associations, but belongs_to works as <select>
• to run mongoid specs RAILS=4.1.7 MONGOID=1 rake

It is a draft version, there are a lot of work to do. To fully support associations, ransack needs to be imroved. May be we should get rid of defined? and put codes into orm/*.

Here is the example of rails app using active_admin/mongoid.

[timoschilling]

Some thing that should be changed:

• We should not use defined?, we should use a ORM adapter. In the best case the adapter could work with ActiveRecord and Mongoid at the same time / in one app.
• comments should work with a Mongoid comments model too
• the test setup script should be a ActiveRecord and Mongoid rails_template.rb should be unified, maybe with sub templates
• the test apps should be in the same folder as rails_<rails_version>_<orm_name>_<orm_version>

[timoschilling]

Maybe we should move this over to the 2.0.0 milestone

[maran]

This feature is very important for me, I made it work but proper support is worth some money to me. I will pledge 1BTC going towards development of this pull request, to be divided among the contributors. So far I think 100% of it is going to @Zhomart. I will pay out once @timoschilling is satisfied with the state of this PR.

[elia]

I think deferring mongoid support after release is a great idea. IMO activeadmin should consolidate the current status and ship. It totally deserves the 1.0 badge given how much it's used in production (even sourced from github).

1.1 could bring experimental support for mongoid and maybe after that it will be able to work with simple ActiveModel objects (which I'm most interested in).

Just my personal view of course, but still curious if it's shared by @seanlinsley and @timoschilling

[timoschilling]

Ok, we will add a basic (beta) support on 1.0

[maran]

It's been a while. Anybody did any work on this lately?

[pencilcheck]

It's sad but @Zhomart version still has a lot of holes with the current master of activeadmin, it just doesn't work out of box, and in the errors are methods that should've been overwritten a long time ago.

[Zhomart]

activeadmin is improved a lot in the last month. Now I got latest master changes and started adding mongoid as @timoschilling suggested earlier. It doesn't work yet, after finishing basic mongoid support and comments I'll make pull request. I'm working on https://github.com/Zhomart/active_admin/tree/mongoid.

[timoschilling]

@Zhomart what do you think about activerecord-hackery/ransack#469, how should we deal with it?

[Zhomart]

@timoschilling we need to improve it too.

[timoschilling]

@Zhomart If you work on the adapter interface, please create a issue for the discussion about the API.

[Zhomart]

I need a little help to support multiple ORMs simultaneously. We have method ActiveAdmin::Filters::FormBuilder#filter that can only work for one ORM. To support other ORMs it should find ORM using property klass. Example:

def filter(method, options={})
  if defined?(ActiveRecord) && klass.ancestors.include?(ActiveRecord::Base)
    # then do active record related code
  elsif # something something
  end
end

Also, implementations of this methods for different ORMs should be placed in active_admin/orm/*. I tried multiple solutions, all of them failed.
There are more than 15 methods in different classes that need to be written specifically for one ORM.

[timoschilling]

NOTE: all this thinks are not testet, they are just ideas

How should we detected wich adapter we need to use for wich model?
I see 2 ways:

1. With a config.default_orm_adapter = "ActiveRecord" and param for register:
   ActiveAdmin.register Foo, adapter: "Mongoid"
2. With a detection Based on the model. I have build something similar for a configurable pretty_format: https://github.com/activeadmin/activeadmin/compare/master...idea/pretty_format_api

The usage of the Adapter could look like:

class ActiveAdmin::Filters::FormBuilder
  def filter(method, options={})
    ActiveAdmin.adapter_for(model).filter(method, options = {})
  end

  # or

  delegate :filter, to: :model_adapter

  def model_adapter
    @model_adapter ||= ActiveAdmin.adapter_for(model)
  end
end

class MongoidAdapter < AbstractAdapter
  detection do |object|
    object.is_a?(::Mongoid::Document)
  end

  def filter? # true only if Ransack supports the orm
    true
  end

  def filter(method, options={})
    # ...
  end
end

I hope that will help you. Otherwise provide more informations.

[timoschilling]

@Zhomart you should include #3551 into your work

[Zhomart]

@timoschilling I like your ideas. I'll try to implement them.

[pencilcheck]

https://github.com/elia/activeadmin-mongoid/issues/97

I believe the setup works for me in Rails 4.1.1, Mongoid 4.

If you are encountering this bug https://github.com/elia/activeadmin-mongoid/issues/95, just use my branch.

gem 'activeadmin-mongoid', github: 'pencilcheck/activeadmin-mongoid', branch: 'patch-1'

[juni0r]

@Zhomart Is there a setup that is currently working?

I used to have

gem 'activeadmin', github: 'Zhomart/active_admin', branch: 'mongoid'
gem 'ransack', github: 'activerecord-hackery/ransack', branch: 'master'

which was working fine. Now I get a whole bunch of errors e.g.

undefined method `except' for #<Mongoid::Criteria:0x007fbfabd957c0>

[Zhomart]

@juni0r
try

gem 'ransack', github: 'activerecord-hackery/ransack'
gem 'activeadmin',         git: '[email protected]:Zhomart/active_admin.git', branch: 'mongoid-old'

Example is here https://github.com/Zhomart/ActiveAdminMongoidExample.
Current Zhomart/active_admin is under development. I'm experimenting different implementations.

Update: bitbucket repo renamed to active_admin

[juni0r]

@Zhomart Thanks for the quick reply!

I'm getting

Fetching [email protected]:Zhomart/mongoid.git
Permission denied (publickey).
fatal: Could not read from remote repository.

Please make sure you have the correct access rights and the repository exists.

[timoschilling]

Maybe @Zhomart can push that branch to github too

[Zhomart]

The branch mongoid-old is already at github https://github.com/Zhomart/active_admin/tree/mongoid-old.

Try

gem 'activeadmin',         github: 'Zhomart/active_admin', branch: 'mongoid-old'

[juni0r]

@Zhomart This works. Thank you so much for your help and your work on Mongoid support! Thanks also to @timoschilling for incorporating this into Activeadmin. I'm very glad I can finally use it with Mongoid!

[Ames]

Thank you all for the awesome work on this!
Apologies if this is not the appropriate place to post this (maybe it should be a ransack ticket?).

I just updated to:

gem 'rails', '~> 4.2'
gem 'mongoid', '~> 4.0.1'
gem 'ransack', github: 'activerecord-hackery/ransack'
gem 'activeadmin', github: 'Zhomart/active_admin', branch: 'mongoid-old'

and everything seems to work except for sorting. I'm noticing something funny about the query:

Started GET "/users?order=first_name_desc" for 127.0.0.1 at 2015-02-13 17:09:23 -0500
...
  MOPED: 127.0.0.1:27017 QUERY        database=mydb collection=users selector={"$query"=>{}, "$orderby"=>{"\"users\".\"first_name\""=>-1}} flags=[] limit=30 skip=0 batch_size=nil fields=nil runtime: 1.7580ms
...

It looks like it's including the model name and some extra quotes. I believe it should just be {"first_name"=>-1}. Perhaps the cause of the sorting issue lies elsewhere and this funny-looking query is incidental. Curious if anyone else has seen this issue, or has any tips on how to debug this. Thanks!

_Edit:_
I found the code that creates the order clause and "fixed" it. The fix probably breaks Active Record support. https://github.com/Ames/active_admin/commit/f0ce6ac8edc90f3063828ca46731ee7fe0141a86

[awsmsrc]

@zhomart we're using your mongoid-old branch and sorting doesnt appear to be working at all?

Is this a known issue or should I spend some time investigating?

[Zhomart]

@awsmsrc thanks for the report. I've temp. fixed it on the mongoid-old branch.

[awsmsrc]

Thanks @Zhomart what is the difference between mongoid and mongoid-old? which should i be using?

[Zhomart]

@awsmsrc mongoid-old is the my first attempt to integrate mongoid to activeadmin using "defined?" everywhere; and comments doesn't work.
I currently use mongoid to experiment more correct ways intagrate as we discussed in this thread.

[pencilcheck]

Should I use Zhomart mongoid fork or is activeadmin supporting mongoid now?

[Zhomart]

I'm currently using this temprorary solution:

# Try the one after EDIT:. This one is too old.
gem 'activeadmin',         github: 'Zhomart/active_admin', branch: 'formtastic2'
gem 'activeadmin-mongoid', github: 'Zhomart/activeadmin-mongoid', branch: 'ransack-mongoid'

I've stuck on design problem while adding mongoid to the activeadmin, as it was initially designed to use only one ORM.

EDIT:

Try this one:

gem 'ransack',             github: 'Zhomart/ransack', branch: 'mongoid'
gem 'activeadmin',         git: '[email protected]:Zhomart/active_admin.git', branch: 'mongoid-old'

gem 'formtastic'
gem 'formtastic-bootstrap', '~> 3.0.0'

[pencilcheck]

Cool, thanks. It's a shame that's as far as we can do for now. :\

EDIT

It has some issues when I setup with a brand new rails project. I will try a different repo then. Thanks.

[ptrikutam]

@pencilcheck did you ever get it working with your Rails project? What version of Mongoid were you using?

[leandromoreira]

Nice!
Are there any updates about it? what is the best work around for using active admin with mongoid and rails 4 today?

[gonzaM]

Any update on this? Is ActiveAdmin supporting mongoid with rails 4?

[IvRRimum]

How about rails 5?

[varyonic]

FYI ActiveAdmin with Rails 5 and Mongoid 6

[IvRRimum]

@varyonic Neat!

[gerryhd]

Anyone know of a workaround for Rails 4 + Mongoid 5? I'm getting a "undefined method 'connection'" error which I assume is because of ActiveAdmin current incompatibility with mongo. The solutions provided here (specifically for Rails 5 and Mongoid 6) gives me gem dependencies problems.

[varyonic]

Try this branch and open any issue with Elia and Nic here

[varyonic]

Any further issues or suggestions should be directed to activeadmin-mongoid. We may reopen this or other issues related to better activeadmin-mongoid integration when ready.