Rails: Strong parameters: allow hashes with unknown keys to be permitted

[spohlenz]

From what I can tell, strong parameters currently has no ability to permit a hash with unknown keys. Where this would be particularly useful is for the newly supported Hstore and JSON data types -- a good example of this use case can be found here: http://schneems.com/post/19298469372/you-got-nosql-in-my-postgres-using-hstore-in-rails

I would have expected that passing an empty hash as an permitted value would allow a hash to be passed through. e.g.

params = ActionController::Parameters.new(product: { name: 'Test', data: { weight: '12kg' } })
params.require(:product).permit(:name, data: {})

however this does not pass the data hash through (though it is not documented that it should work).

Assigning the data parameter separately is an option but it complicates my code unnecessarily -- I would prefer to be able to stick with mass-assignment for all attributes.

Happy to work on a patch for this if this proposal is reasonable. I've only just started looking into strong parameters though so there may be drawbacks I haven't considered.

[spohlenz]

Another possibility which I think I like even better would be to allow permit! to take an optional parameter, to allow this:

params.require(:product).permit(:name).permit!(:data)

[sideshowcoder]

I am really interested in a way to solve this as well, so +1. I think your 2. approach looks cleaner to me as well, passing {} to permit all sub keys looks weird to me.

[senny]

strong parameters is not designed to handle every possible situation. We wanted to keep the API on point to handle the most common situations. As assigning a hash with unknown keys more or less defeates the purpose of strong parameters (restricting allowed keys), It's not supported because it could make your code vulnerable.

As you mentioned already you can simply fall back on a normal assignment if you know that you want to permit unknown keys.

/cc @fxn @rafaelfranca

[fxn]

Confirm, the API is designed to whitelist every single key. If your use case does not fit well then you need to resort to Ruby. That flexibility is also by design, in the end it is just Ruby.

[spohlenz]

strong parameters is not designed to handle every possible situation. We wanted to keep the API on point to handle the most common situations.

Understand, but this doesn't exactly seem like an uncommon use case (particularly with the introduction of Hstore and JSON types), and with attr_accessible/attr_protected being removed from Rails core, options are severely reduced.

I would argue that this actually makes the API more consistent, since permit! would accept a list of keys, closely mirroring permit.

As assigning a hash with unknown keys more or less defeates the purpose of strong parameters (restricting allowed keys), It's not supported because it could make your code vulnerable.

Except you'd only be permitting an unknown hash on a single key within the params. It's certainly more secure than doing a permit! on the whole params hash (which is supported behaviour).

As you mentioned already you can simply fall back on a normal assignment if you know that you want to permit unknown keys.

My main problem with this is that I go from having to test one message (Product.create) to having to test three messages (Product.new, @product.data= and @product.save). The separate assignment for my hash also has to be duplicated across both my create and update actions.

Here's a concrete example of how my proposal would improve things. Before:

class ProductsController < ApplicationController
  def create
    @product = Product.new(product_params)
    @product.data = params[:product][:data]
    @product.save!
  end

  def update
    @product = Product.find(params[:id])
    @product.data = params[:product][:data]
    @product.update!(product_params)
  end

private
  def product_params
    params.require(:product).permit(:name)
  end
end

After:

class ProductsController < ApplicationController
  def create
    @product = Product.create!(product_params)
  end

  def update
    @product = Product.find(params[:id])
    @product.update!(product_params)
  end

private
  def product_params
    params.require(:product).permit(:name).permit!(:data)
  end
end

I do think this needs more discussion. It's all well and good to say just do it manually, but it feels wrong to have to work around a feature (and make my code more complicated), particularly when the alternatives have been removed.

[fxn]

To avoid duplication set the trusted parameter in the helper (untested):

def product_params
  params.require(:product).permit(:name).tap do |whitelisted|
    whitelisted[:data] = params[:product][:data]
  end
end

[spohlenz]

Thanks @fxn. That is a reasonable solution I had not thought of. :)

[fxn]

@spohlenz awesome :), this feedback was useful we are going to document a few edge cases like this or similar.

[hakanensari]

@senny It may be helpful to have the Strong Parameters README mirror the last paragraph on using with Hstore and JSON data types. Latter ends up being a more obvious reference point thanks to Google.

[senny]

@hakanensari I'm not sure if duplicating the guides in the README is a good thing. They will get out of sync quickly. I'd like to keep the additional examples in the guides because they are the reference for Rails.

Maybe we could link from the README to the relevant section in the guides? (http://guides.rubyonrails.org/action_controller_overview.html#more-examples)

@fxn what do you think?

[fxn]

@senny definitely linking.

[lanej]

The solution mentioned by @fxn only works if you have decided to log instead of raise when un-permitted parameters are present. There are plenty of situations (metadata, configuration, etc.) where whitelisting a sub-hash is completely valid.

I think that @spohlenz solution might be the most similar and I will have to implement something like it.

[fxn]

I like this approach https://github.com/rails/rails/pull/12609.

[lanej]

that's a sufficient workaround but hardly a good solution. what about params.require(:product).permit(:name, :data => Hash) and allowing Hash as a permitted scalar value IF it is explicitly defined.

[fxn]

Yes, I believe this use case deserves API.

Something like data: :permit! would be nice, I'd need to think about a way to express that that feels like a natural addition to the existing API.

[lanej]

fyi, the approach in #12609 is not feasible for hashes of a depth greater than 2.

[jhubert]

:+1: This has also come up with us as an issue for supporting HStore metadata values on our models.

Also, and perhaps I'm doing something wrong, but the solution from @fxn seems to actually set nil values on the params if they aren't already set:

params = ActionController::Parameters.new({ test: true })
params.permit(:test).tap do |whitelisted|
  whitelisted[:more] = params[:more]
end
# => {"test"=>true, "more"=>nil}

[lanej]

@jhubert that definitely makes sense. whitelisted is a hash and you are setting whitelisted[:more] to params[:more] which is nil.

[jhubert]

@lanej Thank. Yeah, that makes sense. It was unexpected behavior for me and my tests failed because the parameter was being set to nil instead of being unprovided. Figured I would mention it in case other people just copy / paste it expecting it to just permit the variable if it was provided and not change the params themselves.

[richardkmichael]

@jhubert FWIW, ActionController::Parameters#fetch() will raise ActionController::ParameterMissing, instead of simply returning nil.

[1] pry(main)> p = ActionController::Parameters.new({test: true})
=> {"test"=>true}
[2] pry(main)> p[:foo]
=> nil
[3] pry(main)> p.fetch :foo
ActionController::ParameterMissing: param is missing or the value is empty: foo # .... 
[4] pry(main)> p.fetch :foo, nil
=> nil

[nhattan]

thanks, this works for me:

def product_params
  params.require(:product).permit(:name).tap do |while_listed|
    while_listed[:data] = params[:product][:data]
  end
end

[chinshr]

@fxn agree, that's an acceptable solution. @nhattan, I ended up adding testing for nil value.

def product_params
  params.require(:product).permit(:name).tap do |whitelisted|
    whitelisted[:data] = params[:product][:data] if params[:product][:data]
  end
end

[NullVoxPopuli]

i think this would be a neat feature to add.
I have data that looks like this currently:
=> {"47"=>{"M"=>{"quantity"=>"2"}, "XL"=>{"quantity"=>"4"}, "XXXL"=>{"quantity"=>"1"}}}
where it maps the ID of an object to some properties for that particular object...

but maybe there is a better way to do that? idk. ('usually is)

[cohesivejones86]

My company doesn't use the key value of the nested we expect the nested hash to have and id property if its persisted for example
{ root_model: {
nested_model: {
GUID: { id: 1, value: 'something persisted' },
ANOTHER GUID: { value: 'a new record as it has no id'},
GUID: { id: 3, _destroy: true,
value: 'a record to delete but maybe we should
have another controller for this'
}
}
}
maybe we need to have wildcards ie %id% or %GUID% or just stop using nested attributes

[nhattan]

@chinshr You're right! thanks for pointing this case.

[joelpresence]

This is an issue for us as well - we have nested hashes where near the top we have unknown keys (e.g. product ids) but underneath those unknown keys we have hashes with known keys. Can we somehow incorporate this into the strong params API? As it stands it doesn't work well for serialized Hash with arbitrary keys at any point in the nesting ...

Thanks!

[chrisbloom7]

Adding this from http://stackoverflow.com/a/24752108/83743 since it's what finally worked for me:

If you use :raise instead of :log for config.action_controller.action_on_unpermitted_parameters in your environment then remember to remove properties from params before calling permit. Then the method will be

def product_params
  properties = params[:product].delete(:properties)
  params.require(:product).permit(:title, :description).tap do |whitelisted|
    whitelisted[:properties] = properties
  end
end

[gregblass]

Really stuck here with this issue. See my stackoverflow question here:

http://stackoverflow.com/questions/31945048/rails-4-strong-params-with-multiple-objects-and-integer-keys/31947212#31947212

In summary, I'm submitting 2-4 records at once.

This is what I've reproduced in the console:

params = ActionController::Parameters.new( { knockouts: {1 => {volume: 3.0, temperature: 2.0}, 2 => {volume: 4.1, temperature: 2.5}}} )

permitted = params.require( :knockouts ).permit( :id => [ :volume, :temperature ])
Unpermitted parameters: 1, 2
=> {} 

I just want to whitelist the first value to be any integer...the ID of the record coming through. Is that possible?

[ArunSakthivel]

I think, This will help,

params.require( :knockouts ).permit(params[:knockouts].keys.map {|c| {:"#{c}" => [:volume, :temperature]}})

[nhattan]

@ArunSakthivel I think the result he wants is:

=> {1 => {volume: 3.0, temperature: 2.0}, 2 => {volume: 4.1, temperature: 2.5}}

but your code still returns {}

[uberllama]

FYI, this is code I'm using in production to get around this issue.

# Strong params workaround to allow arbitrary json column field values on params
#
# @example
#   post_params
#     permit_json_params(params[:post], :custom_field_values) do
#       params.require(:post).permit(:title, :body)
#     end
#   end
#
# @param hash [Hash]   params hash
# @param key  [Symbol] key to allow arbitrary values for
#
def permit_json_params(hash, key)
  json_values = hash.delete(key)
  permitted_params = yield
  permitted_params[key] = json_values if json_values
  permitted_params
end

[ecuageo]

I know this won't work for everyone's use case. If you have some validation in your model through store_accessor you would have a list of keys to permit through stored_attributes.
klass = params.require(:type).constantize
params.permit(:type, content: klass.stored_attributes[:settings])

[PandaWhisperer]

None of the workarounds suggested in this thread seemed to work for me (Rails 4.2.4), so I devised the following workaround:

def product_params
  properties_keys = params[:product][:properties].keys
  params.require(:product).permit(:title, :description, properties: properties_keys)
end

Hope that helps someone.

[jhubert]

@PandaWhisperer :+1: Thanks

[lanej]

@PandaWhisperer @jhubert problem with that solution is that it requires you to enforce the presence of params[:product] AND params[:product][:properties] before you can then allow / disallow.

[PandaWhisperer]

@lanej you are correct. Might need to throw an if in there to safeguard against that.

[sashman]

@uberllama I have modified your answer a little and it worked nicely for any number of dynamic keys. Example params: post: { something: {...}, something_else: {...} }

  def post_params
    permit_key_params(params[:post]) do
      params.require(:post)
    end
  end

  def permit_key_params(hash)
    permitted_params = yield
    hash.keys.each do |key|
      values = hash.delete(key)
      permitted_params[key] = values if values
    end
    permitted_params
  end

[iagopiimenta]

This monkey patch can help: https://gist.github.com/iagopiimenta/43a66712c021a9d3f540#file-monkey_patch_rails_parameters-rb-L21

It can be used like this:

params = ActionController::Parameters.new(
  form: {
    scalar: 'foobar',
    many: [
      {
        field1: 'foo'
      },
      {
        fiedl2: 'bar'
      }
    ],
    single: {
      filed3: 'baz'
    }
  }
)

params.require(:form).permit(:scalar, many: Hash, single: Hash)

[aliibrahim]

I modified @PandaWhisperer solution to safeguard against non-existing key:

def product_params
  properties_keys = params[:product].try(:fetch, :properties, {}).keys
  params.require(:product).permit(:title, :description, properties: properties_keys)
end

[koenpunt]

For nested params I now use the following:

def item_params
  params.require(:item).permit(values: permit_recursive_params(params[:item][:values]))
end

def permit_recursive_params(params)
  params.map do |key, value|
    if value.is_a?(Array)
      { key => [ permit_recursive_params(value.first) ] }
    elsif value.is_a?(Hash) || value.is_a?(ActionController::Parameters)
      { key => permit_recursive_params(value) }
    else
      key
    end
  end
end

Edit:
_Updated version can be found in this gist_

[MSCAU]

@koenpunt - very nice, thanks. I need something like that as I am trying to store a tree (of variable depth) in a JSONB column. However, your permit_recursive_params output breaks for me where the value is an array of hashes.

This tree:

budget: {
    tree: {
        name: "Australia",
        value: 39904,
        id: 6,
        description: "",
        type: {
            name: "Other",
            icon: "<i class="fa fa-pagelines"></i>"
        },
        children: [
            {
            name: "Boxes",
            value: 451609,
            id: 7,
            description: "Used to store things.",
            type: {
                name: "Storage",
                icon: "<i class="fa fa-archive"></i>"
            },
            children: [
etc...

becomes this in PostgreSQL 9.4:

{"id": "6", "name": "Australia", "type": {"icon": "<i class=\"fa fa-pagelines\"></i>", "name": "Other"}, "value": "39904", "children": {"0": {}, "1": {}}, "description": ""}

FWIW, here's the strong parameter definition I am using:

params.require(:budget).permit({tree: permit_recursive_params(params[:budget][:tree])})

Can you advise what do do to get those children to render correctly?

[koenpunt]

With the data you provide, I believe I get correct results: https://gist.github.com/koenpunt/ac279e05cfeb0954ca763344fc0240b4

[bdmac]

@koenpunt I'm hitting an error as well with permit_recursive_params calling map on a String object eventually. Here is a small example:

params = ActionController::Parameters.new(
  budget: {
    tree: {
      name: "Australia",
      value: 39904,
      id: 6,
      description: "",
      type: {
        name: "Other",
        icon: "<i class=\"fa fa-pagelines\"></i>"
      },
      children: [
        { required: ["phone"] },
        { required: ["email"] }
      ]
    }
  }
)

Running this through permit_recursive_params gives NoMethodError: undefined method 'map' for "phone":String

[bdmac]

A working version looks like this:

def permit_recursive_params(params)
  (params.try(:to_unsafe_h) || params).map do |key, value|
    if value.is_a?(Array)
      value = value.first
      if value.is_a?(Array) || value.is_a?(Hash)
        { key => [ permit_recursive_params(value) ] }
      else
        { key => [] }
      end
    elsif value.is_a?(Hash) || value.is_a?(ActionController::Parameters)
      { key => permit_recursive_params(value) }
    else
      key
    end
  end
end

[koenpunt]

@bdmac good catch, I've updated my gist to check if the first value responds to map, and if so, pass it along.

[koenpunt]

And now that to_unsafe_h is used, we no longer have to check for is_a?(ActionController::Parameters)

[MSCAU]

Thanks, @koenpunt. From the server logs, it looks as though the JSON I submit to the server (ie. as I originally presented it):

children: [{},{}, etc... ]

is getting converted by Rails to

"children": { "0": {}, "1": {}, etc... }

I just need to figure out why...

[MSCAU]

Hmmm, seems the issue is in the JS before the AJAX call, sorry. jQuery needs contentType to be set to "application/json". See http://stackoverflow.com/questions/6410810/rails-not-decoding-json-from-jquery-correctly-array-becoming-a-hash-with-intege

[Nowaker]

Hey @fxn and @senny, it's the year of 2016 now. Rails got background jobs and even websockets, things that were originally neglected/ignored. Since 2013, the use of free-form JSON and Hstore values has rapidly grown too, and maybe it's high time to not ignore them either. The values submitted to them are free-form by design, therefore not subject to any whitelisting/filtration. But they don't live alone. They live inside models whose fields _are_ whitelisted and filtered against mass assignment.

Consider Stripe gateway. They allow users to store free-form metadata on customer object, and that's just a hash. How to achieve that in Rails if the JSON request looks like this: POST /customers, {description: ..., source: ..., metadata: {... free form data here...}}. Answer: fight the framework - and many people in this thread did it their own way. Currently, if there's any JSON field without a predefined "schema" inside the params, Strong Parameters has to be worked around for each field. In 2016, letting a non-checked JSON in is a common situation.

There's already @sdepold's pull request to allow this here: https://github.com/rails/strong_parameters/pull/231. I hope you can reevaluate this feature and give it a green light. If there's any issue/feedback about that PR, I can take it over from the original submitter and finish it up if he's unreachable. Please let me know.

[fxn]

@Nowaker agree, we've been conservative, but time says this use-case deserves a dedicated API.

The :* proposal seems good to me. The patch would need some work, but the basic idea of a wildcard symbol to say "accept whatever here" might work.

The patch is no big deal, I might write one based on that PR (and give credit of course).

[fxn]

Implemented here.

[stiig]

it would be nice if it also allow to set structure incoming hash, like a

params.permit(preferences: [{:scheme, font: {:name, :size}}])

for next cases in params:

params = ActionController::Parameters.new(
    username: "fxn",
    preferences: [
        {
            scheme: "Marazul",
            font: {
                name: "Source Code Pro",
                size: 12
            }
        },
        {
            scheme: "new scheme",
            font:
                {
                    name: "Another Font",
                    size: 14
                }
        }
    ])

[temirov]

I must be missing something, but Rails 5.0.1 still gives me grief:

irb(main):011:0> ActionController::Parameters.new(
irb(main):012:1*   items: [
irb(main):013:2*     { data: { a: 1}},
irb(main):014:2*     { data: { b: 2}}
irb(main):015:2>   ]
irb(main):016:1> ).permit(items: [ data: {}]).to_h
=> {"items"=>[{"data"=>{}}, {"data"=>{}}]}

[fxn]

Patch versions don't have new features, this is going to come with 5.1.

[ghost]

I modified @PandaWhisperer solution to safeguard against non-existing key:

def product_params
properties_keys = params[:product].try(:fetch, :properties, {}).keys
params.require(:product).permit(:title, :description, properties: properties_keys)
end

@aliibrahim thanks, that worked for me!

[stevebissett]

I can't see a solution to this that has been merged.
Is anyone able to point me to one (@fxn)?

I have the following input hash (obfuscated sample):

      {
        "name" => "Test",
        "size" => 150,
        "some_array_of_objects" =>
          [
            {"some_key" => {"name" => "test_2", "valid" => "true"},
             "another_key" =>
               {
                 "very_deep_hash" => {"abc" => "def"},
                 "i_dont_care_about_this_key" => 5
               }
             },
            {"some_key" => {"name" => "test_2", "valid" => "true"},
             "another_key" =>
               {
                 "very_deep_hash" => {"abc" => "def"},
                 "i_dont_care_about_this_key" => 5
               }
             }
          ]
      }

(Edited)

I want to whitelist all the keys in some_array_of_objects.
Even if I do know all the keys down to the lowest depth, I can't seem to get it working with arrays.

Can someone point me in the right direction?

[Nowaker]

@stevebissett The right direction would be for some_array_of_objects to be a hash. See https://github.com/rails/rails/commit/e86524c0c5a26ceec92895c830d1355ae47a7034 - it allows arbitrary hashes, not arbitrary values. @fxn suggested :* but it didn't end up being implemented.

[stevebissett]

@Nowaker I can't turn that into a hash, because I need multiple of them, (edited slightly).

[Nowaker]

"some_array_of_objects" => {elements: [...]}

[olmesm]

Rails 5 workaround -

def product_params
      load_params = params.require(:product).permit(:name)
      load_params[:json_data] = params[:product][:json_data]
      load_params.permit!
end

I realize this doesn't whitelist the data within the json_data param, but it excludes same-level unpermitted params.

Oh and hi @stevebissett :+1:

[Deepak275]

@spohlenz i have faced similer problem and after lot of googling and going through various stackoverflow Q/A was still not able to figure out good solution. So, i tried myself some hack.
So, here what i did
params = ActionController::Parameters.new(driver: { driver_data: { name: "zyx", address: "adfasf" } })
driver_data_keys = params[:driver][:name].keys
params.require(:driver).permit(driver_data: [driver_data_keys])

Advantages

• Do for rails 4.
• I know it is a hack, but still it is looking good than i have found different solution.
• it is the best soluion for whiltlisting any or any no of key withuto getting worried about handling it.
  I hope it will help. Happy coding.

[skozz]

The @olmesm solution works for Rails 4.2.7 too 👍

[batmanbury]

This thread started quite a while ago, but it's still active, so here's something I just discovered. I was able to permit a jsonb attribute to my User model with the syntax normally used with accepts_nested_attributes_for:

def user_params
  params.require(:user).permit(:email, :password, data: [:one, :two, :three])
end

Where data is your json attribute, and the array of properties are what you wish to permit. I'm not sure if this is functioning as intended by Rails, or if it just works through some fluke. It may not work with every use case, or with deeply nested attributes, but I'm curious to see how it works for others here. I'm using Rails 5.1.1.

[cianmcelhinney]

Running on Rails 4.2.6 this works for me:

def user_params
  params.require(:person).permit(:name, :description, :age, custom_attributes: [:key, :value])
end

This will allow the following params to be permitted:

{
  "person": {
    "name": "John",
    "description": "has custom_attributes",
    "age": 42,
    "custom_attributes": [
      {
        "key": "the key",
        "value": "the value"
      }
    ]
  }
}

This can be seen in the source here:
https://github.com/rails/rails/blob/bdc73a438a97f2e0aceeb745f4a95f95514c4aa6/actionpack/lib/action_controller/metal/strong_parameters.rb#L522

If you don't know what might be in custom_attributes you could do

def user_params
  params.require(:person).slice(:name, :description, :age, :custom_attributes) 
end

[lanej]

In the majority of these comments, the keys are known. This thread's subject was for unknown keys.

A few of the better workarounds are:

• explicit permit! of a nested hash : https://github.com/rails/rails/issues/9454#issuecomment-280853714
• Adding a permit_recursive_params function : https://github.com/rails/rails/issues/9454#issuecomment-199357625

Fix is coming in 5.1.2 https://github.com/rails/rails/commit/e86524c0c5a26ceec92895c830d1355ae47a7034

[anklos]

@olmesm

def product_params
      load_params = params.require(:product).permit(:name)
      load_params[:json_data] = params[:product][:json_data] if params[:product][:json_data]
      load_params.permit!
end

need to add a if condition otherwise it add a nil value for json_data, which causes issue if you call update on an object with the param

but good workaround, thank you!

[benbonnet]

@anklos your example work, saves the data correctly (thx alot)

but

It still complains about the concerned json column being unpermitted. Is that normal ?

[rmcsharry]

@deesx I would like to know the answer to that also. 50+ rep bonus for the answer

[xiaopow]

@olmesm , @anklos ,
This solution

def product_params
      load_params = params.require(:product).permit(:name)
      load_params[:json_data] = params[:product][:json_data] if params[:product][:json_data]
      load_params.permit!
end

works for most of my data, except when my json_data looks like this

{
  key: [
    {
      key2: [],
      key3: {},
    },
    {
      key2: [],
      key3: {},
    },
  ],
  key4: "123",
}

I get the following error. I am using Mongoid by the way.

Unpermitted parameter: :json_data
Completed 500 Internal Server Error in 55ms (ActiveRecord: 0.0ms)
ActionController::UnfilteredParameters - unable to convert unpermitted parameters to hash:

[michaelbridge]

Perhaps this is overkill, but one simple workaround that I didn't see here is as follows:

params.permit( ... ).merge(params.to_unsafe_hash.slice(:json_data))

The other workarounds don't appear to work for arbitrarily nested hashes and arrays.