Rails: Nested params are parsed incorrectly (unexpectedly) in controllers

[simsalabim]

Hi all, I ran into some inconsistency with controller parsing nested params.

Steps to reproduce

post '/', params: {
  "items" => [
    { "brand" => { "name" => "Apple" }, "id" =>1, "color" => "pink" },
    { "brand" => { "name" => "Samsung" }, "id" => 2, "color" => "gold" },
  ]
}

# what appears in controller
{"items"=>[{"brand"=>{"name"=>"Samsung"}, "id"=>"1", "color"=>"pink"}, {"id"=>"2", "color"=>"gold"}]}

https://gist.github.com/simsalabim/6785d3c85bdd20f2e7c5

System configuration

Rails version: 5.0.0.beta3 (current master)
Ruby version: 2.3.0

Oddly enough, the gist fails on earlier versions of Rails too (4-2-stable, 4-1-stable), while our production app works fine on 4.2.5.

Could anyone please tell me what I'm missing and where is the best place to start looking for params being processed in a controller action? Thanks.

[vipulnsward]

I can confirm this on master.

[rthbound]

@simsalabim Does this behave the same in dev/production mode as it does in your test environment?

It reminds me of this: https://github.com/rack/rack/issues/951 and #23624.

[simsalabim]

@rthbound RAILS_ENV=production/test/development ruby bug_report.rb return unexpected results consistently. I debugged Rack a little last night but had no clue where to start from. Thanks for the link!

[simsalabim]

It looks exactly like rack/rack#951. I can confirm that with the following the test will pass:

{
  "items" => [
    { "q" => "s", "brand" => { "name" => "Apple" }, "color" => "pink" },
    { "q" => "s", "brand" => { "name" => "Samsung" }, "color" => "gold" },
  ]
}

The bug turns dramatic if a hash is the only element of array:

# the following will always fail
{
  "items" => [
    { "brand" => { "name" => "Apple" } },
    { "brand" => { "name" => "Samsung" } },
  ]
}

[sikachu]

Looks like this is Rack issue and it will be fixed soon. Going to keep this open in the meantime.

[dhh]

@sikachu Any update?

[sikachu]

@dhh I think we're waiting for rack/rack#1029 to be merged, then we can test against Rack master possibly.

[dhh]

Maybe @tenderlove can help us getting that committed?

On Fri, Mar 11, 2016 at 3:46 PM, Prem Sichanugrist <[email protected]

wrote:

@dhh https://github.com/dhh I think we're waiting for rack/rack#1029
https://github.com/rack/rack/pull/1029 to be merged, then we can test
against Rack master possibly.

—
Reply to this email directly or view it on GitHub
https://github.com/rails/rails/issues/23997#issuecomment-195395820.

[rthbound]

Looks like https://github.com/rack/rack/pull/1029 is now merged. Time to retest?

[rthbound]

The tests in the reproduction script now pass against master.

[vipulnsward]

Thanks, this is now pending a rack release.

[dhh]

@tenderlove Can you push out a new rack release to close this out?

[jrabramson]

Using the newest rack seems to make the hash munging even worse, any update on this?

[vipulnsward]

@jrabramson do you mean you see this on latest rack master?

[jrabramson]

yes, it parses hashes inside of arrays even worse than the beta3 rack

EDIT: some sample data;

{
      data: {
        type: "thing",
        attributes: {
          type: "subthing",
          id: "6f8a6546-5822-4ed5-a5c5-25420dbae498",
          cards: [
            {
              default: true,
              layers: [
                {
                  type: "title",
                  attributes: {
                    text: "You've Got Mail!"
                  }
                },
                {
                  type: "topic-subject",
                  attributes: {
                    url: "test"
                  }
                }
              ]
            }
          ]
        }
      }
    }

becomes:

{"data"=>
  {"attributes"=>
    {"cards"=>
      [{"default"=>"true", "layers"=>[{"attributes"=>{"text"=>"You've Got Mail!"}}]},
       {"layers"=>[{"type"=>"title"}]},
       {"layers"=>[{"attributes"=>{"url"=>"test"}}]},
       {"layers"=>[{"type"=>"topic-subject"}]}],
     "id"=>"6f8a6546-5822-4ed5-a5c5-25420dbae498",
     "type"=>"subthing"},
   "type"=>"thing"}}

[haggen]

I'm getting something similar in 4.2.5:

This:

{
  thing: [
    { stuff: [1, 2] },
    { stuff: [3, 4] }
  ]
}

Ends up like this:

{
  thing: [
    { stuff: [1, 2, 3, 4] },
    { }
  ]
}

Is there a working version of rack I could use?

[jrabramson]

@haggen if you want to make it work the way you'd expect for now, you can change first_key to child_key in /lib/rack/query_parser.rb line 100

[haggen]

@jrabramson That's odd. The file you mentioned only exist in the branch master (version 2.0.0-beta) in rack repository. But Rails < 5 uses 1.6-stable. Thought it was worth mentioning.

I could locate the code you're talking about in file the utils.rb. I'm trying your suggestion right now.

Thanks a lot!

Edit. Actually the line you mentioned is wrong https://github.com/rack/rack/blob/master/lib/rack/query_parser.rb#L100 Did you mean the line 106?

if params_hash_type?(params[k].last) && !params[k].last.key?(first_key)

Also if it's fixed in rack why the issue still open here?

[jrabramson]

Ahh yea, in that version it hadn't seen seperated out yet.

[haggen]

@jrabramson I'm afraid it didn't work.

Here's what I did: I took this patch and applied to the file utils.rb in the branch 1.6-stable, fixing the line numbers first. It kinda fixed one problem, but created another, now instead of this:

{
  thing: [
    { stuff: [1, 2, 3, 4] },
    { }
  ]
}

I got this:

{
  thing: [
    { stuff: [1] },
    { stuff: [2] },
    { stuff: [3] },
    { stuff: [4] },
  ]
}

See how it respected the first [] in the field name, creating new objects instead o merging into the first. But now it splits the array inside the object into multiple parent objects.

My field's name is ...[thing][][stuff][].

Then I tried changing back first_child to child_key as you mentioned (but kept the other changes described in the patch I linked above) and it got back to the previous problem.

I'm sad. 😢

[jrabramson]

Looking at the structure of what you want, I think it's impossible with the way rack currently parses nested hashes

[haggen]

@jrabramson Well, that sucks. Thanks a lot for the heads up!

[jrabramson]

@haggen can you pass the data as a json body instead of params?

[haggen]

@jrabramson Guess I have to now. 😝 It's actually a jsonb column but I was using fields_for otherwise I'd have to use JS to generate the JSON before submitting it in a hidden field or something. Thanks a lot for all the help, really appreciate it!

[vivid]

Hi all! I've stumbled upon a similar problem - not sure if I have to report it as a separate issue.

In short: when submitting multipart form data with hash that has an element with blank value (i.e. blank text field of a form), it doesn't appear in params (the key is missing). When submitting the same form not as a multipart form data, the field presents in the hash (which is a correct behavior). Attached is a test script demonstrating both scenarios.

[vivid]

Update: after switching to use rack version from the master test passes, so pls ignore it :)

[tenderlove]

Sorry if this is a bad question, but this test:

post '/', params: {
  "items" => [
    { "brand" => { "name" => "Apple" }, "id" =>1, "color" => "pink" },
    { "brand" => { "name" => "Samsung" }, "id" => 2, "color" => "gold" },
  ]
}

How does your app actually post data like that in production? I'm having a hard time thinking of a way to make a form that posts an array of nested hashes.

[haggen]

@tenderlove If I remember correctly this does the trick:

<%= f.fields_for :items, [OpenStruct.new], index: nil do |f| %>
  <%= f.fields_for :brand, OpenStruct.new do |f| %>
    <%= f.text_field :name %>
  <% end %>
<% end %>

Now if you're specifically referring to the fact that items is a direct key of params I don't think I've ever seen it.

[tenderlove]

OK. I think the parameter parser in Rack has the same ambiguity (with regard to this query) as it did in previous versions. The change I made was that params would get encoded in the controller test and then decoded again so it's more similar to production behavior. So, I think the correct fix is to make this type of controller test post the same data that the form would post in production. Is there any way you can show me a sample form and a sample POST request? I can probably do it, but if you happen to have that it would be a bit faster. Thank you!!

[tenderlove]

@haggen does Rails process the request correctly for that form? I'm trying to figure out a situation where it would be possible for Rails to generate and post the data structure that @simsalabim is showing.

[simsalabim]

How does your app actually post data like that in production? I'm having a hard time thinking of a way to make a form that posts an array of nested hashes.

@tenderlove this is a good question. It's an API call, hence no form.

[sgrif]

It's an API call, hence no form

Could you not just use JSON then and eliminate the ambiguities?

[tenderlove]

@tenderlove this is a good question. It's an API call, hence no form.

Thanks. Are you posting JSON, or are you encoding to the query params? The reason I'm asking is because the Rails parameter + Rack query parameter parser has never supported a structure like this. For example:

require 'active_support/all'
require 'rack/utils'
require 'rack'

p Rack.release

x = {
  "items" => [
    { "brand" => { "name" => "Apple" }, "id" =>1, "color" => "pink" },
    { "brand" => { "name" => "Samsung" }, "id" => 2, "color" => "gold" },
  ]
}.to_query
generated = CGI.unescape(x)

p generated

p Rack::Utils.parse_nested_query generated

If you run this against older versions of Rack, it won't successfully round trip. The thing that changed in Rails is that we encode the parameters in the controller test, then decode them in order to make the test similar to what actually happens in production. If this doesn't round trip in your test, I suspect that the test isn't correct. That is why I asked if there is a form or something that is able to produce this data structure. If you're using an API endpoint, I would like to know how you are actually encoding the parameters (whether that is via JSON or query parameters).

Thank you!

[bmedenwald]

I suspect this is related, but here's a different payload that exhibits the problem better. I've boiled this down a ways, but it comes from a subset of Stripe's web hooks, which are sent as query parameters and not as a JSON body.

require 'active_support/all'
require 'rack/utils'
require 'rack'

p Rack.release

x = {
 "data" => [
  { "id" => "sub_7fIggpcyHh1", "period" => { "start" => 1452012860, "end" => 1455900860 }, "plan" => { "id" => "Plan-A", "object" => "plan" } },
  { "id" => "sub_7fIggpcyHh1", "period" => { "start" => 1452012860, "end" => 1455900860 }, "plan" => { "id" => "Plan-B", "object" => "plan" } },
 ]
}.to_query
generated = CGI.unescape(x)

p generated

p Rack::Utils.parse_nested_query generated

If I run the above utilizing Rails 4.2.6 and Rack 1.6.4, it ultimately comes out correct. It's an array with 2 objects, each with a key/value and 2 objects.

{"data"=> [
  {"id"=>"sub_7fIggpcyHh1",
    "period"=>{"end"=>"1455900860", "start"=>"1452012860"},
    "plan"=>{"id"=>"Plan-A", "object"=>"plan"}},
  {"id"=>"sub_7fIggpcyHh1DfD",
    "period"=>{"end"=>"1455900860", "start"=>"1452012860"},
    "plan"=>{"id"=>"Plan-B", "object"=>"plan"}}
]}

but in Rails 5.0.0.rc1 and Rack 2.0.0.rc1 this is what I'll receive in the last step. It's now an array of 5 objects, all the objects are different. It changes the meaning pretty extensively.

{"data"=> [
  {"id"=>"sub_7fIggpcyHh1", "period"=>{"end"=>"1455900860"}},
  {"period"=>{"start"=>"1452012860"}, "plan"=>{"id"=>"Plan-A"}},
  {"plan"=>{"object"=>"plan"},
    "id"=>"sub_7fIggpcyHh1DfD",
    "period"=>{"end"=>"1455900860"}},
  {"period"=>{"start"=>"1452012860"}, "plan"=>{"id"=>"Plan-B"}},
  {"plan"=>{"object"=>"plan"}}
]}

[bmedenwald]

@tenderlove Here is another example, this time from a nested form with models using accepts_nested_attributes_for 2 levels deep. The models look like this:

class Company < ActiveRecord::Base
  accepts_nested_attributes_for :groups, allow_destroy: true
end

class Group < ActiveRecord::Base
  accepts_nested_attributes_for :memberships, allow_destroy: true
end

class Membership < ActiveRecord::Base
end

The payload from a single form that allowed both CRUD of Groups and Memberships to those Groups generates a payload like this:

x = {
  "company" => {
    "groups_attributes" => [
      { "id" => 1, "memberships_attributes" => [ { "id" => 1, "_destroy" => true } ] },
    ]
  }
}.to_query

Using the same script above, this works with Rails 4.2.6/Rack 1.6.4, providing a single object in the groups_attributes array:

{ "company" => { "groups_attributes" => [ { "id" => "1", "memberships_attributes" => [ { "_destroy" => "true", "id" => "1" } ] } ] } }

But doesn't in Rails 5.0.0.rc1/Rack 2.0.0.rc1, splitting them up into 2 items in the groups_attributes:

{ "company" => { "groups_attributes" => [ { "id" => "1", "memberships_attributes" => [ { "_destroy" => "true" } ] }, { "memberships_attributes" => [ { "id" => "1" } ] } ] } }

It feels like this issue belongs back in the 5.0.0 milestone. A data structure like this may not be common, but for those that do it's a big regression.

[tenderlove]

@bmedenwald not sure if this _is_ the same issue that OP reported, but I have been able to reproduce the case you've given. We were able to roundtrip the query you provided with Rack 1.4.6, but not on 2.0.0. I tracked down the problem to rack/rack@7f00781a7b9c75ddb86d0ab52132885ee88d5bac . Unfortunately reverting that commit breaks other tests. I'm tempted to revert rack/rack#1029, but that will cause rails/rails#23624 to be unfixed.

Fixing one bug causes another, but I'd rather _not_ break existing apps than fix something already known to be broken.

@matthewd @jeremy any opinions?

[tenderlove]

@bmedenwald also thank you very much for providing a test case that would break in one version and not another. I really appreciate it!

[tenderlove]

require 'rack/utils'
require 'rack'
require 'minitest/autorun'
require 'cgi'

p Rack.release

class MyTest < Minitest::Test
  def query
    "data[][id]=sub_7fIggpcyHh1&data[][period][end]=1455900860&data[][period][start]=1452012860&data[][plan][id]=Plan-A&data[][plan][object]=plan&data[][id]=sub_7fIggpcyHh1&data[][period][end]=1455900860&data[][period][start]=1452012860&data[][plan][id]=Plan-B&data[][plan][object]=plan"
  end

  def test_round_trip
    x = {
      "data" => [
        { "id" => "sub_7fIggpcyHh1", "period" => { "start" => '1452012860', "end" => '1455900860' }, "plan" => { "id" => "Plan-A", "object" => "plan" } },
        { "id" => "sub_7fIggpcyHh1", "period" => { "start" => '1452012860', "end" => '1455900860' }, "plan" => { "id" => "Plan-B", "object" => "plan" } },
      ]
    }

    generated = CGI.unescape(query)

    assert_equal x, Rack::Utils.parse_nested_query(generated)
  end
end

The output of to_query on x in this test case didn't change between Rails releases, but parsing did.

[tak1n]

I'm not sure if this issue already handles this case but I currently was in process of upgrading one of our apps to rails 5. The problem is nested attributes are broken on rails 5.

I made a public available reproduction app: https://github.com/tak1n/nested_attributes_reproduction

I have a integration test which hits an update action:
https://github.com/tak1n/nested_attributes_reproduction/blob/master/test/integration/book_test.rb

In this update action I have a binding.pry where I get following params:

./bin/rake test TEST="test/integration/book_test.rb" TESTOPTS="--name='/BookTest#test_something$/'"
Run options: "--name=/BookTest#test_something$/" --seed 62421

# Running:


From: /home/benny/dev/onlim/testapp/app/controllers/books_controller.rb @ line 3 BooksController#update:

    2: def update
 => 3:   binding.pry
    4: end

[1] pry(#<BooksController>)> params
=> <ActionController::Parameters {"book"=>{"title"=>"Cool", "pages_params"=>[{"id"=>"1", "content"=>"another content"}, {"id"=>"2", "_destroy"=>"1", "content"=>"another new page"}]}, "format"=>"json", "controller"=>"books", "action"=>"update", "id"=>"1"} permitted: false>

It should be 3 page_params hashes but instead it merged the hash for deleting and the hash for creating a new page into one.

This {"id"=>"2", "_destroy"=>"1", "content"=>"another new page"} should be:

{"id"=>"2", "_destroy"=>"1"},
{"content"=>"another new page"}

[javierjulio]

@tak1n I checked your test file https://github.com/tak1n/nested_attributes_reproduction/blob/master/test/integration/book_test.rb but I believe there is an issue because this definitely does work for me (same case) in Rails 5 integration tests but you need to specify as: :json there, the format option is for controller tests. Change the code to what you see below and see if it passes:

  test "something" do
    params = {
      book: {
        title: 'Cool',
        pages_params: [
          { id: @page.id, content: 'another content' },
          { id: @delete_page.id, _destroy: 1 },
          { content: 'another new page' }
        ]
      }
    }

    put book_url(@book.id), params: params, as: :json
  end

Note that the as: :json is a separate hash from params entirely. Has to be separate based on docs. Also while the URL does have .json at the end I don't believe that is sufficient as when I first started doing this I looked at what happens under the hood and the test seems to really do certain things based on the as: option being set. Give it a try and let me know if that works.

[cvoege]

Is there any progress on fixing the actual parsing here?

Currently, if you try to post a FormData object via AJAX, it will randomly break the entire structure because of this issue. This means that if you try to post a form with any sort of nested structure it will completely break the structure. Normally I just AJAX JSON, which doesn't suffer from this issue, but any time I upload a file I need to use FormData and that means I have to use the broken and unpredictable parser. What's the recommended solution to this?

[repinel]

This only seems to happen on the test environment. Git bisect pointed my to c546a2b. Here is one more test case:

begin
  require "bundler/inline"
rescue LoadError => e
  $stderr.puts "Bundler version 1.10 or later is required. Please update your Bundler"
  raise e
end

gemfile(true) do
  source "https://rubygems.org"
  gem "rails", github: "rails/rails"
end

require "action_controller/railtie"

class TestApp < Rails::Application
  config.root = File.dirname(__FILE__)
  secrets.secret_token    = "secret_token"
  secrets.secret_key_base = "secret_key_base"

  config.logger = Logger.new($stdout)
  Rails.logger  = config.logger

  routes.draw do
    patch "/" => "test#update"
  end
end

class TestController < ActionController::Base
  include Rails.application.routes.url_helpers

  def update
    render plain: test_params.to_h.inspect
  end

  def test_params
    params.require(:test).permit(children_attributes: [:id, :name, :description, :_destroy])
  end
end

require "rails/test_help"
require "minitest/autorun"

class TestControllerTest < ActionController::TestCase
  test "accessing cookies directly" do
    patch :update, id: 999, test: {
      children_attributes: [{ name: "New Hello", description: "foo" },
                            { id: 1, name: "Hi", description: "goo" },
                            { id: 2, _destroy: true }]
    }
    assert response.ok?
    assert_equal '{"children_attributes"=>[{"name"=>"New Hello", "description"=>"foo"}, {"id"=>"1", "name"=>"Hi", "description"=>"goo"}, {"id"=>"2", "_destroy"=>true}]}', response.body
  end
end

Rails version: 5.0.3, 5.1.1, master
Ruby version: 2.4.1
rack version: 2.0.3
rack-test version: 0.6.3

[repinel]

Actually, the reason it was only failing in the tests for me is because the they were really not matching production.

Here is the corrected test:

patch :update, id: 999, test: {
  children_attributes: {
    "0" => { name: "New Hello", description: "foo" },
    "1" => { id: 1, name: "Hi", description: "goo" },
    "2" => { id: 2, _destroy: true }
  }
}

Sorry about the confusion.

[arg]

Currently, if you try to post a FormData object via AJAX, it will randomly break the entire structure because of this issue. This means that if you try to post a form with any sort of nested structure it will completely break the structure. Normally I just AJAX JSON, which doesn't suffer from this issue, but any time I upload a file I need to use FormData and that means I have to use the broken and unpredictable parser. What's the recommended solution to this?

I can confirm the issue is reproducible on Rails 5.1.1. My similar scenario:

class Item < ApplicationRecord
  has_many :images
  accepts_nested_attributes_for :images
  # some attributes: name, description, etc.
end

class Image < ApplicationRecord
  # includes attachments logic powered by Shrine gem (not related to the issue)
  # attachment data is stored in "asset_data" attribute
  # also has some authorship information fields (owner, copyright, etc.)
end

Form data:

item[name] = 'name'
item[description] = 'description'
item[images_attributes][][asset] = File1,File2 # upload with <input type='file' name='item[images_attributes][][asset]' multiple>
item[images_attributes][][owner] = 'owner name 1'
item[images_attributes][][copyright] = 'copyright info 1'
item[images_attributes][][owner] = 'owner name 2'
item[images_attributes][][copyright] = 'copyright info 2'

Expected params:

item: {
  name: 'name',
  description: 'description'
  images_attributes: [
    {
      asset: File1
      owner: 'owner name 1'
      copyright: 'copyright info 1'
    },
    {
      asset: File2
      owner: 'owner name 2'
      copyright: 'copyright info 2'
    }
  ]
}

Actual params:

item: {
  name: 'name',
  description: 'description'
  images_attributes: [
    {
      asset: File1
    },
    {
      asset: File2
      owner: 'owner name 2'
      copyright: 'copyright info 1'
    },
    {
      copyright: 'copyright info 2'
    }
  ]
}

The actual result is corrupted. Notice the missing owner name 1, incorrect copyright info disposition, additional element with only copyright info 2.

[jsantos]

Not sure if this is related, but here's my current issue:

I have the following route, intervals#bulk_update:

resources :intervals, only: [:index, :update, :destroy] do
    collection do
      put :bulk_update
  end
end

And the following payload is sent over JSON (test subject below):

put :bulk_update, params: {
  intervals: [
    {
      id: @stop.id,
      interval_type: 'stop',
      stop: {
        id: @stop.intervalable.id,
      },
      start_time: @stop.start_time,
      end_time: @stop.end_time
    },
    {
      id: @trip.id,
      interval_type: 'trip',
      trip: {
        id: @trip.intervalable.id,
        mode_id: modes(:foot).id
      },
      confirmed: true,
      start_time: @trip.start_time,
      end_time: @trip.end_time
    }
  ]
}, format: :json

That confirmed field is a bit spontaneous, so it doesn't show up on every request. Here's what I get on the controller params:

{
  "intervals": [
    {
      "end_time": "2017-02-10 06:06:14 UTC",
      "id": "428",
      "interval_type": "stop",
      "start_time": "2017-02-10 04:06:15 UTC",
      "stop": {
        "id": "186"
      },
      "confirmed": "true"
    },
    {
      "end_time": "2017-02-10 06:26:14 UTC",
      "id": "429",
      "interval_type": "trip",
      "start_time": "2017-02-10 06:06:14 UTC",
      "trip": {
        "id": "180",
        "mode_id": "4"
      }
    }
  ],
  "format": "json",
  "controller": "api/v2/intervals",
  "action": "bulk_update"
}

It's hard to notice, but the confirmed field jumped from the second array element to the first one...

However, if I switch format: :json to as: :json, the following goes on params:

{
  "intervals": [
    {
      "id": 433,
      "interval_type": "stop",
      "stop": {
        "id": 189
      },
      "start_time": "2017-02-10T04:06:15.000Z",
      "end_time": "2017-02-10T06:06:14.000Z"
    },
    {
      "id": 434,
      "interval_type": "trip",
      "trip": {
        "id": 181,
        "mode_id": 4
      },
      "confirmed": true,
      "start_time": "2017-02-10T06:06:14.000Z",
      "end_time": "2017-02-10T06:26:14.000Z"
    }
  ],
  "format": "json",
  "controller": "api/v2/intervals",
  "action": "bulk_update",
  "interval": {
  }
}

This time the confirmed field ended up in the right place, but I got an extra "inteval": {} on the params hash. Is this expected?

Another curious thing (that I verified while writing this): If I keep format: :json and just name the parameter something else, results vary; so far what I noticed if that if I name it in a way that goes alphabetically after end_time, like verified, the param goes on the right place...

Ruby version: 2.3.3
Rails version: 5.0.4
Rack version: 2.0.3

[mockdeep]

I'm not sure if it's the same issue, but we're seeing something similar when upgrading to Rails 5.0. We have a test that looks like:

put(:update, params: valid_params)

in the test, valid_params is:

{
  :id=>"update_recipients",
  :routing_wizard=>{
    :steps=>[
      {
        :recipients_attributes=>[{:name=>"Homer Simpson", :email=>"[email protected]"}],
        :type=>"RecipientStep",
        :seq=>2
      },
      {
        :recipients_attributes=>[{:name=>"Marge Simpson", :email=>"[email protected]"}],
        :type=>"RecipientStep",
        :seq=>3,
        :skippable=>true,
        :label=>"blah!"
      }
    ]
  }
}

and on the server they come out as:

{
  "routing_wizard"=>{
    "steps"=>[
      {
        "recipients_attributes"=>[
          {"email"=>"[email protected]", "name"=>"Homer Simpson"},
          {"email"=>"[email protected]", "name"=>"Marge Simpson"}
        ], 
        "seq"=>"2", "type"=>"RecipientStep", "label"=>"blah!"
      },
      {"seq"=>"3", "skippable"=>"true", "type"=>"RecipientStep"}
    ]
  },
  "id"=>"update_recipients",
  "controller"=>"routing_wizard",
  "action"=>"update"
}

Notice that somehow the recipients_attributes from both steps got lumped together under the first step on the server. We're not using json, as this is a normal form post. It works fine when I walk through this flow on our app directly.

[mockdeep]

Okay, after doing quite a bit of digging, I've learned that this issue is related to the fact that the the ActiveSupport Hash#to_query extension sorts the query parameters before joining them. When I try removing the sort! the results show up on the server as expected. It looks like Rack::Utils.parse_nested_query handles things differently depending on the order of the params. I came across this other issue which looks like the same thing but was closed a couple of years ago.

[lunaru]

Is there an update on the status of this? It's still reproducible under Rails 5.1.5 and breaks even if the tests use format: :json

Update: I missed @javierjulio's comment above, but switching from format: :json to as: :json fixes the problem.

Also referenced in https://github.com/rspec/rspec-rails/issues/1700

[tenderlove]

I think we can close this because it was fixed here: https://github.com/rails/rails/pull/33093

If not, lets reopen or make a new ticket. Thanks!

[amex-doug]

@tenderlove Is there a way to address this issue for Rails 4? I tried the as solution and it seems like my endpoint doesn't detect the params and the headers at all.

post path, params: params, headers: valid_header, as: :json

Using rails 4.2.11.1 with rack 1.6.12