Rails: MissingTemplate issues in Rails 3.0

[rishav]

This issue was reported earlier, but its not fixed in 3.0.x latest and some people have reported in 3.1.x as well.

https://github.com/rails/rails/issues/701

[hlxwell]

I think we somehow need to

1. Let Mime parse _/_, like Mime["text/*"] => text/html, text/json, text/xml
2. Find exist template which match to the mime type list.
3. If find the template, render it, or else, raise "MissingTemplate" error.

Another thing, I found adding a respond_to :xxx still not works for the action without explicit render method.

[softwaregravy]

A good suggestion from the previous thread was to introduce the concept of a default mime type to avoid raising missing template errors. In the instances I observed, the requests that weren't specifically requesting JSON or XML would be properly satisfied by behaving as if they requested HTML.

I think it would be reasonable to have a default to HTML, and provide a hook to override/remove the default.

[hlxwell]

@softwaregravy The "default mime type" only could be used in Configuration, because, "HTML" doesn't meet all the needs of us. like API project, people request "text/*", default to html, will raise error also. because all we provide in that project is json or xml.

If it's code in Rails, it should be intelligent a little bit. :)
suggestions?

[hlxwell]

I think someone did "implement code that handles text/_, appplication/_, and image/*" in commit 6f6e754bac4c78f657feb0ea119447546aa87197 by one year ago, I saw it in latest rails source code. but not for 3.1.3.

Above 3.2, Mime::Type.parse("text/*") give correct list.

[hlxwell]

ok, I think we've been involved into a wrong direction.

It's NOT about "rendering correctly for the mime type of the text/*".
It's about "when no template and explicit render method is gaven, we should give missing template error, or else, should give 406, instead of 500 - missing template error".

For example:

http://localhost/books.json <= we only provide html template, but didn't write respond_to :html, it will give MissingTempate, but not 406. and the code even won't go to inquire "accepts" here, since we've gaven content_type.

[hlxwell]

This is what I did in my project. it works.

  rescue_from ActionView::MissingTemplate do |exception|
    all_supported_extensions = Mime::SET.map(&:symbol)

    # if unsupported format return 406
    unsupported_format = all_supported_extensions.include?(request.format.to_sym)

    # if provided at least one other format return 406
    provided_other_format = all_supported_extensions.map { |format|
      @lookup_context.find_all("#{exception.path}.#{format}")
    }.flatten.compact.any?

    if unsupported_format or provided_other_format
      head :not_acceptable and return
    end

    # if hasn't any template and no explicit `render` method re-raise the error
    raise exception
  end

[rishav]

Can some one shed some light on that fact the why is this an issue in 3.0.x , how were we handling such issues in 2.x ? May be some one from Core

[hlxwell]

This could be a solution to fix this problem in Rails 3.0.x

module ActionController
  module ImplicitRender

    def default_render
      # lookup template exist? go to render it
      render and return if template_exists?(action_name.to_s, _prefix)

      has_template =  begin
                        old_formats, @lookup_context.formats = @lookup_context.formats, Mime::SET.symbols
                        template_exists?(action_name.to_s, _prefix)
                      ensure
                        @lookup_context.formats = old_formats
                      end

      # if lookup template not exist then go to check if any template existed,
      # if so show 406, if not, go render, it will give MissingTemplate error.
      has_template ? head(:not_acceptable) : render
    end

  end
end

[hlxwell]

https://github.com/rails/rails/pull/4176

[ippa]

Using exception_notification and rails 3.2 I get tons of "ActionView::MissingTemplate"-mails from various odd user agents like:

---

HTTP_USER_AGENT : Mozilla/4.0 (PSP (PlayStation Portable); 2.00)

HTTP_ACCEPT : _/_;q=0.01

---

HTTP_USER_AGENT : Mozilla/4.0 (MobilePhone SCP-3800/US/1.0) NetFront/3.4 MMP/2.0

HTTP_ACCEPT : application/vnd.wap.xhtml+xml, application/x-pmd, application/vnd.phonecom.mmc-xml, audio/midi, audio/vnd.qcelp, application/xhtml+xml; profile="http://www.wapforum.org/xhtml", multipart/vnd.sprint-pre-cache, multipart/mixed; q=0.5, multipart/related,text/vnd.wap.wml, text/vnd.sun.j2me.app-descriptor, text/x-pcs-gcd, text/css, application/ecmascript, text/ecmascript, image/png, image/gif, image/jpeg; q=0.5, image/vnd.wap.wbmp; q=0.2, audio/qcelp, application/pmd, audio/mid, text/plain

---

HTTP_USER_AGENT : Motorola-V3m-Red Obigo/Q04C1 MMP/2.0 Profile/MIDP-2.0 Configuration/CLDC-1.1

HTTP_ACCEPT : application/xhtml+xml; profile="http://www.wapforum.org/xhtml", application/vnd.wap.xhtml+xml, image/png, image/gif, image/jpeg; q=0.5, image/x-bmp; q=0.3, image/vnd.wap.wbmp; q=0.2, audio/midi, audio/vnd.qcelp, audio/qcelp, audio/aac, audio/aacPlus, audio/amr, audio/mp3, audio/mpeg, audio/imy, audio/imelody, audio/mp4, audio/mp4a-latm, audio/3gpp, audio/3gpp2, video/mp4, video/mp4v-es, video/3gpp, video/3gpp2, application/x-pcs-mcd+xml, application/sdp, application/x-pmd, application/x-cmx, application/pmd, multipart/mixed; q=.1, application/vnd.wap.wmlscriptc, application/vnd.wap.wmlc, text/vnd.wap.wml, text/css, text/vnd.sun.j2me.app-descriptor, text/x-pcs-gcd, text/plain, text/ecmascript, application/ecmascript, multipart/vnd.sprint-pre-cache

Trying the "request.format = :html" sollution now.

[hlxwell]

@ippa I think this is not a correct solution.

The correct way to do it is to add "respond_to :html" for the actions. as josevalim said.

(P.S. although I think adding respond_to for all actions is pretty awkward.)

[ippa]

@hlxwell: okey... the exception-mail-spam stopped anyhow, and the pages still render fine in all browsers I can test. I'll test your / josevalims suggestion as well.

[ippa]

false positive, "request.format = :html" didn't stop the mails. Trying "respond_to" now.

[StefanH]

Having a similar issue in rails 3.2.1 when a client sends an accept header like */*;q=0.6 (as the google bot does occasionally). It seems that Mime::Type.parse doesn't work correctly for accept headers with q values and only a single mime-type. I looked into the code of action_dispatch/http/mime_type.rb and when I change line 114 to this:

Mime::Type.lookup(accept_header[/^[^;]*/])

The */*;q=0.6 is handled correctly. This seems like the correct way to parse an accept header with one mime-type and a q value, but I don't know enough about mime type handling in rails to be sure.

[hlxwell]

      # input: 'text'
      # returned value:  [Mime::JSON, Mime::XML, Mime::ICS, Mime::HTML, Mime::CSS, Mime::CSV, Mime::JS, Mime::YAML, Mime::TEXT]
      #
      # input: 'application'
      # returned value: [Mime::HTML, Mime::JS, Mime::XML, Mime::YAML, Mime::ATOM, Mime::JSON, Mime::RSS, Mime::URL_ENCODED_FORM]
      def parse_data_with_trailing_star(input)
        Mime::SET.select { |m| m =~ input }
      end

Look at this, the latest code on master has this to convert _/_ to correct mime.

[StefanH]

@hlxwell I was not talking about parse_data_with_trailing_star (actionpack/lib/action_dispatch/http/mime_type.rb:185), but about parse (actionpack/lib/action_dispatch/http/mime_type.rb:109), which doesn't use parse_data_with_trailing_star in this case, where you have an accept header with only one mime-type and a q-value. It is the same in rails 3.2.

[phallstrom]

Just to get this into the search engines... An accept header like the following, while valid, will also trigger the MissingTemplate error. Specifically, the "; profile=..." bit. The spec says this is valid, but Rails does not like it at all. http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html

curl -Iv -H 'Accept: application/xhtml+xml; profile="http://www.wapforum.org/xhtml", application/vnd.wap.xhtml+xml'

(edit) Digging into it further, it appears Rails doesn't like anything other than 'q' as an extension to the mime type.

[rafaelfranca]

Hey guys. Could you confirm if this issue was fixed in the 3.2? Thanks.

[rchampourlier]

I'm facing this issue with Rails 3.0.12 and found no fix. I looked at this StackOverflow question, but it wasn't solving the problem correctly.

So I've investigated the issue and implemented a solution relying on a Rack middleware which "corrects" the HTTP_ACCEPT header (currently it's for GoogleBot's */*;q=0.6 and similar ones, but it can be easily adapted to match other configurations). It just removes this unnecessary q=0.6 separator and let alone the */*, which is correctly processed by Rails.

You can find the middleware, an integration test and an initializer in this gist. Enjoy!

[rafaelfranca]

I'm closing this issue since seems that it was fixed in 3-2-stable branch

[vitamino]

I still have this problem in 3.2.1

A ActionView::MissingTemplate occurred in site#index:

Missing template site/index, application/index with {:locale=>[:it], :formats=>[:jpeg, "image/pjpeg", :png, :gif], :handlers=>[:erb, :builder, :prawn, :prawn_dsl, :prawn_xxx]}. Searched in:

• "/app/app/views"
• "/app/vendor/bundle/ruby/1.9.1/gems/devise-2.0.0/app/views"

vendor/bundle/ruby/1.9.1/gems/actionpack-3.2.1/lib/action_view/path_set.rb:58:in `find'

I tried to add respond_to :html to the controller without luck.

Any ideas?

[antage]

I have this issue in 3.2.8: ActionView::MissingTemplate: Missing template teasers/index, application/index with {:locale=>[:en], :formats=>["hc/url;*/*"], :handlers=>[:erb, :builder, :jbuilder, :haml]}

[steveklabnik]

@antage can you give us a way to reproduce? If so, I'll re-open.

[antage]

@steveklabnik It's simple, just send a request with curl:

curl -H"Accept: hc/url;*/*" http://localhost:3000/

or

curl -H"Accept: hc/url; */*" http://localhost:3000/

[steveklabnik]

What about an app to make that request to?

[antage]

I tested 5 different applications (4 use 3.2.8, 1 uses 3.2.7). All have this issue.
I have tried to create a new application with single template for application#test action. It has the issue also.

[steveklabnik]

I have tried to create a new application with single template for application#test action. It has the issue also.

Okay awesome! can you push this up to github, please?

[antage]

@steveklabnik No problem.
Repo - https://github.com/antage/rails-issue4127
Command to reproduce the issue:
curl -H"Accept: hc/url;*/*" http://localhost:3000/

[steveklabnik]

Thank you. We have almost 400 issues, having a ready-made example really helps.

[purp]

Note: one possible hack to help get you through is:

class FooController
  rescue_from  ActionView::MissingTemplate, :with => :missing_template

  def missing_template
    render :nothing => true, :status => 406
  end
end

It's not great to manage content negotiation on template lookup but it'll do until there's a long-term solution.

One interesting alternative would be:

class FooController
  respond_only_to :html
end

... and have that cause content negotiation to fail for non-HTML requests unless a method specifies an exception using #respond_with. I'd offer a patch but my eyes are still bleeding from following a stacktrace through ActionController::Metal::* and AbstractController::Render* trying to understand how this works.

[purp]

What I ended up doing:

class ApplicationController < ActionController::Base
  rescue_from ActionView::MissingTemplate, :with => :missing_template

  def missing_template(exception)
    if exception.is_a?(ActionView::MissingTemplate) &&
      !Collector.new(collect_mimes_from_class_level).negotiate_format(request)
      render :nothing => true, :status => 406
    else
      logger.error(exception)
      render_500
    end
  end

If there's a better way, I'd love to hear about it.

[sfsekaran]

@purp Why the need for !Collector.new(...).negotiate_format(request) there?

Any reason you don't just do the following instead of render :nothing => true, status => 406?:

raise ActionController::RoutingError.new('Not Found')

Seen here: http://stackoverflow.com/a/4983354/241367

[purp]

@sfsekaran: The Collector is the class that actually provides format negotiation; that statement tells you whether there is an acceptable MIME format available to you. If there is no acceptable format available, it returns nil, indicating that you cannot provide any acceptable response to this request. In that case, you're expected to return an error with status 406: Not Acceptable. The crux of this issue is simply that, in certain circumstances, Rails does not recognize an unacceptable request before attempting template lookup.

An ActionController::RoutingError returns status 404: Not Found which isn't appropriate.

[sfsekaran]

Thanks @purp for the succinct explanation. Understood.

[myobie]

What circumstances cause Rails to not know it's unacceptable until it's too late? I'd really like to work on fixing this, but I can't seem to begin to find where things are going wrong.

[noahlh]

Just a quick check-in -- I'm on Rails 3.2.12 and am having this exact issue with a production app. I've got a bunch of requests coming in that have an HTTP_ACCEPT header of something like "image/*;w=74;h=74" that are throwing this exception.

I used the code that @purp wrote above in my ApplicationController and it's solving it nicely (until there's a better solution).

[purp]

@myobie Look at the comment from @antage above. He reduced it concisely.

[myobie]

@purp I made a middleware to replace a set of accept headers with text/html and prepended it before all the other middleware. The whole thing is weird though because I'm sure every rails app has this issue.

[robvandenbogaard]

Is this closed because of inactivity?

It's still an issue as the solution of @purp is being described as "solving it nicely (until there's a better solution)". If there's something wrong with the way Rails deals with formats and Rails does indeed in some circumstances not recognize an unacceptable request before attempting template lookup, I'm not sure it should be closed.

Is it correct this is still an issue in 3.2.x? I might overlook things but the only fix I see is for formats when the Accept request-header is an empty string. (Fix #7774)

[purp]

@rishav Could you please say why you closed this issue? @antage gave a concise reduction in response to @steveklabnik asking for it; my hack is not a final solution.

[lukaszx0]

This issue is still present on 4.0.2. Reopening.

[PikachuEXE]

I found someone (or bot) requesting with images/webp
Trying @purp 's code soon

[duksis]

still there in 4.0.3 and 3.2.17

[PikachuEXE]

I tried @purp code
but I changed

!Collector.new(collect_mimes_from_class_level).negotiate_format(request)
render :nothing => true, :status => 406

to

# Need to specify `ActionController`, maybe due to the code being put in concern module
!ActionController::MimeResponds::Collector.new(collect_mimes_from_class_level).negotiate_format(request)
# Just use `head`
head 406

[duksis]

actually the images/webp problem in accept header
can easily be solved by wrapping a explicit render in:

respond_to do |format|
  format.html { render '.....' }
end

if you don't want to do the ApplicationController level rescuing.

[rails-bot]

This issue has been automatically marked as stale because it has not been commented on for at least
three months.

The resources of the Rails team are limited, and so we are asking for your help.

If you can still reproduce this error on the 4-1-stable, 4-0-stable branches or on master,
please reply with all of the information you have about it in order to keep the issue open.

Thank you for all your contributions.

[j1wilmot]

This is still an issue. Example using rails 4.1.8.

rails new test_format && cd $_
rails g scaffold user name
rails s -p 3001

In browser go to localhost:3001/users.gif

Rails will throw following:

Started GET "/users.gif" for 127.0.0.1 at 2014-11-19 15:18:02 -0500
Processing by UsersController#index as GIF
Completed 500 Internal Server Error in 3ms

ActionView::MissingTemplate (Missing template users/index, application/index with {:locale=>[:en],     :formats=>[:gif], :variants=>[], :handlers=>[:erb, :builder, :raw, :ruby, :coffee, :jbuilder]}.

Expected: Would like to see rails return an empty response with a 406 status if format is not supported.
Example of expected: https://github.com/rails.monkey

[iangreenleaf]

Confirmed still occurs in 4.2.0.

[firedev]

Also I am getting errors with the following * HTTP_ACCEPT : : */*

[123gpg321]

Confirmed still occurs in Rails 3.2.13. Raising ActionView::MissingTemplate even with a #render :status => 422, :json => {:errors => 'error'}

[gregkare]

We've ran into this issue with Rails 4.1.11: when a user sets the Accept header to "_/_;" we get a ActionView::MissingTemplate exception.

Adding defaults: { format: 'html' } to the route fixed it, but I'm not sure it's the right way, especially since it means changing all the routes in our application.

Edit: This seems to be a better solution, in a controller

before_filter :set_format_to_html

def set_format_to_html
  request.format = 'html'
end

[blarralde]

Works great @gregkare

[purp]

That workaround may result in accepting requests which aren't able to properly accept an HTML response, @gregkare, @blarralde. Works if you don't care about this condition (which you likely don't), but if the thing on the other end wants JSON or somesuch, and if you wanted to produce JSON, that before_filter might cost you some debugging time until someone finds and removes it.

[blarralde]

@purp Just found that out. I've added if request.format.to_sym.nil? to validate that no other valid format had been set. What do you think?

This is what my code looks like:

DEFAULT_RESPONSE_FORMAT = :html  
before_filter :set_default_response_format

def set_default_response_format
  request.format = DEFAULT_RESPONSE_FORMAT if request.format.to_sym.nil?
end

[purp]

Hmmm. Don't know, @blarralde. It's been a couple of years since I was deep in that code. However, @antage's noted (Oct 19, 2012) that "hc/url;_/_" tripped it, which I think wouldn't show nil. Depends on how request.format maps to the Accept header.

In the end, if it works okay for you, you're probably good. Man, we need to get this one fixed.

[blarralde]

I've tested it multiple bogus accept headers, and since they're not in https://github.com/rails/rails/blob/master/actionpack/lib/action_dispatch/http/mime_types.rb they don't have a symbol set. When a format is passed to the URL or as a accept header it takes precedence over the default response format. So:

• curl www.example.com returns HTML
• curl -H "Accept: hc/url;/" www.example.com returns HTML
• curl www.example.com/.json and curl -H "Accept: hc/url;/" www.example.com/.json return JSON if a JSON template is defined, missing template otherwise

[blarralde]

My previous solution created bugs in some parts of the app where format might be implied. So eventually I've replaced it with:

BOGUS_REQUEST_FORMATS = ['*/*;', '/;', 'hc/url;*/*']
DEFAULT_RESPONSE_FORMAT = :html  
before_filter :set_default_response_format

def set_default_response_format
  request.format = DEFAULT_RESPONSE_FORMAT if request.format.in? BOGUS_REQUEST_FORMATS
end

[akaspick]

I was experiencing this issue with bots (and could reproduce with a curl command). JSON was my default format, which was wrong. I noticed I had this in my application_controller.rb

respond_to :json, :html

Changed it to

respond_to :html, :json

And the issue is resolved.

[tannusesquerdo]

I have the same issue.

ActionView::MissingTemplate: Missing template /home/index with {:formats=>[:json, :js, "_/_"]

[lustremedia]

Having this issue in 4.2.3
HTTP_ACCEPT text/plain

Mostly from bots

[ryancheung]

I confirmed this issue in Rails 4.2 with:

curl -H"Accept: */*;" http://localhost:3000/

I found this issue was caused by a browser sending a invalid Http Accept header, which is */*;. This mean users of using this kind of buggy browser cannot visit your site.

This happens rarely unless you're in China and programmers in Baidu.com made a fucking silly mistake. And most importantly, they have a large number of users using the buggy browser they made.

Can't Rails framework just ignore this and set to the default format :html instead of throwing a exception? @dhh

[purp]

Sadly, @ryancheung, that leads to unintended consequences as a few folks in this thread have discovered. Accepting */* is legit, and usually a bot that wants to accept anything. If you change the default format, you end up killing anything that wants json (or other formats), which may work for your app, but might not work for all apps.

See this comment above and the next seven or so comments for an illustration of where it goes wrong.

[purp]

THIS IS FINALLY FIXED IN RAILS 5.0.0!

The example to reproduce this below will properly return 406 Not Acceptable in Rails 5.0.0. Huge thanks to whoever reworked content negotiation to eliminate this issue.

Here's a quick and easy way to reproduce with any Rails version prior to 5.0.0:

> rails new rails-4127 && cd rails-4127 && rails server

# New terminal
> curl -H"Accept: hc/url;*/*" -I http://localhost:3000/

If it's broken, you'll get this in the curl terminal:

HTTP/1.1 500 Internal Server Error
Content-Type: text/html; charset=utf-8
Content-Length: 141744
[...]

... and something like this in the server terminal:

Started GET "/" for ::1 at 2016-08-08 17:29:47 -0700
Processing by Rails::WelcomeController#index as hc/url;*/*
Completed 500 Internal Server Error in 17ms (ActiveRecord: 0.0ms)

ActionView::MissingTemplate (Missing template rails/welcome/index, rails/application/index with {:locale=>[:en], :formats=>["hc/url;*/*"], :variants=>[], :handlers=>[:erb, :builder, :raw, :ruby, :coffee, :jbuilder]}. Searched in:
  * "/usr/local/var/rbenv/versions/2.2.1/lib/ruby/gems/2.2.0/gems/railties-4.2.7/lib/rails/templates"
):
[...]

In Rails 5.0.0 you get this in the curl terminal:

HTTP/1.1 406 Not Acceptable
Content-Type: text/html; charset=utf-8
[...]

... and something like this in the server terminal:

Started HEAD "/" for ::1 at 2016-08-08 17:53:48 -0700
Processing by Rails::WelcomeController#index as hc/url;*/*
  Parameters: {"internal"=>true}
Completed 406 Not Acceptable in 55ms (ActiveRecord: 0.0ms)

If you still need a workaround, here's what a slightly wiser me would offer now:

class ApplicationController < ActionController::Base
  rescue_from ActionView::MissingTemplate, :with => :catch_unacceptable_requests

  def catch_unacceptable_requests(exception)
    if !ActionController::MimeResponds::Collector.new(collect_mimes_from_class_level).negotiate_format(request)
      head :not_acceptable
    else
      raise exception
    end
  end

_(about friggin' time! ;)_

[ryancheung]

@purp Thanks! This should only be worked around in my own site. For illegal accept headers, either fallback to :html or return 406.

[h8rry]

@purp I am using Rails 5 and still see "missing a template for this request format and variant". If I previously upgraded from 4 to 5, what are the things I should remove manually to fix this? Thanks!

[purp]

@h8rry I'm unclear what you're asking here. Are you saying that you implemented the fix above in Rails 4, and upon upgrading to Rails 5 you're having this problem, or is this problem with a new Rails 5 app?