Chi: Render: managing request and response payload as types

[pkieltyka]

The render subpkg is pretty cool, and its great for managing the different kinds of content types a response may need, including streams. However, one of the important parts for a maintainable API is to manage the request and response payloads (the inputs and the outputs of an endpoint).

For request payloads, one simple idea is to have a Bind() middleware used in chi's r.With() inline middleware routing, that will take a request body and unmarshal it to a kind of struct. Perhaps.

type ArticleRequest {
  ID int64 `json:"id"`
  Name string `json:"name"`
}
// ...
r.Get("/", index)
r.With(render.Request(&ArticleRequest{})).Post("/articles", newArticle)

.. something like that.. the render.Request() would make a new &ArticleRequest{} object and put it on the context under render.RequestKey or something.. its a decent plan, except it would require some kind of reflection to make the new &ArticleRequest{} object.

As for response payloads, that should be much simpler, just..

type ArticleResponse {
  *data.Article // embed the core Article type
  // add more methods on it..
}
//...
render.Respond(w, r, &ArticleResponse{a})

if someone has other ideas too that would be awesome!

[elithrar]

Thanks for raising this! Some thoughts:

• Right now there are two popular libs (amongst many others) that tackle this, with different (but large) APIs: https://github.com/mholt/binding and http://www.gorillatoolkit.org/pkg/schema - thus: _what would render do differently here_?
• I like that—from the example—it's integrated with chi, but you will undoubtedly get requests around skipping fields in the request body, up/down-casing fields, etc (take a look at the issues/PRs for gorilla/schema...).
• reflect is unavoidable if this is going to be plug-and-play.

[pkieltyka]

hey @elithrar @VojtechVitek and others, I did a bit of work on the render subpkg today. Ive written request body decoding to be a bit nicer and I introduced render.Bind() as a middleware to automatically bind a request body to a struct value/object.

https://github.com/pressly/chi/pull/85/files

look at CreateArticle and CreateArticle2 for both versions and check the router definition too. After doing it though, I like the Decode() stuff, but I'm probably going to remove the auto-binding stuff since it doesnt help that much.

[elithrar]

@pkieltyka

Overall - like it a lot. It's simple, although we should expose decoding errors to make it clear when the POST body was empty vs. we couldn't decode (for whatever reason).

[pkieltyka]

@elithrar thanks a lot - check again when you have some time, I've made further improvements to the render API that I'm pretty happy with. I did remove the Bind() trick. As well, I added a few todos for me to complete writing some methods in render that easily build response payloads from a data type and a response type.

[elithrar]

@pkieltyka Do you have the diff between the current HEAD on that branch and the prior?

[pkieltyka]

Yea it's there, just check in the branch's commits

On Oct 14, 2016, at 12:31 AM, Matt Silverlock [email protected] wrote:

@pkieltyka Do you have the diff between the current HEAD on that branch and the prior?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub, or mute the thread.

[pkieltyka]

btw, I believe with this new language change in Go 1.8 that allows struct objects to be easily converted to another type with the same columns but differing struct tags could be a very useful way to implement some of the ideas in this issue/PR

https://beta.golang.org/doc/go1.8#language for more details

[msaron]

I would like to add a suggestion. I am coming from developing apps using hapijs developed by Walmart for the backend. One really cool feature they have is that they use the Joi library to validate and convert all incoming and outgoing information. I have not seen this in any other library. Once the request reaches your handler, you can rest assured that the data you have received (via request parameters or query or headers) is validated and clean. It also checks all outgoing information is what you wanted it to be. It dramatically reduces developer effort in creating apps. I don't know if this is possible in chi, but I found it to be the most useful library in hapijs.

[VojtechVitek]

@msaron nice. I'd be interested in tag-based validator. There might be some projects like this out there, but I don't know them.

For example:

type UserRequest struct {
    Name      string `validate:"maxlen=50"`
    Username  string `validate:"maxlen=20" validate:"regexp=^[a-z0-9]+$"`
    Email     string `validate:"email"`
    AvatarURL string `validate:"url"`
    Age       int    `validate:"min=13" validate:"max=120"`
}

[msaron]

@VojtechVitek I think I have found a library that is similar to Joi. Here it is go-ozzo. I am going to give it a shot.

[elithrar]

Does that belong in chi? Why not improve mholt's lib? or gorilla/schema?

On Wed, Jan 25, 2017 at 11:55 AM msaron [e-mail] wrote:

@VojtechVitek https://github.com/VojtechVitek After reviewing almost
all of the libraries out there and reviewing their pros and cons, I have
come to the conclusion that I would like to merge the following three
libraries and we would get something like the Joi library.

1. binding library https://github.com/mholt/binding by mholt
2. A flexible validation library framework by mccoyst
   https://github.com/mccoyst/validate
3. A simple efficient and robust tag validation library by goburrow
   https://github.com/goburrow/validator/

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/pressly/chi/issues/84#issuecomment-275215049, or mute
the thread
https://github.com/notifications/unsubscribe-auth/AABIcMrBhvwDcPVXaA3efnuK_FVYas7jks5rV6ijgaJpZM4J4Gz2
.

[pkieltyka]

I don't believe it belongs in the chi core libs, but could be a very useful project to be used together

[msaron]

The problem is that they all use tag based validation (see example above by @VojtechVitek ). Tag based validation is error prone and is not at all flexible. For example, say I want to determine that a value is in a dynamically generated list of items and the list depends on the userid of the request. This would be very difficult to do using tag based validation. Another negative of tag based validation is that errors are not caught at compile time. And for date fields how would I validate the date field to check that it is not greater than the current date? We would be trying to force fit the tags implementation on dynamic values.

The go-ozzo library is the closed thing to a flexible library.

I am going to incorporate the validation rules from goburrow validator into the go-ozzo validator which have been really thought out and powerful. For example goburrow has the "min" validation rule, which applies the rule depending upon what has been passed in. If a string is passed in, then it checks for the minimum length. If a slice has been passed in, then it checks for the length of the array. If an integer has been passed in, then it checks if the value is equal to or greater than the min value. And so on...

I would strongly request that if you do develop a validation library to supplement the chi library, it should not be a tag based library. Please see the example from the Joi library in NodeJS (developed by Walmart) to see how elegant it is.

[pkieltyka]

@msaron stay tuned. I have something non-tag based and veryyyy simple for request and response paylods for chi/render.

[msaron]

@pkieltyka If possible, could you add a CSV renderer also? I have implemented it like so below, but it would be great if it was part of the chi library.

func CSV(w http.ResponseWriter, r *http.Request, v interface{}, filename string) {

  // Get result as CSV bytes using "github.com/gocarina/gocsv"
  csvContent, err := gocsv.MarshalBytes(v) 

  if err != nil {
    http.Error(w, err.Error(), http.StatusInternalServerError)
    return
  }

  newfilename := strings.ToLower(filename)
  ext := filepath.Ext(newfilename)

  if ext == ".csv" {
    newfilename = filename
  } else {
    newfilename = filename + ".csv"
  }

  w.Header().Set("Content-Type", "text/csv")
  w.Header().Set("Content-Disposition", "attachment; filename="+newfilename)
  w.Write([]byte(csvContent))
}

[VojtechVitek]

@msaron anything with third party dependency (like github.com/gocarina/gocsv above) can't make it into chi.

[msaron]

@VojtechVitek I understand that but what I was suggesting was that chi implement a CSV renderer without using any third-party library.

[VojtechVitek]

@msaron I see. Well, we accept Pull Requests. But one thing to be aware of: We tend to keep chi very minimal. And imho, CSV dependency is pretty big, even for render pkg.

[pkieltyka]

Yea, sorry I dont understand the need for a CSV renderer inside of chi. However, please do create a separate project/pkg and tell us about it. I think we should start a list for projects that are made to work with chi. Perhaps a separate PKGS.md in the repo

[pkieltyka]

@msaron Keep in mind, a nice pattern would be for you to make your own render package, and just use chi/render behind the scenes - kind of like pkg composition. Take a look at https://github.com/goware/lg/blob/master/lg.go for an example in how we get in front of the logrus package. I believe this is the best approach and lets you take control for all your renderering needs. This is also my intention with chi -- to always give control to the dev, no framework lock-in, and we do that by design.

[msaron]

@pkieltyka Yes, I believe that is the best approach.

[pkieltyka]

btw - I've pushed a new version here: https://github.com/pressly/chi/pull/173 and a complete example: https://github.com/pressly/chi/blob/render/_examples/rest/main.go

[pkieltyka]

completed + merged in https://github.com/pressly/chi/tree/v2.1.0