Hi! I'm trying to determine between Gin or Martini. I'm really impressed by Gin's speed and performance, and how easy the APIs are! However, in Martini, people can "clone" the mgo (mongodb driver for Go) session and pass it into the routing service, so I can assume only a minimal number of connection is being established. However it seems that with Gin I can't do this:
reqSession := session.Clone()
c.Map( reqSession.DB( "goattrs" ) )
Can someone tell me how or if there is a better way to achieve this (connection pooling) in Gin?
@windweller Gin does not use reflexion. but it has workaround for most of the use cases.
You could create a middleware:
func main() {
router := gin.Default()
// you could attach the mongo middleware to every requests by using router.Use(),
// but it is better to call the middleware only when we need it, right?
// router.Use(mapMongo)
// we create a group instead
mongo := router.Group("/", mapMongo)
mongo.POST("/post", func(c *gin.Context) {
//db := c.MustGet("mongo_session").(*mgo.Database)
db := getDB(c)
})
router.Run(":8080")
}
func mapMongo(c *gin.Context) {
// http://blog.mongodb.org/post/80579086742/running-mongodb-queries-concurrently-with-go
// Request a socket connection from the session to process our query.
// Close the session when the goroutine exits and put the connection back
// into the pool.
reqSession := session.Copy()
defer reqSession.Close()
c.Set("mongo_session", reqSession.DB("goattrs"))
c.Next() // we need to execute the rest handlers of the chain here because the session is released when we leave this scope
}
func getDB(c *gin.Context) *mgo.Database {
return c.MustGet("mongo_session").(*mgo.Database)
}
You code:
reqSession := session.Clone()
c.Map( reqSession.DB( "goattrs" ) )
is not correct, even for martini. Remember to close the session!!!
Check out the martini middleware:
func martiniMongo (c martini.Context ) {
reqSession := session.Clone()
defer reqSession.Close()
c.Map( reqSession.DB( "goattrs" ) )
c.Next()
}
func ginMongo(c *gin.Context) {
reqSession := session.Copy()
defer reqSession.Close()
c.Set("mongo_session", reqSession.DB("goattrs"))
c.Next()
}
it is almost the same!
Thank you so much!! This is very helpful :)
I actually restructured my code using this method, however after using defer reqSession.Close()
, it does not seem like the connection is being closed after the request is finished. This results in every new request opened a new connection to MongoDB, and this is not good.
Do I have to call c.Abort()
to activated the "deferred" session close?
@windweller no, in fact defer ensures that it will be called even if the request crashes.
Abort() only sets a flag... it is idempotent.
defer func() {
fmt.Println("calling defer")
reqSession.Close()
}()
let me know your findings.
maybe you should use: session.Clone()
instead of session.Copy()
. I am not sure...
Hi! Yes it seems like after adding the printing message, defer is being called successfully!
[GIN-debug] Listening and serving HTTP on :8080
calling defer
[GIN] 2015/06/04 - 10:08:30 | 202 | 4.443974ms | 127.0.0.1:65302 | POST /v1/register
calling defer
[GIN] 2015/06/04 - 10:08:39 | 202 | 1.294063ms | 127.0.0.1:65314 | POST /v1/register
However, on the mongodb backend, I see this:
2015-06-04T10:08:30.770-0400 I NETWORK [initandlisten] connection accepted from 127.0.0.1:65303 #2 (1 connection now open)
2015-06-04T10:08:39.500-0400 I NETWORK [initandlisten] connection accepted from 127.0.0.1:65315 #3 (2 connections now open)
yet this is a direct call without adding it to Gin's middleware:
2015-06-04T10:08:07.943-0400 I NETWORK [initandlisten] connection accepted from 127.0.0.1:65273 #1 (1 connection now open)
2015-06-04T10:08:07.978-0400 I ACCESS [conn1] Successfully authenticated as principal admin on nag
2015-06-04T10:08:07.979-0400 I NETWORK [conn1] end connection 127.0.0.1:65273 (0 connections now open)
This call's code is here:
func (db *DBConfig) Connect() *DBConfig {
connectionString := db.GetConnString()
session, err := mgo.Dial(connectionString)
defer session.Close()
return db
}
I'm trying to figure out what's going on! It makes me think maybe this is a Gin's middleware issue instead of mgo's issue?
@windweller who wrote the Connect() method?
it does not make any sense. you are closing the session before returning.
Oh that's just me testing if the defer session.Close()
would work or not, it's not in my actual code base, and it's not being used.
@windweller try Clone() instead of Copy()
there is no way it is a Gin bug
http://godoc.org/labix.org/v2/mgo#Session.Clone
Clone works just like Copy, but also reuses the same socket as the original session, in case it had already reserved one due to its consistency guarantees. This behavior ensures that writes performed in the old session are necessarily observed when using the new session, as long as it was a strong or monotonic session. That said, it also means that long operations may cause other goroutines using the original session to wait.
I am going to install mongodb to try to help you
Yeah, I changed it to .Clone() a while back. I don't think it's necessarily a Gin bug....sorry. I really like Gin though!!
Here is my code:
func DatabaseConnection() gin.HandlerFunc {
return func(c *gin.Context) {
//connect to MongoDB
sess, err := mgo.Dial(mongodb.Config.ConnString)
if err != nil {
c.JSON(http.StatusInternalServerError, util.NewErrors().AddDBDialError(err))
}
sess.SetSafe(&mgo.Safe{})
reqSession := sess.Clone()
// defer func() {
// fmt.Println("calling defer")
// reqSession.Close()
// }()
defer reqSession.Close()
c.Set("mongo_session", reqSession.DB("nag"))
c.Next()
}
}
Here is how it's being hooked up:
v1 := router.Group("/v1", DatabaseConnection())
@windweller don't worry! I am investigating it!
As I said before "calling defer" is being printed out successfully, but the connection did not close. Maybe it's a mgo
's problem, but a normal defer session.Close() works. And THANK YOU!
@windweller
I am using
func mapMongo(c *gin.Context) {
s := SESSION.Clone()
defer s.Close()
c.Set("mongo", s.DB("test"))
c.Next()
}
and it works perfect:
2015-06-04T22:20:44.235+0200 I NETWORK [initandlisten] connection accepted from 127.0.0.1:58118 #2 (1 connection now open)
when I remove c.Close() then tons of connections appear. In fact it works well with Copy and Clone
Yes, I got the same thing! But that connection is not closed though, right? If you send 5 requests, it will open 5 connections and none of them is closed (or maybe I'm understanding this wrong...)
but, why are you dialling in each request?
sess, err := mgo.Dial(mongodb.Config.ConnString)
if err != nil {
c.JSON(http.StatusInternalServerError, util.NewErrors().AddDBDialError(err))
}
??
http://godoc.org/labix.org/v2/mgo#Dial
This method is generally called just once for a given cluster. Further sessions to the same cluster are then established using the New or Copy methods on the obtained session. This will make them share the underlying cluster, and manage the pool of connections appropriately.
you are leaking the Dial session, you are creating two sessions each request, and closing one.
@windweller
the code I am using:
package main
import (
"fmt"
"github.com/gin-gonic/gin"
)
import "gopkg.in/mgo.v2"
var SESSION *mgo.Session
func init() {
session, err := mgo.Dial("localhost")
Must(err)
SESSION = session
}
func main() {
router := gin.Default()
router.Use(mapMongo)
router.GET("/", index)
router.Run(":8080")
}
func index(c *gin.Context) {
db := c.MustGet("mongo").(*mgo.Database)
db.C("table").Count()
fmt.Println(db)
c.String(200, "hello world!")
}
func mapMongo(c *gin.Context) {
s := SESSION.Clone()
defer s.Close()
c.Set("mongo", s.DB("test"))
c.Next()
}
func Must(err error) {
if err != nil {
panic(err.Error())
}
}
Most helpful comment
@windweller
the code I am using: