Iris: Support Uint64 for MVC

[Dexus]

It would be nice if the MVC would also support the uint64. Unfortunately, data may be lost due to the int64.

[kataras]

I think there is no need to over-do with it, int64/int are OK, the end-dev can just make a check for negative values inside the body, it is just 1 line, they can even make custom macros to support uint64 validation and etc. But if you explain me a good reason that 100% iris should have it built'n I will see how we can do that without breaking changes for the next minor version.

[Dexus]

Hello and thank you for the answer.
Currently our API will reach over 9223372036854775807 (int64 maxvalue) in short IDs, because we work with uint64 IDs. Unfortunately, this only applies to customers who still work with our API v1. Since we don't currently have this problem with API v2, it wouldn't be a problem, but we have set up a longer runtime until shutdown for some customers.

The advantage is clear that you would save a lot of double conversions if it could be achieved directly with a single type add.

For the MVC controllers this would only be a small change and only needs some small changes like the long for in64 instead of int.

Thank you very much.

[Dexus]

In https://github.com/kataras/iris/blob/13945151e2326a18bb7f63260fdf8755f9b824b4/core/router/macro/interpreter/ast/ast.go it looks easy. But unfortunately we lack the deep view for possible places that would have to be considered as well.

[kataras]

OK, first of all the correct path to look is at: https://github.com/kataras/iris/blob/master/core/router/macro/macro.go#L223 , if you look the comment it says: // it could be uint64 but we keep int64 for simplicity, so it was already a debate.

Now, you need more than nine quintillion IDs? You already know that this is bad design of your API if that's the case but I understnad your backwards issue with your "v1".

The long and int types are already accept only positive numbers (you can use string or regexp like the cordinates example on the _examples/routing for negatives and use the ctx.Params().GetInt/XXX funcs). In reality the source path you commented below it is just for separating the int from int64 in the code itself, the rules for both of them are the same exactly, the only thing changes is _how you get the value of the parameter value via ctx.Params().GetXXX_ so we can convert the long -> int64 to long -> uint64 without breaking changes there: because, again, you get the number via ctx.Params().GetInt64 now and we can just add a GetUint64 and they will convert it if it is not the same type (i.e int vs int64, int64 vs uint64 and etc).

But you didn't ask for that, you asked for MVC, there I can add a check for the function's input parameter type and map it to the already existed long, I will think it a bit more and I will respond you with a commit, you will review, test it for your needs and tell me if that needs any more improvements, always without breaking changes to our existing web servers, ok?

EDIT: OK, we can do support both int64 and uint64 in the same param type long and you choose what you want to get by using the new ctx.Params().GetInt64 or GetUint64 respectfully). In the MVC the small code block that validates and maps the macro parameter type with the function's input argument is located at: https://github.com/kataras/iris/blob/master/core/router/macro/interpreter/ast/ast.go#L72-L107 and https://github.com/kataras/iris/blob/master/core/router/macro/interpreter/ast/ast.go#L155

[kataras]

At the other hand we can validate the input on int , int64 wich will accept the go range and negatives by using a different macro validator (regexp-based) but this will may lead to breaking changes for existing web servers and in the end the end-developer can add his/her own regexp / macro validator as iris do under the hoods. If we are going to do that we will do that on the major version 11 and not in a minor of v10.

My opinion: For the next minor update of v10, we can add the ctx.Params().GetUint64 which will solve ur issue (although you can do it by yourself using the v := ctx.Params().Get("myParam"); vUint64, _ := strconv.ParseUint(v, 10, 64) <- it will not lead to an error if you do it with {myParam:long} or {myParam:int} but you can use string as well) what I mean and what you will be able to do:

package main

import (
    "github.com/kataras/iris"
)

func main() {
    app := iris.New()
    app.Get("/int64/{id:long}", func(ctx iris.Context) {
        n, _ := ctx.Params().GetInt64("id")
        ctx.Writef("%d", n)
    })

    app.Get("/uint64/{id:long}", func(ctx iris.Context) {
        n, _ := ctx.Params().GetUint64("id") 
        ctx.Writef("%d", n)
    })

    app.Run(iris.Addr(":8080"))
}
/* * if you try to put more than max int64 on the /int64 path it will lead to MaxInt64 value but it will not lead to 404, this needs a breaking change that I want to do for v11 if we are all ok with it*/

and on the v11 I will make the necessary changes to do that check on the input parameter itself so it will lead to 404 if it is not a correct range of int, int64 or uint64 and etc. Does this sounds ok with you?

[kataras]

More info for your case, you can also use the built'n int-type macro func named range, i.e: /{paramName:int range(1,18446744073709551615) or max(18446744073709551615) and use the new (just-pushed) ctx.Params().GetUint64

[kataras]

I let you know, the community, that I am starting to code the v11, it will contain:

• the above adjustements to make the whole experience even more expected for parameter types, as it is our top-1 unique feature with their ability to be customized via our "macro" functions): I will introduce the uin64 new param type and will do param-route-level check for the existing number types as we have for the rest as well (i.e int, int64,uint64 will have their limits by-default and int will be able to accept negative numbers as well, currently it does not accept negative numbers), the bool and boolean will be the same, same for int / number, same for long/int64 for backwards-compatible at least for the names.
• and you will be able to set a parameter name with numbers as well, currently (v10) /{param2} is not valid but from v11 and after it will be a valid parameter name.
• The MVC will also be adjusted for the new param types, maybe uint8 and etc.

I think that will resolve any misunderstanding between param types and go standard types, although do not forget that using the ctx.Params.GetXXX you will still be able to resolve the parameter type -at the fly- without the route-level check, it is useful for dynamic types.

Also, new features will be introduce on v11.1.0, but I will announce them on the right time :)
Please post here any other ideas about number parameter types.

[kataras]

@Dexus Done on v11 branch, read the code so you can contribute on the future, you deserve it: https://github.com/kataras/iris/commit/648ce1336f96787383fb97b3c79df6f9b9334231

Also, you can use the GetUint64 which was pushed on current v10 as well to solve your issue with the current :int path parameter.

[Dexus]

Thank you @kataras! We are now using the v11 branch and so far everything works, in our first tests.

Thank you so much so far.

This will make our API work until the v1 API expires. That's really worth its weight in gold! I also find the other new changes very positive. This makes it very advantageous for the future APIs.

It certainly makes sense to support uint8, uint16, uint32 as well as all other "default" types. So that the existing ctx.Params().GetXXX functions also work perfectly in an MVC controller. So we here would definitely welcome that.

Thanks and greetings
Josef

[kataras]

You are very welcomed, it is still in-progress, i.e just now I added support for more macro functions types with https://github.com/kataras/iris/commit/aa317968e28e6286ec2bfc39e0fb9aaae320337a for int,int8,int32,int64,uint8,uint32,uint64, a slice of strings (as syntax of [value1,value2]) and a string (before that you were able to register a macro function with any type but the parser on /param:int min(1) for example the min's argument was only allowed to be string or int, now you can define a uint64 macro function as well wich can accept uint64 things) see here for example: https://github.com/kataras/iris/blob/aa317968e28e6286ec2bfc39e0fb9aaae320337a/_examples/routing/dynamic-path/main.go#L120
and https://github.com/kataras/iris/blob/aa317968e28e6286ec2bfc39e0fb9aaae320337a/_examples/routing/dynamic-path/main.go#L135

As you see the min(20) it is converted to uint64 if that expected, if we had func(minValue int)... it would be converted to int if possible, you got the point.

[kataras]

uint8 parameter type and mvc and hero added. I made it only for uint8 into one simple commit so you can look how it's working under the hoods, it is very easy, read: https://github.com/kataras/iris/commit/6da6c47e123279223bd45150b1db5adf9748503d

[kataras]

README.md section which gives more details to parameter types and "macros" added at v11 branch: https://github.com/kataras/iris/tree/v11#parameters-in-path