Flux.jl: REST API for real-time prediction

[brilhana]

It's unclear how to deploy a Flux model in a production environment for online prediction. It would be useful to be able to deploy a model as an HTTP endpoint exposing a REST API for real-time scoring.

TensorFlow and PyTorch both support HTTP and gRPC APIs for batch and real-time scoring.

[SaremS]

You can try to build one yourself with https://github.com/GenieFramework/Genie.jl or https://github.com/wookay/Bukdu.jl

[pshashk]

Or with https://github.com/JuliaWeb/HTTP.jl

function predict(req)
    io = IOBuffer(HTTP.payload(req))
    x = JSON3.read(io)
    y = model(x)
    HTTP.Response(200, JSON3.write(y))
end

router = HTTP.Router()
HTTP.@register(router, "POST", "predict", predict)
HTTP.serve(router, "0.0.0.0", port)