Postgrest: Binary Endpoints

[pkel]

I tried to setup an endpoint for providing binary data. My main motivation is to serve images for <img> tags without javascript. I documented my progress in a PR on the documentation repo.

There is one remaining issue. The raw-media-types option is not powerful enough to provide a transparent binary endpoint that behaves like a webserver serving files. Firefox sends

Accept: image/webp,*/*

on <img> requests and

Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8

on navigation requests (e.g. user does right click view image).

I tried the following settings:

• */*. Navigation requests for regular json endpoints fail with */* requested but more than one column was selected.
• image/webp. Same problem.
• image/*. Does not work for <img> requests.

From my unprivileged point of view, I see two ways forward:

1. A raw-media-endpoint configuration option that triggers raw output for the specified tables, views, and procedures.
2. A local parameter response.raw that allows to trigger raw output for a single request with something like perform set_config('response.raw', 'true', true);.

The second variant can emulate the first via pre-request. The first would suffice for my use case.

[pkel]

I found a nice workaround. For serving images, we need a caching reverse proxy anyway, so we could use it for setting the Accept header to application/octet-stream on the binary endpoint only. Consider this nginx location:

  location /files/ {
    rewrite /files/([^/]+).*  /rpc/file?id=$1  break;
    return 404;
    proxy_set_header Accept application/octet-stream;
    ...
  }

This has the additional benefit, that we can share URLs of the form /files/42/cats.jpg. This looks a bit more natural than /rpc/file?id=42.

This approach does not use the raw-media-types option at all.

[steve-chavez]

@pkel Nice nginx snippet. I think that would be good for the how-to in https://github.com/PostgREST/postgrest-docs/pull/307.

I see what you mean about raw-media-types, right now it makes all endpoints accept the specified media types(making the default responses fail) when actually you only wanted to apply it for a single endpoint.

A local parameter response.raw that allows to trigger raw output for a single request with something like perform set_config('response.raw', 'true', true);

Ideally, we would detect the Content-Type header was overridden with set_config( 'response.header.content-type', ..) but this has some downsides implementation-wise. We would have to run two queries instead of one for a single response(and maybe other stuff I'm not seeing now).

A raw-media-endpoint configuration option that triggers raw output for the specified tables, views, and procedures.

This option would be simpler to implement, seems configurator-pg would allow us to specify something like this in the config:

db-uri = ...

raw-media-endpoints {
  images = "image/png, image/webp"
  files   = "image/jpeg,.."
  rpc/files = "image/apng" ## not sure if slashes are accepted
  rpc_files = "image/apng" ## otherwise maybe underscore could be used
}

The config option name could take another name, but that would be the main idea.

[steve-chavez]

@Dansvidania You did some work on raw-media-types before. Would you be interested in taking this one?(no rush since there's a workaround)

[Dansvidania]

Thanks for bringing it to my attention. I'm going to check what I can do.

[Dansvidania]

@steve-chavez What is the expected relation between raw-media-types and raw-media-endpoints ?

I imagine -types could stay for retro-compatibility and be applied as a default for any endpoint that does not have a definition in -endpoints. Or, since the -types is a special case of -endpoints the two config options could be mutually exclusive?

for example

raw-media-endpoints {
  images = "image/png, image/webp"
  files   = "image/jpeg,.."
  [...]
  */otherwise/default/_ = "video/*, audio/*"
}

instead of

raw-media-endpoints {
  images = "image/png, image/webp"
  files   = "image/jpeg,.."
  [...]
}

raw-media-types = "video/*, audio/*"

[steve-chavez]

@Dansvidania I think we should aim at replacing raw-media-types with raw-media-endpoints. For the reason mentioned here(makes default responses fail). So mutually exclusive it is.

Also, we should deprecate raw-media-types after raw-media-endpoints is implemented.