Ipfs-companion: Companion should auto resolve ENS names to IPFS

[momack2]

People are using ENS names as a way to find content on IPFS (ex https://twitter.com/eduadiez/status/1093547099235516416). Let's make it even easier by enabling companion to resolve links like ipfs://foo.eth to the appropriate IPFS hash + content!

Would it potentially also be possible to check for any .eth links to see if they are hosted on IPFS and if so resolve through IPFS?

[lidel]

_(sorry for any mistakes, went on a research spree and wanted to post this while I have entire context in my head)_

tl;dr

I did an initial run over prior art for ENS support and identified some techiques and challenges. Notes below. Overall, it should be possible to do, but we may need to land some changes in go-ipfs or public gateway.

Unexpectedly, this research gave me an idea how to solve Origin problem for localhost gateway(!) – see https://github.com/ipfs/go-ipfs/issues/5982 ← i am really excited about this one!

(1) ENS lookup without running Ethereum client

The main challenge will be resolving ENS to IPFS CID from Ethereum transaction without running Ethereum client (and avoid shipping half of MetaMask or additional binary just to resolve ENS domains)

Most of the prior art ships ethereum client in some form:

• MetaMask is able to support ENS because by definition it has access to Ethereum blockchain
• (cpacia/ens-chrome-extension) requires cpacia/ens-lite (thin ens resolver binary) to run in addition to extension.

Potential solutions:

• (1A) Lookups via some HTTP API

  • One way to do it is to do lookups via remote HTTP API (similar to DNSLink detection with embedded js-ipfs). I did some research and it seems ethjs-ens supports doing lookups via third-party HTTP resolver (ethjs-provider-http). We could use well-known public one (eg. infura), or run our own endpoint.
  • a neat way to do this would be to reuse existing /api/v0/dns endpoint in go-ipfs (used for DNSLink lookups) and make it support .eth domains. This way we would have a reliable resolver at both localhost go-ipfs and public gateway for use by js-ipfs in browser context.
  • digression: to avoid shipping ethereum client with go-ipfs we could just hardcode 'trusted' endpoint for .eth lookups, but it smells a bit, if we go this route we probably should keep HTTP API lookup in ipfs-companion as sugested in (2B)

• (1B) Delegate act of ENS resolution to remote resolver, then redirect to ENS-aware HTTP proxy or gateway

  • does not add any dependencies to ipfs-companion
  • see (2) below
• (1C) ?

(2) Support foo.eth domains in browser

Prior art: ens-gateway-eth-domain extension seems to do interesting hacks to solve UX challenges:

• registers a proxy for *.eth and points at unencrypted HTTP port [sic!] at some IP owned by some company (159.203.100.160:80)
  
  
  
  • if domain that was not used recently(?), remote server returns HTTP 404 for requested Host (screenshot)
  
  
  • extension detects 404 for *.eth and replaces the tab with HTML-based handler shipped with extension itself, in the background ENS resolution happens and user is then presented with blue button "Visit website" (screenshot)
  
  
• foo.eth loaded from remote proxy works, hostname is in location bar, but transport layer is unencrypted (screenshot)

Known Limitations:

• .eth is not a valid/known tld, so foo.eth is not supported in location bar, needs to be http://foo.eth
• remote proxy is unable to provide certificate for a domain with bogus TLD, so transport is unencrypted HTTP (not an issue is HTTP proxy runs on localhost)

Potential solutions (given lessons learned from above prior art):

• (Solution 2A) redirect *.eth to go-ipfs being ENS-aware HTTP proxy
  
  
  
  • this could be HTTP port of go-ipfs running on localhost or separate, simple service at some remote IP provided by PL (used as a fallback when companion runs with embedded js-ipfs)
  
  
  • (preferaly localhost port, to avoid unencrypted HTTP over public internet)
  
  
  • "ENS-aware HTTP proxy" would work like this:
  
  
  • read Host HTTP Header
    
    
    
    
    
    • if ENS is in cache, return immediately
    
    
    
    • if ENS is not in cache, return some 'wait' code (eg. 305 Use Proxy) and perform ENS loookup (similar to existing DNS lookup for /api/v0/dns) in the background, as suggested in (1A)
    
    
    
    • ipfs-companion detects 'wait' HTTP code and displays loading screen + retries in background via HEAD request until gets response other than 'wait'
    
    
    
  
  
  • if ENS points at IPFS, return HTTP 200 with payload like a regular gateway
  
  
  • if ENS points at non-IPFS content, return some other code + (TBD)
  
  
  • Alternatively, ENS-aware HTTP proxy would just block until ENS is resolved :upside_down_face:
  
  
• (Solution 2B) do lookup in companion and redirect *.eth to go-ipfs HTTP proxy
  
  
  
  • similar to 2A, but go-ipfs does not need to know anything about ENS, we just redirect to <foo>.ipfs.localhost or <bar>.ipns.localhost
  
  

Benefits of running "resolver as HTTP proxy"

Just to reiterate why this approach is exciting:

• we do not add any additional pieces to existing stack (go-ipfs+ipfs-companion or public gateway + ipfs-companion)
• simple setup - just set HTTP proxy - and we are able to add support for ENS and other naming systems (Namecoin's *.bit ?) as we go
• cool side-effect if go-ipfs acts as HTTP proxy: we are able to support <cidv1b32>.ipfs.localhost solving Origin problem for local gateway at go-ipfs
  
  
  
  • I created dedicated issue for this at: https://github.com/ipfs/go-ipfs/issues/5982
  
  

Related Reading

• EIP 1577: contenthash field for ENS https://eips.ethereum.org/EIPS/eip-1577
• Use the chrome.proxy API to manage Chrome's proxy settings: https://developer.chrome.com/extensions/proxy
• Why we need subdomains: Security concern - IPFS ENS Resolving https://github.com/MetaMask/metamask-extension/issues/5724
• Prior Art (extensions with ENS support)
  
  
  
  • https://github.com/MetaMask/metamask-extension
  
  
  • https://github.com/cpacia/ens-chrome-extension
  
  
  • https://chrome.google.com/webstore/detail/ens-gateway-eth-domain-br/jkaiofboahfpipgijdgdmbdldlgcipgo
  
  
  • https://chrome.google.com/webstore/detail/peername/kkdihlopcnkjinfjhbeopjfmnfpcoaop?hl=en (custom tlds)
  
  
• CC
  
  
  
  • Past request for namecoin support https://github.com/ipfs-shipyard/ipfs-companion/issues/577
  
  
  • Origin problem for local gateway at go-ipfs (https://github.com/ipfs-shipyard/ipfs-companion/issues/667 and https://github.com/ipfs/in-web-browsers/issues/109)
  
  

[parkan]

FWIW I am unable to load the ENS resolvers (other than manually looking it up in the ENS manager):

DOES NOT work

• https://oasis.dappnode.meeseeks.app/ -- dies w/SSL error because it only supports one subdomain deep
• http://oasis.dappnode.eth.show/ -- tries to resolve the content as a swarm address, fails
• metamask -- "metamask error: not found"

works

https://manager.ens.domains/name/oasis.dappnode.eth (needs to be set to mainnet), click through to https://gateway.ipfs.io/ipfs/QmXXnnme2tC1JVxvHMM52xwy1gDfZnymrA1soTwiMTLqRK

[parkan]

I still need to process @lidel's post fully (a lot of info there!) but a question that came to mind regarding the "bogus" nature of the domain is: how does .local and go work? I am fairly sure I've used internal network sites at past jobs that had SSL certs -- were those just forcibly added to the machine?

[lidel]

@parkan In short, domain resolution is effectively delegated to HTTP Proxy. If we make go-ipfs work as an HTTP Proxy it will be able to create responses for arbitrary domains on the fly. (I wrote some initial notes in https://github.com/ipfs/go-ipfs/issues/5982)

As for SSL, yes, it is possible to have HTTPS for a fake intranet domain. In bigger orgs machines handed out to employees often come with additional CA Root cert in system keystore, or some VPN software injects it. That CA is responsible for validation of certs for fake domains (and make employer able to look into HTTPS traffic).

[lidel]

Quick brain dump about "light/delegated ENS resolver" (a way to deliver 1B)

Note: This comment was extracted to https://github.com/protocol/collab-ens/issues/1

• Goal: enable ENS lookups in contexts where running ethereum client is not feasible
• How: do not invent things, as a part of migration path we should reuse DNS libraries and formats, especially DNS-over-HTTPS
• What: DNS server responding to queries with domains within .eth TLD
  
  
  
  • returns A record with IP of HTTP gateway that gives access to ENS websites to people without Ethereum or IPNS clients:
    
    e.g. <domain-name>.eth.link
  
  
  • returns TXT record with the value of ENS record
  
  
  • if ENS points at IPFS, it would look like regular DNSLink, reusing all the code that already supports it
  
  
• Does not need to be binary, MVP could just be DNS-over-HTTPS with JSON API (to enable use in ipfs-companion)
  
  
  
  • prior art exists:
  
  
  • https://developers.google.com/speed/public-dns/docs/dns-over-https
  
  
  • https://developers.cloudflare.com/1.1.1.1/dns-over-https/json-format/
  
  

Value added by delegated approach:

• Enables ENS support in legacy software that only speaks DNS and HTTP.
• IPFS Companion could validate ENS names via ENS-over-HTTP JSON API.
• GO-IPFS could simply send DNSLink lookups to a different DNS server when /ipns/<domain-name>.eth is requested (keeping changes to go-ipfs extremely minimal)
• Browser vendors could add support for ENS by simply swapping DNS-over-HTTPS server for .eth TLD, or adding support to their own servers
  
  
  
  • Fun fact: Firefox is using https://mozilla.cloudflare-dns.com/dns-query for DNS-over-HTTPS and the URL can be changed via about:config
  
  

[parkan]

would be good to mirror/summarize this in https://github.com/protocol/collab-ens/

[parkan]

it may also be worthwhile to step back a moment and consider how the DNS zone hierarchy is structured and to what extent does the ENS system follow or diverge from that: for example, if we consider ENS as simply the authoritative nameserver for the .eth zone (enacted via dns-over-https), could there be an even more elegant way to incorporate it?

although, perhaps fudging TXT records with a prefix is not _quite_ elegant per se

[lidel]

@parkan I've extracted "DNS" and "HTTP Proxy" to issues there. LEt me know if something else should be extracted.

[..] if we consider ENS as simply the authoritative nameserver for the .eth zone (enacted via dns-over-https), could there be an even more elegant way to incorporate it?

I believe that if DNS-compatible ENS resolver exists and follows DNS RFCs, plugging it into mainstream DNS as the authoritative nameserver for *.eth could happen, but requires some entity to pay ICANN for that TLD (https://newgtlds.icann.org/en/)

[parkan]

thank you, that will help direct the work across the various participants!

the TLD is... $200k or so? that's approximately one Lambo, maybe someone in the Ethereum community can swing that 😆