Dbt: Reinstatement of of ssh tunnel

[chrisburrell]

Hi

I'm looking at : https://github.com/fishtown-analytics/dbt/issues/93 which suggests that once upon a time ssh-host would have been a valid property under a target of profiles.yml where a user can set up an ssh tunnel to connect to the data source.

I would love to be able to use this feature again.

Some context

In our case, the scenario is that our data is brought in by stitch in many schemas. I'd like to give local access to a data engineer to some of that data, but that redshift is secured using a data bastion sitting in front.

Some issues we're trying to resolve

In our case, we can develop a lot of the scripts locally against postgres, however there are limitations:

• Speed is not greatly representative locally (anonymised production data / cut down data sets, etc.)
• Redhisft & Postgres differ in function names - we use jinja to abstract that away in the model files, but we can't test portions of jinja until we're in Redshift itself and using production data.

What we would like to do

• Allow a local user to connect to Redshift through our data bastion. They have access to all the source models) but can only right to a 'user' schema (say 'chris').
• They can then test the actual DBT scripts per-se.

[drewbanin]

Thanks @chrisburrell - I'll have a look at why we removed this in the first place. Hopefully our next iteration of it can avoid whatever pitfalls we encountered last time around :)

[chrisburrell]

@drewbanin - Thanks for the easy workaround
ssh -l $BASTION_USER $BASTION_HOST -p 22 -N -C -L "5439:${REDSHIFT_HOST}:5439";

Then use bash.rc or any other means to run it quickly/constantly

[drewbanin]

I don't think this is something we're going to want to add to dbt. It's easy enough to do on a user's machine, and the complexity (running a separate background thread/process) makes it not worth it. There are lots of good open source ssh tunnel tools out there that do a good job of this - closing this.