Would expect to be able to use arrow keys, tabs etc. to interact with bash
When trying tab completion, it either actually tabs the cursor. When trying for last command, ^[[A is put in
Just want to be able to interact with the shell like I normally would if I was native in linux
I'm unable to replicate this - It must be something with your Powershell setup. (win10 pro with Docker for Windows)

Please do share your findings once you find the settings that were changed.
As @FrenchBen has described in other issue reports, the bash shell is not opened when the Exec button is pressed. The sh shell is the default. Powershell does not interact properly with the sh shell so you should switch to bash. I am adding this text to the FAQ, which explains how to deal with this issue:
Kitematic uses sh for the Exec function. This can cause unwanted behavior, such as up arrows being displayed as '^[[A' in Powershell terminal. This is unavoidable, because not all docker containers have a bash shell (or csh, zsh, etc.). One way to use the bash shell, which does not have these limitations, is simply to type "bash" into the terminal each time that a new session is started.
If you want your bash shell to be the default shell for when you 'exec' into the container, then in Settings > Advanced, simply add an entry for SHELL with a value of /bin/bash. [1]
https://github.com/docker/kitematic/wiki/Common-Issues-and-Fixes
Most helpful comment
As @FrenchBen has described in other issue reports, the bash shell is not opened when the Exec button is pressed. The sh shell is the default. Powershell does not interact properly with the sh shell so you should switch to bash. I am adding this text to the FAQ, which explains how to deal with this issue:
https://github.com/docker/kitematic/wiki/Common-Issues-and-Fixes