Incubator-superset: Automatically create users from csv file

[pajachiet]

Goal

We need to automate the creation and maintenance of Superset users. This is useful to setup rapidly new Superset instances, or for tests.

Personal hack

Based on a csv files listing users and their password, we developed a script that create missing users and set their passwords. The script that we use currently is called
create_superset_users.py in this gist. It works with a dockerized Superset instance (easy modification to adapt otherwise).

One csv list users, with the following header _first_name;last_name;email_
Another list their passwords, with the following header _email;password_
Other columns may be added if needed. The same file could be used if you don't need to separate informations.

Feature request

Would it be possible to develop a similar feature directly into Superset ?
A complete feature would be to export the current list of users as a csv, modify it if needed, then import it again.

[ratb3rt]

FWIW, a) you shouldn't directly work with the ab_user table - use object model instead. and b) the user management is handled by Flask-AppBuilder - that might be a better place to implement a bulk user loader

[robert-digit]

It sounds like we had similar needs. Note that you can extend the
SecurityManager class and call self.add_user inside a superset_config.py.

It would look a bit like this (not tested, just typing out the general idea
for what we use) :

CustomSecurityManager(SecurityManager):
def my_bulk_insert_method(self):
for newuser in users_to_add:
# Pull your username, etc. from your csv
user = self.add_user(
username=username,
first_name=user_firstname,
last_name=user_lastname,
email=user_email,
role=user_role
)

CUSTOM_SECURITY_MANAGER = CustomSecurityManager

On Thu, Jul 6, 2017 at 1:59 PM, Rich @ RadICS [e-mail]
wrote:

FWIW, a) you shouldn't directly work with the ab_user table - use object
model instead. and b) the user management is handled by Flask-AppBuilder -
that might be a better place to implement a bulk user loader

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/apache/incubator-superset/issues/3083#issuecomment-313388784,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ATMSk8y8XWfNVGlnRl8iLcx-6zSyg3JJks5sLNotgaJpZM4OOPFz
.

[robert-digit]

Considering my note above can this be marked resolved?

[pajachiet]

Thanks a lot for your answers. This looks much better indeed ! And its great to put that in superset_config.py

Where do you import the class SecurityManager ? Is it from flask_appbuilder.security.manager import BaseSecurityManager ?

[JulieRossi]

Hi !

I am working with pajachiet on this subject.

This method looks quite nice indeed but, if I am right, it implies calling the new my_bulk_insert_method at some point, inside superset code (we use superset inside a container, not as a module).

We have decided to try using superset API, to be more specific, we use superset/update_role/ route.
This seems to fit our needs. If someone has any feedback regarding this method, we are really interested.

[mistercrunch]

Notice: this issue has been closed because it has been inactive for 240 days. Feel free to comment and request for this issue to be reopened.

[yqylovy]

@JulieRossi Hi ！
I have a similar demand to create user automatically. Using superset API seems to be a better idea，but I don't know where to start .
Would you mind sharing some detail on how to do it ? It would be a great help,thanks.

[JulieRossi]

Hi @yqylovy, if it helps, here's how our code looks like :
Tool class to be able to use superset api (needs to be connected for some actions)

class UseSupersetApi:
    def __init__(self, username=None, password=None):
        self.s = requests.Session()
        self.base_url = SUPERSET_URL
        self._csrf = get_csrf_token(self.s.get(self.url('login/')))
        self.headers = {'X-CSRFToken': self._csrf, 'Referer': self.url('login/')}
        # note: does not use headers because of flask_wtf.csrf.validate_csrf
        # if data is dict it is used as form and ends up empty but flask_wtf checks if data ...
        self.s.post(self.url('login/'),
                    data={'username': username, 'password': password, 'csrf_token': self._csrf})

    def url(self, url_path):
        return self.base_url + url_path

    def get(self, url_path):
        return self.s.get(self.url(url_path), headers=self.headers)

    def post(self, url_path, data=None, json_data=None, **kwargs):
        kwargs.update({'url': self.url(url_path), 'headers': self.headers})
        if data:
            data['csrf_token'] = self._csrf
            kwargs['data'] = data
        if json_data:
            kwargs['json'] = json_data
        return self.s.post(**kwargs)

And then we use it to create users :

superset = UseSupersetApi('admin', SUPERSET_ADMIN_MDP)

for user in users:
    payload = {'first_name': user['first_name'],
               'last_name': user['last_name'],
               'username': user['username'],
               'email': user['email'],
               'active': True,
               'conf_password': user['password'],
               'password': user['password']}
    superset.post(url_path='users/api/create', json=payload)

users_by_role = _group_users_by_role(users)
for role in users_by_role:
    payload = {'users': users_by_role[role], 'role_name': role}
    superset.post(url_path='superset/update_role/', json=payload)

Don't hesitate if you have any questions

[yqylovy]

@JulieRossi Thanks !!! I tried to login but aways fail , by viewing your code，I found that I missing the csrf_token param and finally successfully login.
Thank you again!

[SWoto]

For those who may need the code to do it. It works outside the host server.

import requests
from bs4 import BeautifulSoup as bs
from bs4 import Comment

class UseSupersetApi:
    def __init__(self, username=None, password=None):
        self.s = requests.Session()
        self.base_url = "http://123.45.67.890:8088/"
        self._csrf = self._getCSRF(self.url('login/'))
        self.headers = {'X-CSRFToken': self._csrf, 'Referer': self.url('login/')}
        # note: does not use headers because of flask_wtf.csrf.validate_csrf
        # if data is dict it is used as form and ends up empty but flask_wtf checks if data ...
        self.s.post(self.url('login/'),
        data={'username': username, 'password': password, 'csrf_token': self._csrf})

    def url(self, url_path):
        return self.base_url + url_path

    def get(self, url_path):
        return self.s.get(self.url(url_path), headers=self.headers)

    def post(self, url_path, data=None, json_data=None, **kwargs):
        kwargs.update({'url': self.url(url_path), 'headers': self.headers})
        if data:
            data['csrf_token'] = self._csrf
            kwargs['data'] = data
        if json_data:
            kwargs['json'] = json_data
        return self.s.post(**kwargs)

    def _getCSRF(self, url_path):
        response = self.s.get(base_url)
        soup = bs(response.content, "html.parser")
        for tag in soup.find_all('input', id='csrf_token'):
            csrf_token = tag['value']
        return csrf_token

superset = UseSupersetApi('admin', 'passoword')
users = [] #some user dicts inside

for user in users:
    payload = {'first_name': user['first_name'],
               'last_name': user['last_name'],
               'username': user['username'],
               'email': user['email'],
               'active': True,
               'conf_password': user['password'],
               'password': user['password'],
               user['roles']} 
    print(superset.post(url_path='users/api/create', json=payload))

[abhishekkumaresan]

@SWoto thanks for sharing the source, I am able to login successfully but the /users/api/create results in 404,
users/add is working for me