Activiti: Multi-schema multi-tenant not work

[minhquankq]

My issue: Activiti generated successful tables for all databases of each tenant which I configured. However, all my action (create model, deploy model, start process instance with tenant id) were recorded to 1 db which config for default tenant (empty string "").
Detail
Below is my implementation:
MultiTenantProcessEngineConfiguration

import javax.sql.DataSource;

import org.activiti.engine.ProcessEngine;
import org.activiti.engine.impl.cfg.multitenant.MultiSchemaMultiTenantProcessEngineConfiguration;
import org.activiti.engine.impl.cfg.multitenant.TenantInfoHolder;
import org.apache.commons.logging.Log;
import org.apache.commons.logging.LogFactory;
import org.springframework.jdbc.datasource.SimpleDriverDataSource;

import com.mysql.jdbc.Driver;

import sdk.activiti.model.VNGTenantInfoHolder;

public class VNGMultiTenantProcessEngineConfiguration extends MultiSchemaMultiTenantProcessEngineConfiguration {
    protected static final Log log = LogFactory.getLog(VNGMultiTenantProcessEngineConfiguration.class);
    public static final String DEFAULT_TENANT = "";
    public static final String VNG_TENANT = "vng";
    public static final String FPT_TENANT = "fpt";

    public VNGMultiTenantProcessEngineConfiguration() {
        super(getTenantInfoHolder());
    }

    public VNGMultiTenantProcessEngineConfiguration(TenantInfoHolder tenantInfoHolder) {
        super(tenantInfoHolder);
    }

    @Override
    public ProcessEngine buildProcessEngine() {
        this.setDatabaseType(MultiSchemaMultiTenantProcessEngineConfiguration.DATABASE_TYPE_MYSQL);
        this.setDatabaseSchemaUpdate(DB_SCHEMA_UPDATE_TRUE);

        // We must register datasource for null tenant, if not we will get can
        // not get datasource for tenant null.
        registerTenant(null,
                createDataSource(
                        "jdbc:mysql://10.30.47.242:3306/activiti6default?autoReconnect=true&failOverReadOnly=false&maxReconnects=5",
                        "u_activiti", "activiti"));
        registerTenant(DEFAULT_TENANT,
                createDataSource(
                        "jdbc:mysql://10.30.47.242:3306/activiti6default?autoReconnect=true&failOverReadOnly=false&maxReconnects=5",
                        "u_activiti", "activiti"));
        registerTenant(VNG_TENANT,
                createDataSource(
                        "jdbc:mysql://10.30.47.242:3306/activiti6vng?autoReconnect=true&failOverReadOnly=false&maxReconnects=5",
                        "u_activiti", "activiti"));
        registerTenant(FPT_TENANT,
                createDataSource(
                        "jdbc:mysql://10.30.47.242:3306/activiti6fpt?autoReconnect=true&failOverReadOnly=false&maxReconnects=5",
                        "u_activiti", "activiti"));

        return super.buildProcessEngine();
    }

    private DataSource createDataSource(String connectionStr, String user, String password) {
        try {
            return new SimpleDriverDataSource(new Driver(), connectionStr, user, password);
        } catch (SQLException e) {
            log.error(String.format("Can not create DataSource with connection string '%s'\nERROR MESSAGE: %s",
                    connectionStr, e.getMessage()));
            return null;
        }
    }

    private static TenantInfoHolder getTenantInfoHolder() {
        VNGTenantInfoHolder tenantInfoHolder = new VNGTenantInfoHolder();

        tenantInfoHolder.addTenant(DEFAULT_TENANT);
        tenantInfoHolder.addUser(DEFAULT_TENANT, "kermit");

        tenantInfoHolder.addTenant(VNG_TENANT);
        tenantInfoHolder.addUser(VNG_TENANT, "kermit");

        tenantInfoHolder.addTenant(FPT_TENANT);
        tenantInfoHolder.addUser(FPT_TENANT, "kermit");

        return tenantInfoHolder;
    }
}

The bean config

  <bean id="processEngineConfiguration" class="sdk.activiti.configuration.VNGMultiTenantProcessEngineConfiguration" />
  <bean id="processEngine" class="org.activiti.spring.ProcessEngineFactoryBean">
      <property name="processEngineConfiguration" ref="processEngineConfiguration" />
  </bean>
  <bean id="repositoryService" factory-bean="processEngine" factory-method="getRepositoryService" />
  <bean id="runtimeService" factory-bean="processEngine" factory-method="getRuntimeService" />
  <bean id="taskService" factory-bean="processEngine" factory-method="getTaskService" />
  <bean id="formService" factory-bean="processEngine" factory-method="getFormService" />
  <bean id="historyService" factory-bean="processEngine" factory-method="getHistoryService" />
  <bean id="managementService" factory-bean="processEngine" factory-method="getManagementService" />
  <bean id="identityService" factory-bean="processEngine" factory-method="getIdentityService" />

When I create model, deploy, and create instance for tenant 'vng' (using REST API), all data were stored in db activiti6default (my expectation is activiti6vng).

https://stackoverflow.com/questions/46948778/activiti-6-multi-schema-multi-tenant

[daisuke-yoshimoto]

@minhquankq If you want to separate data sources for each tenant, you need to separate the process engine itself for each tenant.

[minhquankq]

@daisuke-yoshimoto
according to this post - http://www.jorambarrez.be/blog/2015/10/06/multi-tenancy-separate-database-schemas-in-activiti/, I think activiti can support separate data sources for each tenant with one engine, that why the class MultiSchemaMultiTenantProcessEngineConfiguration has the function registerTenant(tenantId, datasource).
How about your ideas?

[daisuke-yoshimoto]

@minhquankq I'm sorry that I don't know that. I will check it.
By the way, in my project, I use a process engine separately for each tenant.

[minhquankq]

@daisuke-yoshimoto
Process engine separately for each tenant is good, but it take more resource than one process engine, I don't enough resource.

[daisuke-yoshimoto]

@minhquankq

I'm sorry that I made you wait. I examined MultiSchemaMultiTenantProcessEngineConfiguration.
The cause of not working properly is not a bug. Probably I think your usage is wrong.

You need to set the current tenantID for the running thread through the instance of TenantInfoHolder set in MultiSchemaMultiTenantProcessEngineConfiguration.After processing, it is necessary to clear the tenantID.

TenantInfoHolder.setCurrentTenantId(String tenantid)
 ---Your Business Logic ---
TenantInfoHolder.clearCurrentTenantId()

TenantAwareDataSource, which is the implementation class of the data source, selects the connection through the current tenant ID.

Reference

• https://github.com/Activiti/Activiti/blob/develop/activiti-engine/src/test/java/org/activiti/engine/test/cfg/multitenant/MultiTenantProcessEngineTest.java
• https://github.com/Activiti/Activiti/blob/develop/activiti-engine/src/test/java/org/activiti/engine/test/cfg/multitenant/DummyTenantInfoHolder.java

[minhquankq]

@daisuke-yoshimoto

Thanks for your responding, however, I'm using activiti-rest to communicate with activiti-engine, so, I only pass tenantId through the body data. Example, I want to start a process instance for tenant vng , I used API
URL: {{host}}/activiti-rest/service/runtime/process-instances
Method: POST
Body:

{
   "processDefinitionKey":"simple_model",
   "businessKey":  "{{business_key}}",
   "tenantId": "vng"
}

So, I can't use

TenantInfoHolder.setCurrentTenantId(tenantid)
/// ---Your Business Logic ---
TenantInfoHolder.clearCurrentTenantId()

Does activiti-rest support setCurrentTenantId base on request tenantId?

Note: I already implemented VNGTenantInfoHolder which like DummyTenantInfoHolder and used in VNGMultiTenantProcessEngineConfiguration as code example in my issue.

[daisuke-yoshimoto]

@minhquankq

Does activiti-rest support setCurrentTenantId base on request tenantId?

As far as the implementation is seen, I think that it does not support it.

Since activiti-rest uses Spring MVC, how about trying to plug in your own filter with the function of Spring?

[minhquankq]

@daisuke-yoshimoto

As far as the implementation is seen, I think that it does not support it.

I'm not sure we can handle by using spring's filter because there are many requests at the same time, but we only have 1 TenantInfoHolder, right?

[daisuke-yoshimoto]

@minhquankq
It must be created as a singleton.

1. You need to create one instance of TenantInfoHolder.class and set it to MultiSchemaMultiTenantProcessEngineConfiguration.

1. Your filter calls TenantInfoHolder.class#setCurrentTenantId for each request thread before request logic start.

2. Your filter calls TenantInfoHolder.class#clearCurrentTenantId after request logic end.

[daisuke-yoshimoto]

@minhquankq Any update?
Can I close this issue?

[egov-systems]

@daisuke-yoshimoto it's been set as Singleton only and tenant also getting set in the ThreadLocal but ThreadLocal tenant is not gettings used in DbSqlSessionFactory as I've mentioned #1032

[daisuke-yoshimoto]

@egov-systems

DbSqlSessionFactory.class knows schemas of each tenant through the TenantAwareDataSource.class which is the implementation of DataSource.
TenantAwareDataSource selects the connection through the current tenant ID.

[egov-systems]

@daisuke-yoshimoto this issue coming while generating activiti tables on startup not while doing process transitions. As i have mentioned in the other issue, we were using DbSqlSessionFactory only, since its not working as expected we have extended it as i've mentioned in that thread.

[daisuke-yoshimoto]

@egov-systems I'm sorry that I cannot understand your usecase. Please share your full code.

[egov-systems]

@daisuke-yoshimoto np, issue #1032 has the code as well but for your reference am adding pseudocode as well.

spring config process engine config bean

@Bean
    @DependsOn("tenants")
    MultiSchemaMultiTenantProcessEngineConfiguration processEngineConfig(EntityManagerFactory entityManagerFactory,
                                                                         TenantIdentityHolder tenantIdentityHolder,
                                                                         TenantawareDatasource tenantawareDatasource) {
        MultiSchemaMultiTenantProcessEngineConfiguration processEngineConfig = new MultiSchemaMultiTenantProcessEngineConfiguration(tenantIdentityHolder);
        processEngineConfig.setDataSource(tenantawareDatasource);
        processEngineConfig.setTransactionsExternallyManaged(true);
        processEngineConfig.setAsyncExecutorActivate(true);
        processEngineConfig.setAsyncExecutor(new AsyncExecuterPerTenant(tenantIdentityHolder));
        processEngineConfig.setJpaCloseEntityManager(false);
        processEngineConfig.setJpaHandleTransaction(false);
        processEngineConfig.setJpaEntityManagerFactory(entityManagerFactory);
        processEngineConfig.setDatabaseSchemaUpdate(DB_SCHEMA_UPDATE_TRUE);
        processEngineConfig.setDatabaseType(DATABASE_TYPE_POSTGRES);
        processEngineConfig.setHistory(HistoryLevel.FULL.getKey());
        processEngineConfig.setDbSqlSessionFactory(new DBSqlSessionFactory());
        processEngineConfig.setTablePrefixIsSchema(true);
        processEngineConfig.setExpressionManager(new SpringExpressionManager(applicationContext, null));
        processEngineConfig.setConfigurators(Arrays.asList(processAuthConfigurator));
        tenantIdentityHolder.getAllTenants().stream().filter(Objects::nonNull).forEach(tenant ->
                processEngineConfig.registerTenant(tenant, tenantawareDatasource)
        );

        return processEngineConfig;
    }

spring process engine bean

@Bean
    ProcessEngine processEngine(MultiSchemaMultiTenantProcessEngineConfiguration processEngineConfiguration,
                                TenantIdentityHolder tenantIdentityHolder) throws IOException {
        ProcessEngine processEngine = processEngineConfiguration.buildProcessEngine();
        List<Resource> commonBpmnResources =
                resourceFinderUtil.getResources("classpath:processes/common/*.bpmn20.xml");

        for (String tenant : tenantIdentityHolder.getAllTenants()) {
            tenantIdentityHolder.setCurrentTenantId(tenant);

            List<Resource> resources = commonBpmnResources.stream().
                    filter(rsrc -> !resourceNames.contains(rsrc.getFilename())).
                    collect(Collectors.toList());
            for (Resource resource : resources) {
                processEngine.getRepositoryService().createDeployment().
                        enableDuplicateFiltering().name(resource.getFilename()).
                        addInputStream(resource.getFilename(), resource.getInputStream()).deploy();
            }
            tenantIdentityHolder.clearCurrentTenantId();
        }
        return processEngine;
    }

With the above config activiti is creating activiti specific tables in default schema (by default in public schema), irrespective how many schemas we have.

We required activiti to create activiti specific table in each and every schema we provided through ThreadLocal (TenantInfoHolder).

[minhquankq]

@daisuke-yoshimoto
sorry for my slow responding,
Because MultiSchemaMultiTenantProcessEngineConfiguration is singleton and has one TenantInfoHolder instance, so, we can not use setCurrentTenantId and clearCurrenttenantId in filter of Spring because conflict between of users at the same time.

[salaboy]

@minhquankq have you looked at our Activiti Cloud work? It might be better to look into that than fighting with the multi-tentant approach provided by old versions. We will appreciate contributions to fix the multi-tenant approach, but new technologies exist to deal with multi-tenancy in a better way. Please re-open this issue if you want to collaborate with us or get in touch via gitter: https://gitter.im/Activiti/Activiti7?utm_source=share-link&utm_medium=link&utm_campaign=share-link

[sreekanthsnair]

@Salaboy i don't know what you guys are thinking about, the moment we file an issue in github we means the respective application is not working as expected, so I hope by filing an issue we are also contributing to activiti. So i request to consider existing user issues as well instead of ruling out issues filed.

[salaboy]

@sreekanthsnair we all have hope in this project that is why we are reviewing all issues and see how they align with our current roadmap. Issues are extremely important, and because of that we need to make sure that they fit in an overall plan and that we can allocate someone to work. Having abandoned issues doesn't help anyone either. You can't imagine the amount abandoned issues that we are trying to triage. For that reason I kindly ask you to review our latest work and reopen the issue if you still believe that the multi tenant option that you are trying to fix is the adequate one to solve your business problem.

[sreekanthsnair]

@Salaboy i do agree with your point in all respect, but i would like you too see issues filed by community in a different angle, it may not be solvable with your current resource or roadmap but by closing this issue doesn't make any different from keeping it open.

[salaboy]

@sreekanthsnair I’m re opening the issue becuase I want to see a solution that works for you and the overall community. But opened issues that are not being worked on needs to be triaged and assigned to a milestone, if not the project goes into caos. Im looking forward to plan and work the best solution for this issue with you and all the interested parties.

[paksboda]

@salaboy @daisuke-yoshimoto
Do you have any idea if this issue is fixed or not in last release i.e. v5.23.0 (Aug-2019). I am facing simillar issue but not sure if it is because of this or some configuration problem from my side. Please update status if you have any update.

[salaboy]

@paksboda I do not work on the project anymore .. so don't expect any status update from my side.. also check the activity of the 5.x branch.. you might get an idea if things are progressing there or not. HTH

[paksboda]

@daisuke-yoshimoto
I tried latest release i.e. v5.23.0 (Aug-2019) of Activiti 5.x and it works perfectly fine with multiple tenants if you have proper implementation of TenantAwareDataSource and TenantInfoHolder. It is creating activiti schema on different tenant schema and I am using single ProcessEngine with MultiSchemaMultiTenantProcessEngineConfiguration. I tried it on oracle and it works fine. You can close this issue.

[alexpdh]

@daisuke-yoshimoto
I tried latest release i.e. v5.23.0 (Aug-2019) of Activiti 5.x and it works perfectly fine with multiple tenants if you have proper implementation of TenantAwareDataSource and TenantInfoHolder. It is creating activiti schema on different tenant schema and I am using single ProcessEngine with MultiSchemaMultiTenantProcessEngineConfiguration. I tried it on oracle and it works fine. You can close this issue.

Please share your full code.

[cirux]

A TenantInfoHolder working example would be very interesting

[paksboda]

/**

• 
  */

import java.util.ArrayList;
import java.util.Collection;
import java.util.List;

import org.activiti.engine.impl.cfg.multitenant.TenantInfoHolder;

/**

• This is custom tenant info holder class. Most of the cases this
• class will fetch values from {@code UserContextHolder} which is set by
• filter.
  */
  public class CustomTenantInfoHolder
  implements
  TenantInfoHolder {

private static List tenantList = new ArrayList<>();
protected ThreadLocal currentTenantId = new ThreadLocal();

public CustomTenantInfoHolder() {
super();
}

/* (non-Javadoc)

• @see org.activiti.engine.impl.cfg.multitenant.TenantInfoHolder#getAllTenants()
  */
  @Override
  public Collection getAllTenants() {
  return getTenantList();
  }

/* (non-Javadoc)

• @see org.activiti.engine.impl.cfg.multitenant.TenantInfoHolder#setCurrentTenantId(java.lang.String)
  */
  @Override
  public void setCurrentTenantId( String tenantid ) {
  currentTenantId.set( tenantid );
  }

/* (non-Javadoc)

• @see org.activiti.engine.impl.cfg.multitenant.TenantInfoHolder#getCurrentTenantId()
  */
  @Override
  public String getCurrentTenantId() {
  return currentTenantId.get();
  }

/* (non-Javadoc)

• @see org.activiti.engine.impl.cfg.multitenant.TenantInfoHolder#clearCurrentTenantId()
  */
  @Override
  public void clearCurrentTenantId() {
  currentTenantId.set( null );
  }

public static List getTenantList() {
return tenantList;
}

public static void setTenantList( List tenantList ) {
CustomTenantInfoHolder.tenantList = tenantList;
}
}

/**

• 
  */

import java.sql.Connection;
import java.sql.SQLException;
import java.sql.Statement;

import javax.sql.DataSource;

import org.activiti.engine.impl.cfg.multitenant.TenantAwareDataSource;
import org.activiti.engine.impl.cfg.multitenant.TenantInfoHolder;

import com.company.product.commons.exception.DataAccessException;
import com.company.product.commons.exception.ExceptionType;

/**

• This is product specific multi tenant implementation of data source interface.
• Activiti process engine will use this data source to process individual
• tenant request.
  */
  public class productTenantAwareDataSource
  extends
  TenantAwareDataSource {

private DataSource dataSource;
private static final String ALTER_SESSION_SET_CURRENT_SCHEMA = "ALTER SESSION SET CURRENT_SCHEMA = ";

public productTenantAwareDataSource( TenantInfoHolder tenantInfoHolder, DataSource dataSource ) {
super( tenantInfoHolder );
this.dataSource = dataSource;
}

public void init() {
for( String tenantSchema : tenantInfoHolder.getAllTenants() ) {
addDataSource( tenantSchema, dataSource );
}
}

@Override
public Connection getConnection()
throws SQLException {
return getTenantAwareConnection();
}

@Override
public Connection getConnection( String username, String password )
throws SQLException {
throw new UnsupportedOperationException(
"This connection should have access to all tenant schema. This method is not supported." );
}

private Connection getTenantAwareConnection()
throws SQLException {
DataSource dataSource = getCurrentDataSource();
final Connection connection = dataSource.getConnection();
String tenantIdentifier = tenantInfoHolder.getCurrentTenantId();
if ( tenantIdentifier != null ) {
try (Statement stmt = connection.createStatement()) {
stmt.execute( ALTER_SESSION_SET_CURRENT_SCHEMA + tenantIdentifier );
} catch( SQLException e ) {
try {
connection.close();
} catch( SQLException se ) {
}
throw new DataAccessException(
ExceptionType.DATA_ACCESS_EXCEPTION,
"Could not alter JDBC connection to specified schema [" + tenantIdentifier + "]", e );
}
} else {
throw new DataAccessException(
ExceptionType.DATA_ACCESS_EXCEPTION,
"No tenant schema found while getting connection. Please set tenantSchema in tenantInfoHolder first." );
}
return connection;
}
}
/////////////////////////////
Spring Configuration
////////////////////////////

```

```

[paksboda]

@daisuke-yoshimoto
I tried latest release i.e. v5.23.0 (Aug-2019) of Activiti 5.x and it works perfectly fine with multiple tenants if you have proper implementation of TenantAwareDataSource and TenantInfoHolder. It is creating activiti schema on different tenant schema and I am using single ProcessEngine with MultiSchemaMultiTenantProcessEngineConfiguration. I tried it on oracle and it works fine. You can close this issue.

Please share your full code.

I have added sample code in case anyone wants to refer.