Docker-images: Osb Domain Creation fails because of connection problems

[tstelegithub]

I'm using Oracle 12.2.0.1 Enterprise and OracleSOASuite 12.2.1.3

After succesfully executing RCU, OSB domain creation fails with following error:

createDomain.py called with the following inputs:
INFO: sys.argv[0] = /u01/oracle/dockertools/createDomain.py
INFO: sys.argv[1] = -oh
INFO: sys.argv[2] = /u01/oracle
INFO: sys.argv[3] = -jh
INFO: sys.argv[4] = /usr/java/default
INFO: sys.argv[5] = -parent
INFO: sys.argv[6] = /u01/oracle/user_projects/domains
INFO: sys.argv[7] = -name
INFO: sys.argv[8] = soainfra
INFO: sys.argv[9] = -password
INFO: sys.argv[10] = start123
INFO: sys.argv[11] = -rcuDb
INFO: sys.argv[12] = oracleDb:1521/soapdb
INFO: sys.argv[13] = -rcuPrefix
INFO: sys.argv[14] = OSB01
INFO: sys.argv[15] = -rcuSchemaPwd
INFO: sys.argv[16] = start123
INFO: sys.argv[17] = -domainType
INFO: sys.argv[18] = osb
INFO: Creating Node Managers...
INFO: Creating Admin server...
INFO: Creating OSB cluster...
INFO: OSB Servers created.....
INFO: Writing base domain...
INFO: Base domain created at /u01/oracle/user_projects/domains/soainfra
INFO: Extending domain at /u01/oracle/user_projects/domains/soainfra
INFO: Applying JRF templates...
INFO: Applying OSB templates...
INFO: Extension Templates added
INFO: Configuring the Service Table DataSource...
INFO: Getting Database Defaults...
Error: getDatabaseDefaults() failed. Do dumpStack() to see details.
Error: runCmd() failed. Do dumpStack() to see details.
Problem invoking WLST - Traceback (innermost last):
File "/u01/oracle/dockertools/createDomain.py", line 483, in ?
File "/u01/oracle/dockertools/createDomain.py", line 118, in createDomain
File "/u01/oracle/dockertools/createDomain.py", line 312, in extendOsbDomain
File "/u01/oracle/dockertools/createDomain.py", line 237, in configureJDBCTemplates
File "/tmp/WLSTOfflineIni4641236230965879151.py", line 263, in getDatabaseDefaults
File "/tmp/WLSTOfflineIni4641236230965879151.py", line 19, in command
Failed to build JDBC Connection object:
at com.oracle.cie.domain.script.jython.CommandExceptionHandler.handleException(CommandExceptionHandler.java:69)
at com.oracle.cie.domain.script.jython.WLScriptContext.handleException(WLScriptContext.java:2983)
at com.oracle.cie.domain.script.jython.WLScriptContext.runCmd(WLScriptContext.java:735)
at sun.reflect.GeneratedMethodAccessor155.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)

com.oracle.cie.domain.script.jython.WLSTException: com.oracle.cie.domain.script.jython.WLSTException: Got exception when auto configuring the schema component(s) with data obtained from shadow table:
Failed to build JDBC Connection object:

ERROR: Domain Configuration failed. Please check the logs

This is the corresponding Database log:

Fatal NI connect error 12170.

VERSION INFORMATION:
TNS for Linux: Version 12.2.0.1.0 - Production
Oracle Bequeath NT Protocol Adapter for Linux: Version 12.2.0.1.0 - Production
TCP/IP NT Protocol Adapter for Linux: Version 12.2.0.1.0 - Production
Time: 16-APR-2019 10:05:25
Tracing not turned on.
Tns error struct:
ns main err code: 12535

TNS-12535: TNS:operation timed out
ns secondary err code: 12606
nt main err code: 0
nt secondary err code: 0
nt OS err code: 0
Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=host)(PORT=port))
2019-04-16T10:05:25.682722+00:00
SOAPDB(3):WARNING: inbound connection timed out (ORA-3136)

It seems to be an issue with the Database connection (maybe authentication?)

The Database is visible in the Docker Network and can be pinged from the OracleSuit container.

My setenv.sh file:

!/bin/sh

export DC_USERHOME=/home/comdev/tobi/docker

export DC_REGISTRY_SOA=localhost
export DC_REGISTRY_DB=localhost

exportComposeEnv() {

export DC_HOSTNAME=localhost

export DC_ORCL_PORT=1521
export DC_ORCL_OEM_PORT=5500
export DC_ORCL_SID=soadb
export DC_ORCL_PDB=soapdb
export DC_ORCL_SYSPWD=start123
export DC_ORCL_HOST=oracleDb

export DC_ORCL_DBDATA=${DC_USERHOME}/dbdata

export DC_ADMIN_PWD=start123

export DC_RCU_SCHPWD=start123
export DC_RCU_SOAPFX=SOA01
export DC_RCU_BPMPFX=BPM01
export DC_RCU_OSBPFX=OSB01

export DC_DDIR_SOA=${DC_USERHOME}/soadomain
export DC_DDIR_BPM=${DC_USERHOME}/bpmdomain
export DC_DDIR_OSB=${DC_USERHOME}/osbdomain

export DC_SOA_VERSION=12.2.1.3
}

===============================================

createDirs() {
mkdir -p ${DC_ORCL_DBDATA} ${DC_DDIR_SOA} ${DC_DDIR_BPM} ${DC_DDIR_OSB}
chmod 777 ${DC_ORCL_DBDATA} ${DC_DDIR_SOA} ${DC_DDIR_BPM} ${DC_DDIR_OSB}
}

===============================================

== MAIN starts here

===============================================

#
echo "INFO: Setting up SOA Docker Environment..."
exportComposeEnv
createDirs

echo "INFO: Environment variables"

env | grep -e "DC_" | sort

My Database docker-compose service:
soadb:
image: oracle/database:12.2.0.1-ee

ports:
  - "${DC_ORCL_PORT}:1521"
  - "${DC_ORCL_OEM_PORT}:5500"
environment:
  - ORACLE_SID=${DC_ORCL_SID}
  - ORACLE_PDB=${DC_ORCL_PDB}
  - ORACLE_PWD=${DC_ORCL_SYSPWD}
container_name: soadb
volumes:
  - ${DC_ORCL_DBDATA}:/opt/oracle/oradata
networks:
  default:
    aliases:
      - oracleDb

My Oracle Suite Docker-compose service:

osbas:
image: ${DC_REGISTRY_SOA}/oracle/soasuite:${DC_SOA_VERSION}
container_name: osbas
command: /bin/bash -c "sleep 5s; /u01/oracle/dockertools/createDomainAndStart.sh"
env_file:
- ../setenv.sh
ports:
- "7001:7001"
#entrypoint: tail -f /dev/null
environment:
- DOMAIN_TYPE=osb
- ADMIN_HOST=${DC_HOSTNAME}
- ADMIN_PASSWORD=${DC_ADMIN_PWD}
- CONNECTION_STRING=${DC_ORCL_HOST}:${DC_ORCL_PORT}/${DC_ORCL_PDB}
- DB_PASSWORD=${DC_ORCL_SYSPWD}
- DB_SCHEMA_PASSWORD=${DC_RCU_SCHPWD}
- RCUPREFIX=${DC_RCU_OSBPFX}
- MANAGED_SERVER=osb_server1
volumes:
- ${DC_DDIR_OSB}:/u01/oracle/user_projects

I'm also able to connect to the DB from Oracle Suite container using sqlplus:

/sqlplus sys/start123@//oracleDb:1521/soadb as sysdba

So maybe something is wrong how the configureJDBCTemplates(self,db,dbPrefix,dbPassword) within the createDomain.py script is setting up the DB connection

Any help would be really appreciated.

[renancastro]

Exaclty same issue here, not being able to use Oracle SOA Suite image because of this error.

[fabiojose]

I have the same problem!

[fabiojose]

Hi folks.

In my case, the problems were with the OracleDB container. Because of unknown reasons, every time I got the error above, I got the error below too:

***********************************************************************

Fatal NI connect error 12170.

  VERSION INFORMATION:
    TNS for Linux: Version 12.2.0.1.0 - Production
    Oracle Bequeath NT Protocol Adapter for Linux: Version 12.2.0.1.0 - Production
    TCP/IP NT Protocol Adapter for Linux: Version 12.2.0.1.0 - Production
  Time: 18-JUN-2019 01:05:41
  Tracing not turned on.
  Tns error struct:
    ns main err code: 12535

TNS-12535: TNS:operation timed out
    ns secondary err code: 12606
    nt main err code: 0
    nt secondary err code: 0
    nt OS err code: 0
  Client address: (ADDRESS=(PROTOCOL=tcp)(HOST=172.23.0.3)(PORT=52032))
2019-06-18T01:05:41.707988+00:00
SOAPDB(3):WARNING: inbound connection timed out (ORA-3136)

__How I solve it__

I logged in the db container (oracle/database:12.2.0.1-ee) and add this line SQLNET.INBOUND_CONNECT_TIMEOUT=300 int the file located at /opt/oracle/product/12.2.0.1/dbhome_1/network/admin/sqlnet.ora.

No restart is required.

After that no more errors and the things goes up!

[rdas0405]

The solution suggested is acceptable. If required perform "lsnrctl reload" to get the changes effective.

[clouless]

I get the same error only when docker container runs inside my Kubernetes cluster.
On local machine (macOS) directly with docker it does not appear.
On Kubernetes it happens around the fifth connect which then gets timed out.
But your suggestions fix the error, but not really the loooong "almost timeout" of the connection.
(And I am running Oracle Database 12c Release 1 (12.1.0.2) Standard Edition 2)

UPDATE: Solved for me

• Ok I investigated what happened in my setup
• My kubernetes pod was "ready", and my test sql select could connect to Database BUT
• The oracle database was not really ready internally. Meaning: it did not log DATABASE IS READY TO USE!
• My solution was to use kubectl logs mydbpod | grep "DATABASE IS READY TO USE" in a loop and wait until it was successful => then all my following scripts and connections worked fine

Same can be applied for docker with this script wait-for-oracle.sh you can wait for your container to really be ready.

bash wait-for-oracle.sh mycontainernameOrId

#!/bin/bash

# usage: bash wait-for-oracle.sh mycontainername

echo "WAITING FOR ORACLE CONTAINER STARTUP: $1"
for i in 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20
do
  docker logs $1 | grep  "DATABASE IS READY TO USE"
  if [ $? -eq 0 ]; then
    echo "SUCCESS: DB IS READY :) $1"
    exit 0
  else
    echo "."
  fi
  sleep 120
done
echo "ERROR: DB IS NOT UP AFTER SEVERAL TRIES :("
exit 1

[af6140]

My case is the database internally ready for use. Is oracle try to reverse dns lookup the client?

[af6140]

Testing things aroud, do not want to change SQL_NET, it feels like a band aid, after mout /dev/urandom in the container as /dev/random, viola, it starts working.

Different behavior in windows and linux gave me a hint. Most of time it worked on windows machine, but it failed almost all the time on linux server. What is the difference? randomness is one of them.