Important steps of clustering WSO2 API Manager in hybrid mode or worker manager seperated mode

In this post I'm discussing some important facts about additional transports and port mappings in WSO2 Elastic Load Balancer and WSO2 API Manager product clustering scenarios. To find out a complete guide you have to follow WSO2 Product clustering documentations.

WSO2 API Manager is a descendant of WSO2 ESB. So it uses a few more than one transports which is Servlet and Passthrough

I'm using 101 as ELB and APIM's port offset, so the port mapping will be as following

ELB	Port offset 101	APIM	Port offset 101
http	8381	http (pt)	8381
https	8344	https (pt)	8344
http2	8382	http2 (servlet)	9864
https2	8345	https2 (servlet)	9544

When fronting an API-M with ELB, you have to open two more transports in ELB to map to the APIM's extra transports. For that you have to add additional transportReceiver and transportSender configurations.  That is done as following.

ELB - axis2.xml

transportRecceiver

   <transportReceiver name="http" class="org.apache.synapse.transport.passthru.PassThroughHttpListener">  
    <parameter name="port">8280</parameter>  
    <parameter name="non-blocking"> true</parameter>  
    <parameter name="httpGetProcessor" locked="false">org.wso2.carbon.transport.nhttp.api.PassThroughNHttpGetProcessor</parameter>  
   </transportReceiver>  
   <transportReceiver name="http2" class="org.apache.synapse.transport.passthru.PassThroughHttpListener">  
    <parameter name="port">8281</parameter>  
    <parameter name="non-blocking"> true</parameter>  
    <parameter name="httpGetProcessor" locked="false">org.wso2.carbon.transport.nhttp.api.PassThroughNHttpGetProcessor</parameter>  
   </transportReceiver>  
   <transportReceiver name="https" class="org.apache.synapse.transport.passthru.PassThroughHttpSSLListener">  
     <parameter name="port" locked="false">8243</parameter>  
     <parameter name="non-blocking" locked="false">true</parameter>  
     <parameter name="httpGetProcessor" locked="false">org.wso2.carbon.transport.nhttp.api.PassThroughNHttpGetProcessor</parameter>  
     <!--parameter name="bind-address" locked="false">hostname or IP address</parameter-->  
     <!--parameter name="WSDLEPRPrefix" locked="false">https://apachehost:port/somepath</parameter-->  
     <parameter name="keystore" locked="false">  
       <KeyStore>  
         <Location>repository/resources/security/wso2carbon.jks</Location>  
         <Type>JKS</Type>  
         <Password>wso2carbon</Password>  
         <KeyPassword>wso2carbon</KeyPassword>  
       </KeyStore>  
     </parameter>  
     <parameter name="truststore" locked="false">  
       <TrustStore>  
         <Location>repository/resources/security/client-truststore.jks</Location>  
         <Type>JKS</Type>  
         <Password>wso2carbon</Password>  
       </TrustStore>  
     </parameter>  
   </transportReceiver>  
   <transportReceiver name="https2" class="org.apache.synapse.transport.passthru.PassThroughHttpSSLListener">  
     <parameter name="port" locked="false">8244</parameter>  
     <parameter name="non-blocking" locked="false">true</parameter>  
     <parameter name="httpGetProcessor" locked="false">org.wso2.carbon.transport.nhttp.api.PassThroughNHttpGetProcessor</parameter>  
     <!--parameter name="bind-address" locked="false">hostname or IP address</parameter-->  
     <!--parameter name="WSDLEPRPrefix" locked="false">https://apachehost:port/somepath</parameter-->  
     <parameter name="keystore" locked="false">  
       <KeyStore>  
         <Location>repository/resources/security/wso2carbon.jks</Location>  
         <Type>JKS</Type>  
         <Password>wso2carbon</Password>  
         <KeyPassword>wso2carbon</KeyPassword>  
       </KeyStore>  
     </parameter>  
     <parameter name="truststore" locked="false">  
       <TrustStore>  
         <Location>repository/resources/security/client-truststore.jks</Location>  
         <Type>JKS</Type>  
         <Password>wso2carbon</Password>  
       </TrustStore>  
     </parameter>  
   </transportReceiver>  

transportSender

   <transportSender name="http" class="org.apache.synapse.transport.passthru.PassThroughHttpSender">  
     <parameter name="non-blocking" locked="false">true</parameter>  
     <parameter name="warnOnHTTP500" locked="false">*</parameter>  
     <!--parameter name="http.proxyHost" locked="false">localhost</parameter>  
     <parameter name="http.proxyPort" locked="false">3128</parameter>  
     <parameter name="http.nonProxyHosts" locked="false">localhost|moon|sun</parameter-->  
   </transportSender>  
   <transportSender name="http2" class="org.apache.synapse.transport.passthru.PassThroughHttpSender">  
     <parameter name="non-blocking" locked="false">true</parameter>  
     <parameter name="warnOnHTTP500" locked="false">*</parameter>  
     <!--parameter name="http.proxyHost" locked="false">localhost</parameter>  
     <parameter name="http.proxyPort" locked="false">3128</parameter>  
     <parameter name="http.nonProxyHosts" locked="false">localhost|moon|sun</parameter-->  
   </transportSender>  
   <transportSender name="https" class="org.apache.synapse.transport.passthru.PassThroughHttpSSLSender">  
     <parameter name="non-blocking" locked="false">true</parameter>  
     <parameter name="keystore" locked="false">  
       <KeyStore>  
         <Location>repository/resources/security/wso2carbon.jks</Location>  
         <Type>JKS</Type>  
         <Password>wso2carbon</Password>  
         <KeyPassword>wso2carbon</KeyPassword>  
       </KeyStore>  
     </parameter>  
     <parameter name="truststore" locked="false">  
       <TrustStore>  
         <Location>repository/resources/security/client-truststore.jks</Location>  
         <Type>JKS</Type>  
         <Password>wso2carbon</Password>  
       </TrustStore>  
     </parameter>  
     <parameter name="HostnameVerifier">AllowAll</parameter>  
    </transportSender>  
   <transportSender name="https2" class="org.apache.synapse.transport.passthru.PassThroughHttpSSLSender">  
     <parameter name="non-blocking" locked="false">true</parameter>  
     <parameter name="keystore" locked="false">  
       <KeyStore>  
         <Location>repository/resources/security/wso2carbon.jks</Location>  
         <Type>JKS</Type>  
         <Password>wso2carbon</Password>  
         <KeyPassword>wso2carbon</KeyPassword>  
       </KeyStore>  
     </parameter>  
     <parameter name="truststore" locked="false">  
       <TrustStore>  
         <Location>repository/resources/security/client-truststore.jks</Location>  
         <Type>JKS</Type>  
         <Password>wso2carbon</Password>  
       </TrustStore>  
     </parameter>  
     <parameter name="HostnameVerifier">AllowAll</parameter>  
       <!--supports Strict|AllowAll|DefaultAndLocalhost or the default if none specified -->  
    </transportSender>  

The loadbalancer.conf of ELB need to have the information of the APIM cluster  such as domains, and hosts. Normally the hosts are separated as manager and worker. But here since, I'm going to use the APIM instance as a hybrid instance the separation of host names as management and worker will be a wrong configuration.

If an APIM node joined the ELB as a management node, even if it's Passthrough transports are mapped into ELB's one transport, the requests will not be served, since ELB doesn't know if APIM node is capable of serving pt requests.

So the loadbalancer.conf configuration specific to the APIM is as following.

   apimanager {  
     # multiple hosts should be separated by a comma.  
     #url_suffix       esb.wso2.com;  
     domains  {  
       wso2.am160.domain {  
         tenant_range  *;  
         group_mgt_port 4050;  
           hosts mgt.am160.gw.com,am160.gw.com;  
       }  
     }  
   }  

if, however, someone need to use seperate management and worker nodes for gateway, then the elb configuraion should be as following. Note the difference.

   apimanager {  
     # multiple hosts should be separated by a comma.  
     #url_suffix       esb.wso2.com;  
     domains  {  
       wso2.am160.domain {  
         tenant_range  *;  
         group_mgt_port 4050;  
         mgt {  
           hosts mgt.am160.gw.com;  
         }  
         worker {  
           hosts am160.gw.com;  
         }  
     }  
   }  

In the APIM side, other than the passthrough transports, I have opened the Servlet transports naming their transport names to match with those in the ELB. This is not mandatory since it is by default open in any wso2 server. But I've done this specifically so that the transport Receiver names matches with the ELB's ones.

   <transportReceiver name="http2"  
             class="org.wso2.carbon.core.transports.http.HttpTransportListener">  
     <!--  
       Uncomment the following if you are deploying this within an application server. You  
       need to specify the HTTP port of the application server  
     -->  
     <parameter name="port">9763</parameter>  
     <!--  
     Uncomment the following to enable Apache2 mod_proxy. The port on the Apache server is 80  
     in this case.  
     -->  
     <!--<parameter name="proxyPort">80</parameter>-->  
   </transportReceiver>  
   <transportReceiver name="https2"  
             class="org.wso2.carbon.core.transports.http.HttpsTransportListener">  
     <!--  
       Uncomment the following if you are deploying this within an application server. You  
       need to specify the HTTPS port of the application server  
     -->  
     <parameter name="port">9443</parameter>  
     <!--  
     Uncomment the following to enable Apache2 mod_proxy. The port on the Apache server is 443  
     in this case.  
     -->  
     <!--<parameter name="proxyPort">443</parameter>-->  
   </transportReceiver>  

The other most important thing is the port mapping.

Those transports we have defined as http and https in ELB and in APIM will be mapped together when the APIM joins the ELB, since those are the default names that is used as transportSender and transportReceiver names.

The other transports in APIM which are the Servlet transports need to be mapped to ELBs, additionally opened transports, so that ELB knows how to server Management requests of the APIM node.  The configuration is as follows.

     <parameter name="properties">  
       <property name="backendServerURL" value="https://${hostName}:${httpsPort}/services/"/>  
       <property name="mgtConsoleURL" value="https://${hostName}:${httpsPort}/"/>  
       <property name="port.mapping.8382" value="9864"/>  
       <property name="port.mapping.8345" value="9544"/>  
       <!--<property name="subDomain" value="mgt"/>-->  
     </parameter>  

See that I have commented out the subDomain property so that the APIM does not specifically say ELB, that it is a management or worker node. So that ELB can serve Passthrough and Servlet requests together.

Oracle Database Commands

In Oracle unlike mysql, the database is represented by a schema owner.

A database schema is a logical container for data structures, called schema objects. Examples of schema objects are tables and indexes.

A database user has a password and particular database privileges assigned by the database admin.

In Oracle each user owns a single schema, which has the same name as the user. The schema contains the data for the user owning the schema. For example, the user "chamara" owns the schema "chamara" which contains schema objects such as tables in the database. In a production database, the schema owner usually represents a database application rather than a person.

Using the following commands u can create a schema in oracle database, which is a database you can used within your application.

Connect to the database as the db admin;

 sqlplus sys/admin as sysdba  

Drop user/schema if exists;

 drop user chamaraa cascade;

Create user/schema;

 create user chamaraa identified by chamaraa account unlock;

Grant connect permission;

 grant connect to chamaraa;

Grant other permisssions;

 grant create session,dba to chamaraa;

finally;

 commit;
 exit;

Connect to the created schema;

 sqlplus chamaraa/chamaraa@orcl

Running a script;

 SQL> @C:\path\oracle.sql;

Simple HTTP GET request processor with Perl

Perl script;

 #!/usr/bin/perl  
 use HTTP::Daemon;  
 use HTTP::Status;  
 use Net::HTTP;  
 my $server = HTTP::Daemon->new(  
   LocalAddr => 'chamaradell',   
   LocalPort => '3113') || die;  
 system(clear);    
 print ("HTTP Server is up at: <URL: ", $server->url, "\n");  
 while(my $client = $server->accept) {  
   while(my $request = $client->get_request) {  
      if($request->method eq 'GET') {  
        print "GET request accepted\n";        
        $client->send_status_line(200, "OK");  
       $client->send_header("Content-Type", "text/html");  
       $client->send_file_response("index.html");  
       $client->close;  
      }  
   }   
   $client->close;  
   undef($client);  
 }  

A simple html page;

 <!DOCTYPE html>  
 <html>  
   <body>  
    <h1>Index</h1>  
    <p>This is a test HTTP Server<p>  
   </body>  
 </html>  

The HTTP server will be started after running the perl script. Now if the browser is pointed to the correct url;

 http://chamaradell:3113/  

This will give the html page in the browser.

If the content is requested using curl (as following);

 $ curl -v -X GET http://chamaradell:3113  

The content will be returned as;

 * Rebuilt URL to: http://chamaradell:3113/  
 * Adding handle: conn: 0x19ecbd0  
 * Adding handle: send: 0  
 * Adding handle: recv: 0  
 * Curl_addHandleToPipeline: length: 1  
 * - Conn 0 (0x19ecbd0) send_pipe: 1, recv_pipe: 0  
 * About to connect() to chamaradell port 3113 (#0)  
 *  Trying 127.0.1.1...  
 * Connected to chamaradell (127.0.1.1) port 3113 (#0)  
 > GET / HTTP/1.1  
 > User-Agent: curl/7.32.0  
 > Host: chamaradell:3113  
 > Accept: */*  
 >   
 < HTTP/1.1 200 OK  
 < Content-Type: text/html  
 < HTTP/1.1 200 OK  
 < Date: Sat, 08 Feb 2014 17:32:49 GMT  
 * Server libwww-perl-daemon/6.01 is not blacklisted  
 < Server: libwww-perl-daemon/6.01  
 < Content-Type: text/html  
 < Content-Length: 112  
 < Last-Modified: Sat, 08 Feb 2014 17:25:01 GMT  
 <   
 <!DOCTYPE html>  
 <html>  
   <body>  
    <h1>Index</h1>  
    <p>This is a test HTTP Server<p>  
   </body>  
 </html>  
 * Connection #0 to host chamaradell left intact  

The HTTP Status 200 is returned by the server by default. You can return any status using that method and test various http clients with this server, for various conditions.