zabbix

Error: zabbix gateway unable to support wildfly http-remoting service

Posted on Updated on

Here is my workaround to support http-remoting in zabbix,

I have confiured zabbix server with zabbix agent to monitor production server, since we switch jboss-4.2 to wildfly-8.2 it got messed-up, zabbix unable to communicate with  wildfly. the following error found the Zabbix-gateway server. The error seems like wildlfy is communicating with native RMI service.

2017-07-20 01:18:10.247 [pool-1-thread-3] WARN com.zabbix.gateway.SocketProcessor - error processing request
com.zabbix.gateway.ZabbixException: java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 192.168.101.197; nested exception is:
 java.net.ConnectException: Connection refused (Connection refused)]
 at com.zabbix.gateway.JMXItemChecker.getValues(JMXItemChecker.java:97) ~[zabbix-java-gateway-3.2.6.jar:na]
 at com.zabbix.gateway.SocketProcessor.run(SocketProcessor.java:62) ~[zabbix-java-gateway-3.2.6.jar:na]
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142) [na:1.8.0_131]
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617) [na:1.8.0_131]
 at java.lang.Thread.run(Thread.java:748) [na:1.8.0_131]
Caused by: java.io.IOException: Failed to retrieve RMIServer stub: javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: 192.168.101.197; nested exception is:
 java.net.ConnectException: Connection refused (Connection refused)]
 at javax.management.remote.rmi.RMIConnector.connect(RMIConnector.java:369) ~[na:1.8.0_131]
 at javax.management.remote.JMXConnectorFactory.connect(JMXConnectorFactory.java:270) ~[na:1.8.0_131]
 at com.zabbix.gateway.ZabbixJMXConnectorFactory$1.run(ZabbixJMXConnectorFactory.java:76) ~[zabbix-java-gateway-3.2.6.jar:na]
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[na:1.8.0_131]
 at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[na:1.8.0_131]
 ... 3 common frames omitted

I have download srcRPM https://repo.zabbix.com/zabbix/3.2/rhel/7/SRPMS/zabbix-3.2.6-1.el7.src.rpm , to check (JMXItemChecker.java:97) since it was mentioned in the error message. the same like my thought the protocol was hotcoded in the source. I have did some patch work and recompile the RPM and install Zabbix-gatway it worked as expected.

 

 

Please follow the steps which worked for me, you can download the version of zabbix server installed and reinstall the zabbix-gateway rpm.

1, Check the zabbix version installed. those installation was through yum

rpm -qa | grep zabbix
zabbix-java-gateway-3.2.6-1.el7.x86_64
zabbix-server-mysql-3.2.6-1.el7.x86_64
zabbix-agent-3.2.6-1.el7.x86_64
zabbix-release-3.2-1.el7.noarch
zabbix-web-3.2.6-1.el7.noarch
zabbix-get-3.2.6-1.el7.x86_64
zabbix-web-mysql-3.2.6-1.el7.noarch
zabbix-sender-3.2.4-1.el7.x86_64

2, Remove installed zabbix-gateway only

rpm -e zabbix-java-gateway-3.2.6-1.el7.x86_64
warning: /etc/zabbix/zabbix_java_gateway.conf saved as /etc/zabbix/zabbix_java_gateway.conf.rpmsave

3, Download and extract Source RPM

wget https://repo.zabbix.com/zabbix/3.2/rhel/7/SRPMS/zabbix-3.2.6-1.el7.src.rpm
rpm -ivh zabbix-3.2.6-1.el7.src.rpm

4, Modify the source to support http-remoting

cd /root/rpmbuild/SOURCES/
tar -zxvf zabbix-3.2.6.tar.gz
vi zabbix-3.2.6/src/zabbix_java/src/com/zabbix/gateway/JMXItemChecker.java

Modify with below entries, it checks the port 9990 with go to the “if” block others will go to the “else” block, so your wildlfy management port should be default port (9990) or you can modify the code as what you would like to use for wildlfy custom management port.

String conn = request.getString(JSON_TAG_CONN);
 int port = request.getInt(JSON_TAG_PORT);

 Integer remoting = new Integer("9990");
 int retval = remoting.compareTo(port);

if (retval == 0)
 {
 url = new JMXServiceURL("service:jmx:http-remoting-jmx://" + conn + ":" + port);
 }
 else
 {
 url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://" + conn + ":" + port + "/jmxrmi");
 }

//url = new JMXServiceURL("service:jmx:rmi:///jndi/rmi://[" + conn + "]:" + port + "/jmxrmi");
 jmxc = null;
 mbsc = null;

Save and exit the file

5, Pack the source file which extracted to modify the .java

tar -zcvf zabbix-3.2.6.tar.gz zabbix-3.2.6

6, Install dependency to rebuild rpm

yum install iksemel-devel java-devel libssh2-devel OpenIPMI-devel unixODBC-devel gnutls-devel net-snmp-devel postgresql-devel -y

7, Rebuild the rpm

cd /root/rpmbuild/
rpmbuild -v -bb --clean SPECS/zabbix.spec

8, Install new zabbix-gateway

cd /root/rpmbuild//RPMS/

rpm -ivh zabbix-java-gateway-3.2.6-1.el7.centos.x86_64.rpm
rm -rf /etc/zabbix/zabbix_java_gateway.conf
mv /etc/zabbix/zabbix_java_gateway.conf.rpmsave /etc/zabbix/zabbix_java_gateway.conf

9, Now restart zabbix-gateway

service zabbix-java-gateway restart

Now you can see the zabbix can connect to wildfly and read metrics, I believe this could be very helpful those who switching to wildfly.

Newly installed zabbix-java-gateway package

 

 


rpm -qa | grep zabbix
zabbix-java-gateway-3.2.6-1.el7.centos.x86_64
zabbix-server-mysql-3.2.6-1.el7.x86_64
zabbix-agent-3.2.6-1.el7.x86_64
zabbix-release-3.2-1.el7.noarch
zabbix-web-3.2.6-1.el7.noarch
zabbix-get-3.2.6-1.el7.x86_64
zabbix-web-mysql-3.2.6-1.el7.noarch
zabbix-sender-3.2.4-1.el7.x86_64

This entry was posted in zabbix and tagged , , , , , , , .