java jboss进程假死_Jboss集群运行一段时间自己停止。

昌栋

2023-12-01

分析：从几种日志综合看，系统是发现超时(日志中黑色粗体)，然后处于自我保护，做了停机操作。

虽然server-one 有exception，但是时间对不上，而且与停机无关。

处理方法：通过几次试验法，发现如果在linux上使用nohup domain.sh启动就不会有这个我问题。

遗留问题：一般程序如果没有申请守护模式运行，当登陆窗口退出后会立即终止，但jboss显然做了守护，只是不知道为什么还需要nohup影响。

slave的process-controller.log======================================================================

20:04:48,047 INFO [org.jboss.modules] (main) JBoss Modules version 1.2.0.Final-redhat-1

20:04:48,192 INFO [org.jboss.as.process.Host Controller.status] (main) JBAS012017: Starting process 'Host Controller'

20:04:53,745 INFO [org.jboss.as.process.Server:server-two.status] (ProcessController-threads - 3) JBAS012017: Starting process 'Server:server-two'

22:16:02,543 INFO [org.jboss.as.process] (Shutdown thread) JBAS012016: Shutting down process controller

22:16:02,544 INFO [org.jboss.as.process.Host Controller.status] (Shutdown thread) JBAS012018: Stopping process 'Host Controller'

22:16:02,612 INFO [org.jboss.as.process.Server:server-two.status] (ProcessController-threads - 4) JBAS012018: Stopping process 'Server:server-two'

22:16:03,226 INFO [org.jboss.as.process.Server:server-two.status] (reaper for Server:server-two) JBAS012010: Process 'Server:server-two' finished with an exit status of 129

22:16:03,289 INFO [org.jboss.as.process.Host Controller.status] (reaper for Host Controller) JBAS012010: Process 'Host Controller' finished with an exit status of 129

22:16:03,290 INFO [org.jboss.as.process] (Shutdown thread) JBAS012015: All processes finished; exiting

slave的host-controller.log日志=============================================================

20:04:48,967 INFO [org.jboss.modules] (main) JBoss Modules version 1.2.0.Final-redhat-1

20:04:49,166 INFO [org.jboss.msc] (main) JBoss MSC version 1.0.4.GA-redhat-1

20:04:49,260 INFO [org.jboss.as] (MSC service thread 1-2) JBAS015899: JBoss EAP 6.1.0.GA (AS 7.2.0.Final-redhat-8) starting

20:04:49,949 INFO [org.xnio] (MSC service thread 1-2) XNIO Version 3.0.7.GA-redhat-1

20:04:49,961 INFO [org.xnio.nio] (MSC service thread 1-2) XNIO NIO Implementation Version 3.0.7.GA-redhat-1

20:04:49,979 INFO [org.jboss.remoting] (MSC service thread 1-2) JBoss Remoting version 3.2.16.GA-redhat-1

20:04:50,062 INFO [org.jboss.as.remoting] (MSC service thread 1-2) JBAS017100: Listening on 22.188.146.171:9999

20:04:53,693 INFO [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010922: Starting server server-two

20:04:55,704 INFO [org.jboss.as.domain.controller.mgmt] (Remoting "u2r10nwtbapp02:MANAGEMENT" task-3) JBAS010920: Server [Server:server-two] registered using connection [Channel ID 0360ebb9 (inbound) of Remoting connection 5ce5a508 to /22.188.146.171:38676]

20:04:55,855 INFO [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.1.0.GA (AS 7.2.0.Final-redhat-8) (Host Controller) started in 7554ms - Started 11 of 11 services (0 services are passive or on-demand)

20:04:55,893 INFO [org.jboss.as.host.controller] (server-registration-threads - 1) JBAS010919: Registering server server-two

22:15:50,867 WARN [org.jboss.as.host.controller] (Remoting "u2r10nwtbapp02:MANAGEMENT" read-1) JBAS010914: Connection to remote host-controller closed. Trying to reconnect.

22:16:02,597 INFO [org.jboss.as.host.controller] (Host Controller Service Threads - 1) JBAS010923: Stopping server server-two

22:16:02,885 INFO [org.jboss.as.host.controller] (Remoting "u2r10nwtbapp02:MANAGEMENT" read-1) JBAS010926: Unregistering server server-two

22:16:03,228 INFO [org.jboss.as.host.controller] (ProcessControllerConnection-thread - 2) JBAS010926: Unregistering server server-two

22:16:03,282 INFO [org.jboss.as] (MSC service thread 1-4) JBAS015950: JBoss EAP 6.1.0.GA (AS 7.2.0.Final-redhat-8) stopped in 720ms

server-two日志======================================================================

20:05:09,802 INFO [org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping] (ServerService Thread Pool -- 68) Mapped "{[/loanQuery],methods=[GET],params=[],headers=[],consumes=[],produces=[],custom=[]}" onto public java.util.Mapcom.bocsoft.bocebiz.ezsela.loanquery.web.controller.LoanQueryController.loanQuery(org.springframework.ui.ModelMap)

20:05:09,805 INFO [org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping] (ServerService Thread Pool -- 68) Mapped "{[/appealInfoEnter],methods=[GET],params=[],headers=[],consumes=[],produces=[],custom=[]}" onto public java.lang.String com.bocsoft.bocebiz.ezsela.appeal.web.controller.AppealController.appealInfoEnter(java.lang.String,org.springframework.ui.ModelMap)

20:05:09,812 INFO [org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping] (ServerService Thread Pool -- 68) Mapped "{[/appealApply],methods=[POST],params=[],headers=[],consumes=[],produces=[],custom=[]}" onto public java.lang.String com.bocsoft.bocebiz.ezsela.appeal.web.controller.AppealController.appealSubmit(org.springframework.ui.ModelMap,com.bocsoft.bocebiz.ezsela.appeal.server.viewobject.AppealApplyForm)

20:05:09,812 INFO [org.springframework.web.servlet.mvc.method.annotation.RequestMappingHandlerMapping] (ServerService Thread Pool -- 68) Mapped "{[/appealEnter],methods=[GET],params=[],headers=[],consumes=[],produces=[],custom=[]}" onto public java.lang.String com.bocsoft.bocebiz.ezsela.appeal.web.controller.AppealController.appealEnter(org.springframework.ui.ModelMap,javax.servlet.http.HttpServletRequest)

20:05:09,851 INFO [org.hibernate.validator.internal.util.Version] (ServerService Thread Pool -- 68) HV000001: Hibernate Validator 4.3.1.Final-redhat-1

20:05:10,337 INFO [org.springframework.web.servlet.handler.SimpleUrlHandlerMapping] (ServerService Thread Pool -- 68) Mapped URL path [/resources/**] onto handler 'org.springframework.web.servlet.resource.ResourceHttpRequestHandler#0'

20:05:10,671 INFO [org.springframework.web.servlet.DispatcherServlet] (ServerService Thread Pool -- 68) FrameworkServlet 'ezsela': initialization completed in 1383 ms

20:05:10,736 INFO [org.jboss.as.server] (Controller Boot Thread) JBAS018559: Deployed "ezsela.war" (runtime-name : "ezsela.war")

20:05:10,737 INFO [org.jboss.as.server] (Controller Boot Thread) JBAS018559: Deployed "mysql-connector-java-5.1.26.jar" (runtime-name : "mysql-connector-java-5.1.26.jar")

20:05:10,738 INFO [org.jboss.as.server] (Controller Boot Thread) JBAS018559: Deployed "ezsela-message.war" (runtime-name : "ezsela-message.war")

20:05:10,747 INFO [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.1.0.GA (AS 7.2.0.Final-redhat-8) started in 16880ms - Started 602 of 721 services (118 services are passive or on-demand)

22:16:02,611 INFO [org.apache.coyote.http11] (MSC service thread 1-1) JBWEB003075: Coyote HTTP/1.1 pausing on: http-/22.188.146.171:8230

22:16:02,611 INFO [org.apache.coyote.http11] (MSC service thread 1-1) JBWEB003077: Coyote HTTP/1.1 stopping on : http-/22.188.146.171:8230

22:16:02,631 INFO [org.jboss.as.connector.subsystems.datasources] (MSC service thread 1-3) JBAS010409: Unbound data source [java:jboss/datasources/ExampleDS]

22:16:02,644 INFO [org.jboss.web] (ServerService Thread Pool -- 71) JBAS018224: Unregister web context: /ezsela

22:16:02,645 INFO [org.jboss.web] (ServerService Thread Pool -- 72) JBAS018224: Unregister web context: /ezsela-message

22:16:02,653 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/ezsela]] (ServerService Thread Pool -- 71) Destroying Spring FrameworkServlet 'ezsela'

22:16:02,655 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 70) MODCLUSTER000021: All pending requests drained from default-host:/ezsela in 0.005000.1 seconds

22:16:02,665 INFO [org.apache.catalina.core] (MSC service thread 1-4) JBWEB001079: Container org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/] has not been started

22:16:02,689 INFO [org.jboss.as.server.deployment] (MSC service thread 1-2) JBAS015877: Stopped deployment mysql-connector-java-5.1.26.jar (runtime-name: mysql-connector-java-5.1.26.jar) in 96ms

22:16:02,695 INFO [org.jboss.modcluster] (ServerService Thread Pool -- 70) MODCLUSTER000002: Initiating mod_cluster shutdown

22:16:02,699 INFO [org.apache.coyote.ajp] (MSC service thread 1-2) JBWEB003048: Pausing Coyote AJP/1.3 on ajp-/22.188.146.171:8159

22:16:02,719 INFO [org.apache.coyote.ajp] (MSC service thread 1-2) JBWEB003051: Stopping Coyote AJP/1.3 on ajp-/22.188.146.171:8159

22:16:02,717 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/ezsela-message]] (ServerService Thread Pool -- 72) Shutting down log4j

22:16:02,720 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/ezsela-message]] (ServerService Thread Pool -- 72) Closing Spring root WebApplicationContext

22:16:02,710 INFO [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/ezsela]] (ServerService Thread Pool -- 71) Closing Spring root WebApplicationContext

22:16:02,767 INFO [com.alibaba.druid.pool.DruidDataSource] (ServerService Thread Pool -- 71) {dataSource-1} closed

22:16:02,843 INFO [org.jboss.as.server.deployment] (MSC service thread 1-4) JBAS015877: Stopped deployment ezsela-message.war (runtime-name: ezsela-message.war) in 250ms

22:16:02,879 INFO [org.jboss.as.server.deployment] (MSC service thread 1-4) JBAS015877: Stopped deployment ezsela.war (runtime-name: ezsela.war) in 298ms

22:16:02,895 INFO [org.jboss.as] (MSC service thread 1-2) JBAS015950: JBoss EAP 6.1.0.GA (AS 7.2.0.Final-redhat-8) stopped in 273ms

domain的process-controller.log======================================================================

20:04:36,593 INFO [org.jboss.modules] (main) JBoss Modules version 1.2.0.Final-redhat-1

20:04:37,005 INFO [org.jboss.as.process.Host Controller.status] (main) JBAS012017: Starting process 'Host Controller'

20:04:41,977 INFO [org.jboss.as.process.Server:server-one.status] (ProcessController-threads - 3) JBAS012017: Starting process 'Server:server-one'

22:15:50,010 INFO [org.jboss.as.process] (Shutdown thread) JBAS012016: Shutting down process controller

22:15:50,011 INFO [org.jboss.as.process.Host Controller.status] (Shutdown thread) JBAS012018: Stopping process 'Host Controller'

22:15:50,019 INFO [org.jboss.as.process.Server:server-one.status] (ProcessController-threads - 4) JBAS012018: Stopping process 'Server:server-one'

22:15:50,807 INFO [org.jboss.as.process.Server:server-one.status] (reaper for Server:server-one) JBAS012010: Process 'Server:server-one' finished with an exit status of 129

22:15:50,881 INFO [org.jboss.as.process.Host Controller.status] (reaper for Host Controller) JBAS012010: Process 'Host Controller' finished with an exit status of 129

22:15:50,882 INFO [org.jboss.as.process] (Shutdown thread) JBAS012015: All processes finished; exiting

domain的host-controller.log日志=============================================================

20:04:39,510 INFO [org.jboss.remoting] (MSC service thread 1-3) JBoss Remoting version 3.2.16.GA-redhat-1

20:04:39,625 INFO [org.jboss.as.remoting] (MSC service thread 1-2) JBAS017100: Listening on 22.188.146.170:9999

20:04:41,923 INFO [org.jboss.as.host.controller] (Controller Boot Thread) JBAS010922: Starting server server-one

20:04:44,549 INFO [org.jboss.as.domain.controller.mgmt] (Remoting "u2r10nwtbapp01:MANAGEMENT" task-4) JBAS010920: Server [Server:server-one] registered using connection [Channel ID 4d112cc7 (inbound) of Remoting connection 2ba527b8 to /22.188.146.170:54432]

20:04:44,777 INFO [org.jboss.as] (Controller Boot Thread) JBAS015961: Http management interface listening on

20:04:44,778 INFO [org.jboss.as] (Controller Boot Thread) JBAS015951: Admin console listening on

20:04:44,779 INFO [org.jboss.as] (Controller Boot Thread) JBAS015874: JBoss EAP 6.1.0.GA (AS 7.2.0.Final-redhat-8) (Host Controller) started in 7652ms - Started 11 of 11 services (0 services are passive or on-demand)

20:04:44,815 INFO [org.jboss.as.host.controller] (server-registration-threads - 1) JBAS010919: Registering server server-one

20:04:53,580 INFO [org.jboss.as.domain] (slave-request-threads - 1) JBAS010918: Registered remote slave host "slave", JBoss EAP 6.1.0.GA (AS 7.2.0.Final-redhat-8)

22:15:49,981 INFO [org.jboss.as.controller] (MSC service thread 1-1) JBAS014774: Service status report

JBAS014775: New missing/unsatisfied dependencies:

service jboss.server.controller.management.security_realm.ApplicationRealm.properties_authentication (missing) dependents: [service jboss.server.controller.management.security_realm.ApplicationRealm]

22:15:49,986 INFO [org.jboss.as.controller] (MSC service thread 1-4) JBAS014774: Service status report

JBAS014775: New missing/unsatisfied dependencies:

service jboss.server.controller.management.security_realm.ManagementRealm (missing) dependents: [service jboss.serverManagement.controller.management.http, service jboss.remoting.authentication_provider.management]

22:15:50,016 INFO [org.jboss.as.host.controller] (Host Controller Service Threads - 24) JBAS010923: Stopping server server-one

22:15:50,463 INFO [org.jboss.as.host.controller] (Remoting "u2r10nwtbapp01:MANAGEMENT" read-1) JBAS010926: Unregistering server server-one

22:15:50,810 INFO [org.jboss.as.host.controller] (ProcessControllerConnection-thread - 2) JBAS010926: Unregistering server server-one

22:15:50,855 WARN [org.jboss.as.domain] (MSC service thread 1-3) JBAS010929: Connection to remote host "slave" closed unexpectedly

22:15:50,856 INFO [org.jboss.as.domain] (MSC service thread 1-3) JBAS010925: Unregistered remote slave host "slave"

22:15:50,868 INFO [org.jboss.as.controller] (MSC service thread 1-1) JBAS014774: Service status report

JBAS014776: Newly corrected services:

service jboss.server.controller.management.security_realm.ApplicationRealm.properties_authentication (no longer required)

service jboss.server.controller.management.security_realm.ManagementRealm (no longer required)

22:15:50,872 INFO [org.jboss.as] (MSC service thread 1-3) JBAS015950: JBoss EAP 6.1.0.GA (AS 7.2.0.Final-redhat-8) stopped in 876ms

server-one日志======================================================================

21:13:56,068 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/ezsela].[ezsela]] (ajp-/22.188.146.170:8159-15) JBWEB000236: Servlet.service() for servlet ezsela threw exception: java.lang.NullPointerException

at com.bocsoft.bocebiz.ezsela.web.filter.SingleFilter.doFilter(SingleFilter.java:59) [classes:]

at org.springframework.web.filter.DelegatingFilterProxy.invokeDelegate(DelegatingFilterProxy.java:346) [spring-web-3.2.0.RELEASE.jar:3.2.0.RELEASE]

at org.springframework.web.filter.DelegatingFilterProxy.doFilter(DelegatingFilterProxy.java:259) [spring-web-3.2.0.RELEASE.jar:3.2.0.RELEASE]

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]

at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88) [spring-web-3.2.0.RELEASE.jar:3.2.0.RELEASE]

at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107) [spring-web-3.2.0.RELEASE.jar:3.2.0.RELEASE]

at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:246) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]

at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]

at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:230) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]

at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:149) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]

at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.event(JBossWebContext.java:91)

at org.jboss.modcluster.container.jbossweb.JBossWebContext$RequestListenerValve.invoke(JBossWebContext.java:72)

at org.jboss.as.web.security.SecurityContextAssociationValve.invoke(SecurityContextAssociationValve.java:169) [jboss-as-web-7.2.0.Final-redhat-8.jar:7.2.0.Final-redhat-8]

at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:145) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]

at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:97) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]

at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:102) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]

at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:336) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]

at org.apache.coyote.ajp.AjpProcessor.process(AjpProcessor.java:488) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]

at org.apache.coyote.ajp.AjpProtocol$AjpConnectionHandler.process(AjpProtocol.java:420) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]

at org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:920) [jbossweb-7.2.0.Final-redhat-1.jar:7.2.0.Final-redhat-1]

at java.lang.Thread.run(Thread.java:722) [rt.jar:1.7.0_09-icedtea]

21:13:56,075 ERROR [org.apache.catalina.core.ContainerBase.[jboss.web].[default-host].[/ezsela].[ezsela]] (ajp-/22.188.146.170:8159-9) JBWEB000236: Servlet.service() for servlet ezsela threw exception: java.lang.NullPointerException