ACTIVEMQ 遇到无法解决的问题

发布于 2021-11-11 14:20:10 字数 7083 浏览 714 评论 12

最近使用activeMq作为消息总线服务，但是不管如何修改配置也好，总是过不了多久就失去响应，导致前端服务堵死。经检查都是应为JVM在GC的时候僵死导致的，S1区，E区，O区都已经100%占用，但是查看服务端的logs有没有outofmemory异常，找不到这样异常的GC原因，也没有解决办法。amq版本是5.4.3，更新到5.6.0后问题依旧，jdk版本是 "1.6.0_27"，消息速率严格控制在60-70/秒

JVM参数：-server -Xmx3g -Xms3g -Xmn1024m -XX:PermSize=64m -Xss128k -XX:+DisableExplicitGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled -XX:+UseCMSCompactAtFullCollection -XX:+UseFastAccessorMethods -XX:+UseCMSInitiatingOccupancyOnly -XX:CMSInitiatingOccupancyFraction=70 -Dorg.apache.activemq.UseDedicatedTaskRunner=false

启动命令：

/opt/apache-activemq-5.4.3/bin/activemq start xbean:file:../conf/activemq.test.xml

仅使用了一个topic，然后客户端的话采用的是非事务性的发送机制，非持久化的订阅模式，failover模式：

(ip这里隐去)failover:(tcp://xxx.xxx.xxx.xxx:61618?wireFormat.maxInactivityDuration=10000)&timeout=400&maxReconnectDelay=10000

每次运行一段时间以后JVM HEAP就满了，不停的FGC导致服务端失去响应，使用jmap -histo:live XXX检查内存使用，发现大量的对象如下：

 1:       5203857      666166616  [C
   2:       5110200      245289600  org.apache.activemq.command.ProducerId
   3:       5109099      245236752  org.apache.activemq.command.ProducerInfo
   4:       5109306      204372240  org.apache.activemq.command.SessionId
   5:       5204236      166535552  java.lang.String
   6:       5109522      163504704  java.util.concurrent.ConcurrentHashMap$HashEntry
   7:       5109099      122618376  org.apache.activemq.state.ProducerState
   8:       5109225       81747600  org.apache.activemq.command.ConnectionId
   9:          5824       42077672  [Ljava.util.concurrent.ConcurrentHashMap$HashEntry;
  10:         46320        7025536  <constMethodKlass>
  11:         46320        6308096  <methodKlass>
  12:          5313        5534184  [B
  13:          4116        4733048  <constantPoolKlass>
  14:         73509        4319912  <symbolKlass>
  15:          4116        3257576  <instanceKlassKlass>
  16:          3525        2924488  <constantPoolCacheKlass>
  17:         68094        2723760  org.apache.activemq.broker.ProducerBrokerExchange

已经查阅了很多资料，也尝试过很多修改，没有办法解决，特求高人帮忙

如下是mq的启动配置文件

	<!--
		Licensed to the Apache Software Foundation (ASF) under one or more contributor license agreements. See the NOTICE file
		distributed with this work for additional information regarding copyright ownership. The ASF licenses this file to You
		under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the
		License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0 Unless required by
		applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS,
		WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language
		governing permissions and limitations under the License.
	-->
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:amq="http://activemq.apache.org/schema/core"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
  http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">

	<!-- Allows us to use system properties as variables in this configuration file -->
	<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
		<property name="locations">
			<value>file:${activemq.base}/conf/credentials.properties</value>
		</property>
	</bean>

	<!-- 
        The <broker> element is used to configure the ActiveMQ broker. 
    -->
	<broker xmlns="http://activemq.apache.org/schema/core" brokerName="atmq" dataDirectory="${activemq.base}/data"
		destroyApplicationContextOnStop="true" persistent="false" advisorySupport="false">
		<!--
			For better performances use VM cursor and small memory limit. For more information, see:

			http://activemq.apache.org/message-cursors.html Also, if your producer is "hanging", it's probably due to producer
			flow control. For more information, see: http://activemq.apache.org/producer-flow-control.html
		-->
		<destinationPolicy>
			<policyMap>
				<policyEntries>
					<policyEntry topic=">" producerFlowControl="true" memoryLimit="512mb">
						<pendingSubscriberPolicy>
							<vmCursor />
						</pendingSubscriberPolicy>
					</policyEntry>
					<policyEntry queue=">" producerFlowControl="true" memoryLimit="512mb">
						<!--
							Use VM cursor for better latency For more information, see: http://activemq.apache.org/message-cursors.html

							<pendingQueuePolicy> <vmQueueCursor/> </pendingQueuePolicy>
						-->
					</policyEntry>
				</policyEntries>
			</policyMap>
		</destinationPolicy>


		<!--
			The managementContext is used to configure how ActiveMQ is exposed in JMX. By default, ActiveMQ uses the MBean server
			that is started by the JVM. For more information, see: http://activemq.apache.org/jmx.html
		-->
		<managementContext>
			<managementContext createConnector="false" />
		</managementContext>

		<!--
			Configure message persistence for the broker. The default persistence mechanism is the KahaDB store (identified by
			the kahaDB tag). For more information, see: http://activemq.apache.org/persistence.html
		-->
		<persistenceAdapter>
			<kahaDB directory="${activemq.base}/data/kahadb" />
		</persistenceAdapter>


		<!--
			The systemUsage controls the maximum amount of space the broker will use before slowing down producers. For more
			information, see: http://activemq.apache.org/producer-flow-control.html
		-->
		<systemUsage>
			<systemUsage>
				<memoryUsage>
					<memoryUsage limit="256 mb" />
				</memoryUsage>
				<storeUsage>
					<storeUsage limit="1 gb" name="foo" />
				</storeUsage>
				<tempUsage>
					<tempUsage limit="128 mb" />
				</tempUsage>
			</systemUsage>
		</systemUsage>

		<!--
			The transport connectors expose ActiveMQ over a given protocol to clients and other brokers. For more information,
			see: http://activemq.apache.org/configuring-transports.html
		-->
		<transportConnectors>
			<transportConnector name="openwire" uri="tcp://0.0.0.0:61618" />
		</transportConnectors>

	</broker>

	<!--
		Enable web consoles, REST and Ajax APIs and demos Take a look at activemq-jetty.xml for more details
	-->
	<import resource="jetty.xml" />

</beans>

分享到QQ

分享到微博