我有一个RabbitMQ代理,它设置了多个队列。在客户端(Java ),我有多个消费者,他们都像这样监听他们的队列:
队列_1-
它们都使用一个连接但不同的通道。发生的情况是,当我加载所有队列并启动应用程序代理服务时,首先服务一个队列,而不是另一个队列,依此类推。因此,消息一次由各自的消费者一个队列接收。我还想提一下,我正在使用预取计数1来实现消费者流量的公平分配。
我怎样才能让它发生,让所有的队列得到平等的服务。
编辑:下面是创建消费者的代码(非常基本)
import com.rabbitmq.client.*;
import org.apache.log4j.Logger;
import java.io.IOException;
import java.util.UUID;
import java.util.concurrent.TimeoutException;
/**
* Used for consuming and acknowledging messages from defined queue.
*
*/
public class Consumer {
private final static Logger logger = Logger.getLogger(Consumer.class);
// Maximum number of messages that can be on the consumer at a time
private static int prefetchCount = 1;
// Internal enum which contains queue names and their exchange keys
private Queue queue;
private Channel channel;
private String consumerTag;
private String uuid = UUID.randomUUID().toString();
private boolean subscribed = false;
private DeliverCallback deliverCallback = this::handleDeliver;
private CancelCallback cancelCallback = this::handleCancel;
private ConsumerShutdownSignalCallback consumerShutdownSignalCallback = this::handleShutdown;
/**
* The constructors sets the channel to RabbitMQ broker for the specified queue.
* Callback for events are set to their default implementation.
*
* @param queue RabbitMQ queue - this consumer will be assigned to this queue and will only be able to consume from it.
* @see #setDeliverCallback(DeliverCallback)
* @see #setCancelCallback(CancelCallback)
* @see #setConsumerShutdownSignalCallback(ConsumerShutdownSignalCallback)
*/
public Consumer(Queue queue) {
this.queue = queue;
try {
setUpChannel();
} catch (IOException e) {
e.printStackTrace();
}
}
public Class getEntityClass() {
return Queue.getEntityClassForQueue(queue);
}
public String getUuid() {
return uuid;
}
public boolean isSubscribed() {
return subscribed;
}
public DeliverCallback getDeliverCallback() {
return deliverCallback;
}
public void setDeliverCallback(DeliverCallback deliverCallback) {
this.deliverCallback = deliverCallback;
}
public CancelCallback getCancelCallback() {
return cancelCallback;
}
public void setCancelCallback(CancelCallback cancelCallback) {
this.cancelCallback = cancelCallback;
}
public ConsumerShutdownSignalCallback getConsumerShutdownSignalCallback() {
return consumerShutdownSignalCallback;
}
public void setConsumerShutdownSignalCallback(ConsumerShutdownSignalCallback consumerShutdownSignalCallback) {
this.consumerShutdownSignalCallback = consumerShutdownSignalCallback;
}
/**
* <p>
* Subscribes to the set queue. The subscription can be cancelled using
* Checks if the queue is set up properly.
* </p>
* <p>
* Note: this is a non-blocking operation. The client will listen for incoming messages and handle them using
* the provided DeliveryCallback function but the execution of this operation will be on another thread.
* </p>
*
* @throws IOException if I/O problem is encountered.
*/
public void subscribeToQueue() throws IOException {
if (channel != null) {
consumerTag = channel.basicConsume(
queue.getQueueName(),
deliverCallback,
cancelCallback,
consumerShutdownSignalCallback
);
subscribed = true;
} else {
logger.error("Channel does not exist. Unable to consume message.");
}
}
/**
* Confirms the message has been successfully processed.
*
* @param deliveryTag Unique message tag generated by the server.
* @throws IOException if I/O problem is encountered.
*/
public void acknowledgeMessageReceived(long deliveryTag) throws IOException {
if (channel != null) {
channel.basicAck(deliveryTag, false);
} else {
logger.error("Channel does not exist. Unable to acknowledge message delivery.");
}
}
/**
* Sends a negative acknowledgement to RabbitMQ without re-queueing the message.
*
* @param deliveryTag Unique message tag generated by the server.
* @throws IOException if I/O problem is encountered.
*/
public void rejectMessage(long deliveryTag) throws IOException {
if (channel != null) {
channel.basicReject(deliveryTag, false);
} else {
logger.error("Channel does not exist. Unable to reject message delivery.");
}
}
/**
* Cancels consumer subscription to the queue.
* The consumer can be used for acknowledging messages, but will not receive new messages.
* This does not close the underlying channel. To close the channel use closeChannel() method.
*
* @throws IOException
* @see #subscribeToQueue()
* @see #closeChannel()
*/
public void cancelSubscription() throws IOException {
if (channel != null) {
channel.basicCancel(this.consumerTag);
subscribed = false;
} else {
logger.error("Channel does not exist. Unable to cancel consumer subscription.");
}
}
/**
* Explicitly closes channel to the queue.
* After doing this you will not be able to use any of the methods of this class.
*
* @throws IOException if I/O problem is encountered.
* @throws TimeoutException if connection problem occurs.
*/
public void closeChannel() throws IOException, TimeoutException {
if (channel != null) {
channel.close();
channel = null;
logger.info("Closing RabbitMQ consumer channel...");
} else {
logger.error("Channel already closed.");
}
}
/**
* Checks if the queue exists and creates the channel.
* If the queue does not exist channel is set to null and cannot be used.
*
* @throws IOException if I/O problem is encountered.
*/
private void setUpChannel() throws IOException {
channel = ChannelFactory.getInstance().createChannel();
try {
channel.queueDeclarePassive(queue.getQueueName());
channel.basicQos(prefetchCount);
} catch (IOException e) {
// When this exception occurs it renders the channel unusable so it's best set to null.
channel = null;
logger.error(String.format("Queue %s does not exist [%s]", queue.getQueueName(), e.getMessage()));
e.printStackTrace();
}
logger.info("Setting up RabbitMQ consumer channel. Channel successfully initialized: " + (channel != null));
}
/**
* Callback called when a message is delivered to the client.
* Default implementation. Callback acknowledges message received and does nothing with it.
* To use custom implementation use setDeliverCallback method.
*
* @param consumerTag The consumer tag associated with the consumer.
* @param message Message object.
* @see #setDeliverCallback(DeliverCallback)
*/
private void handleDeliver(String consumerTag, Delivery message) {
Envelope envelope = message.getEnvelope();
long deliveryTag = envelope.getDeliveryTag();
logger.info("Message delivered: " + deliveryTag);
try {
channel.basicAck(deliveryTag, false);
} catch (IOException e) {
e.printStackTrace();
}
}
/**
* Callback called when a service is cancelled.
* Default implementation. To use custom implementation specify it in the constructor.
*
* @param consumerTag The consumer tag associated with the consumer.
*/
private void handleCancel(String consumerTag) {
logger.info("Consumer (" + consumerTag + ") cancelled: ");
}
/**
* Called when the consumer is abruptly shutdown due to termination of the underlying connection or channel.
* Default implementation. To use custom implementation specify it in the constructor.
*
* @param consumerTag The consumer tag associated with the consumer.
* @param exception Shutdown reason.
*/
private void handleShutdown(String consumerTag, ShutdownSignalException exception) {
logger.info(String.format("Consumer (%s) shutdown. Reason: %s", consumerTag, exception.getMessage()));
logger.info(exception);
}
}
更新:已解决,显然我的预取计数未设置,因此它是无限的。这就是流量被锁定在一个通道上直到队列耗尽的原因。
我运行生产者,它生成N条消息,我在仪表板上看到它们。当我运行接收器时,它会接收来自队列的所有消息,并且队列为空。 我需要有多个生产者生成消息到同一个队列。多个客户从队列中接收消息。消息将被队列TTL删除。但是现在第一个接收者从队列中获取所有消息。我怎么能做到这一点?
我有以下场景:有3个rabbitmq队列,生产者根据消息的优先级将消息推送到这些队列。(myqueue_high,myqueue_medium,myqueue_low)我希望有一个可以按顺序或优先级从这些队列中提取的单一使用者,即只要消息在那里,它就一直从高队列中提取。它是从介质中拉出来的。如果medium也是空的,它从Low拉出。 我如何实现这一点?我需要编写自定义组件吗?
我刚刚开始玩弄《Spring-Cloud-Stream》中的Kafka活页夹。 我配置了一个简单的消费者: 但当我启动应用程序时,我看到在启动日志中创建了三个独立的消费者配置: 我发现这些配置之间唯一不同的是客户机。id。 除此之外,我不知道为什么只有一个消费者有三种配置。 是因为我也在运行吗? 这是我的:
问题内容: 我有一个JMS客户端,它正在生成消息并通过JMS队列发送到其唯一的使用者。 我想要的是不止一个消费者收到这些消息。我想到的第一件事是将队列转换为主题,以便现有用户和新用户都可以订阅并将相同的消息传递给他们。 显然,这将涉及在生产者和消费者方面修改当前的客户代码。 我还要查看其他选项,例如创建第二个队列,这样就不必修改现有的使用者。我相信这种方法有很多优点,例如(如果我错了,请纠正我)在
TL;DR;我试图理解一个被分配了多个分区的单个使用者是如何处理reach分区的消费记录的。 例如: 在移动到下一个分区之前,会完全处理一个分区。 每次处理每个分区中的可用记录块。 从第一个可用分区处理一批N条记录 以循环旋转方式处理来自分区的N条记录 我找到了或分配程序的配置,但这只决定了使用者如何分配分区,而不是它如何从分配给它的分区中使用。 我开始深入研究KafkaConsumer源代码,#