当前位置: 首页 > 知识库问答 >
问题:

如何配置RabbitMQ以平等地为多个消费者提供多个队列

万开畅
2023-03-14

我有一个RabbitMQ代理,它设置了多个队列。在客户端(Java ),我有多个消费者,他们都像这样监听他们的队列:

队列_1-

它们都使用一个连接但不同的通道。发生的情况是,当我加载所有队列并启动应用程序代理服务时,首先服务一个队列,而不是另一个队列,依此类推。因此,消息一次由各自的消费者一个队列接收。我还想提一下,我正在使用预取计数1来实现消费者流量的公平分配。

我怎样才能让它发生,让所有的队列得到平等的服务。

编辑:下面是创建消费者的代码(非常基本)

import com.rabbitmq.client.*;

import org.apache.log4j.Logger;

import java.io.IOException;
import java.util.UUID;
import java.util.concurrent.TimeoutException;

/**
 * Used for consuming and acknowledging messages from defined queue.
 *
 */
public class Consumer {
    private final static Logger logger = Logger.getLogger(Consumer.class);
    // Maximum number of messages that can be on the consumer at a time
    private static int prefetchCount = 1;

    // Internal enum which contains queue names and their exchange keys
    private Queue queue;
    private Channel channel;
    private String consumerTag;
    private String uuid = UUID.randomUUID().toString();
    private boolean subscribed = false;
    private DeliverCallback deliverCallback = this::handleDeliver;
    private CancelCallback cancelCallback = this::handleCancel;
    private ConsumerShutdownSignalCallback consumerShutdownSignalCallback = this::handleShutdown;

    /**
     * The constructors sets the channel to RabbitMQ broker for the specified queue.
     * Callback for events are set to their default implementation.
     *
     * @param queue RabbitMQ queue - this consumer will be assigned to this queue and will only be able to consume from it.
     * @see #setDeliverCallback(DeliverCallback)
     * @see #setCancelCallback(CancelCallback)
     * @see #setConsumerShutdownSignalCallback(ConsumerShutdownSignalCallback)
     */
    public Consumer(Queue queue) {
        this.queue = queue;

        try {
            setUpChannel();

        } catch (IOException e) {
            e.printStackTrace();

        }
    }

    public Class getEntityClass() {
        return Queue.getEntityClassForQueue(queue);
    }

    public String getUuid() {
        return uuid;
    }

    public boolean isSubscribed() {
        return subscribed;
    }

    public DeliverCallback getDeliverCallback() {
        return deliverCallback;
    }

    public void setDeliverCallback(DeliverCallback deliverCallback) {
        this.deliverCallback = deliverCallback;
    }

    public CancelCallback getCancelCallback() {
        return cancelCallback;
    }

    public void setCancelCallback(CancelCallback cancelCallback) {
        this.cancelCallback = cancelCallback;
    }

    public ConsumerShutdownSignalCallback getConsumerShutdownSignalCallback() {
        return consumerShutdownSignalCallback;
    }

    public void setConsumerShutdownSignalCallback(ConsumerShutdownSignalCallback consumerShutdownSignalCallback) {
        this.consumerShutdownSignalCallback = consumerShutdownSignalCallback;
    }


    /**
     * <p>
     * Subscribes to the set queue. The subscription can be cancelled using
     * Checks if the queue is set up properly.
     * </p>
     * <p>
     * Note: this is a non-blocking operation. The client will listen for incoming messages and handle them using
     * the provided DeliveryCallback function but the execution of this operation will be on another thread.
     * </p>
     *
     * @throws IOException if I/O problem is encountered.
     */
    public void subscribeToQueue() throws IOException {
        if (channel != null) {
            consumerTag = channel.basicConsume(
                    queue.getQueueName(),
                    deliverCallback,
                    cancelCallback,
                    consumerShutdownSignalCallback
            );
            subscribed = true;

        } else {
            logger.error("Channel does not exist. Unable to consume message.");

        }
    }

    /**
     * Confirms the message has been successfully processed.
     *
     * @param deliveryTag Unique message tag generated by the server.
     * @throws IOException if I/O problem is encountered.
     */
    public void acknowledgeMessageReceived(long deliveryTag) throws IOException {
        if (channel != null) {
            channel.basicAck(deliveryTag, false);

        } else {
            logger.error("Channel does not exist. Unable to acknowledge message delivery.");

        }
    }

    /**
     * Sends a negative acknowledgement to RabbitMQ without re-queueing the message.
     *
     * @param deliveryTag Unique message tag generated by the server.
     * @throws IOException if I/O problem is encountered.
     */
    public void rejectMessage(long deliveryTag) throws IOException {
        if (channel != null) {
            channel.basicReject(deliveryTag, false);

        } else {
            logger.error("Channel does not exist. Unable to reject message delivery.");

        }
    }

    /**
     * Cancels consumer subscription to the queue.
     * The consumer can be used for acknowledging messages, but will not receive new messages.
     * This does not close the underlying channel. To close the channel use closeChannel() method.
     *
     * @throws IOException
     * @see #subscribeToQueue()
     * @see #closeChannel()
     */
    public void cancelSubscription() throws IOException {
        if (channel != null) {
            channel.basicCancel(this.consumerTag);
            subscribed = false;

        } else {
            logger.error("Channel does not exist. Unable to cancel consumer subscription.");
        }
    }

    /**
     * Explicitly closes channel to the queue.
     * After doing this you will not be able to use any of the methods of this class.
     *
     * @throws IOException      if I/O problem is encountered.
     * @throws TimeoutException if connection problem occurs.
     */
    public void closeChannel() throws IOException, TimeoutException {
        if (channel != null) {
            channel.close();
            channel = null;
            logger.info("Closing RabbitMQ consumer channel...");

        } else {
            logger.error("Channel already closed.");

        }
    }

    /**
     * Checks if the queue exists and creates the channel.
     * If the queue does not exist channel is set to null and cannot be used.
     *
     * @throws IOException if I/O problem is encountered.
     */
    private void setUpChannel() throws IOException {

        channel = ChannelFactory.getInstance().createChannel();
        try {
            channel.queueDeclarePassive(queue.getQueueName());
            channel.basicQos(prefetchCount);

        } catch (IOException e) {
            // When this exception occurs it renders the channel unusable so it's best set to null.
            channel = null;

            logger.error(String.format("Queue %s does not exist [%s]", queue.getQueueName(), e.getMessage()));
            e.printStackTrace();

        }
        logger.info("Setting up RabbitMQ consumer channel. Channel successfully initialized: " + (channel != null));
    }

    /**
     * Callback called when a message is delivered to the client.
     * Default implementation. Callback acknowledges message received and does nothing with it.
     * To use custom implementation use setDeliverCallback method.
     *
     * @param consumerTag The consumer tag associated with the consumer.
     * @param message     Message object.
     * @see #setDeliverCallback(DeliverCallback)
     */
    private void handleDeliver(String consumerTag, Delivery message) {
        Envelope envelope = message.getEnvelope();
        long deliveryTag = envelope.getDeliveryTag();

        logger.info("Message delivered: " + deliveryTag);

        try {
            channel.basicAck(deliveryTag, false);

        } catch (IOException e) {
            e.printStackTrace();

        }
    }

    /**
     * Callback called when a service is cancelled.
     * Default implementation. To use custom implementation specify it in the constructor.
     *
     * @param consumerTag The consumer tag associated with the consumer.
     */
    private void handleCancel(String consumerTag) {
        logger.info("Consumer (" + consumerTag + ") cancelled: ");
    }

    /**
     * Called when the consumer is abruptly shutdown due to termination of the underlying connection or channel.
     * Default implementation. To use custom implementation specify it in the constructor.
     *
     * @param consumerTag The consumer tag associated with the consumer.
     * @param exception   Shutdown reason.
     */
    private void handleShutdown(String consumerTag, ShutdownSignalException exception) {
        logger.info(String.format("Consumer (%s) shutdown. Reason: %s", consumerTag, exception.getMessage()));
        logger.info(exception);
    }
}

共有1个答案

章光华
2023-03-14

更新:已解决,显然我的预取计数未设置,因此它是无限的。这就是流量被锁定在一个通道上直到队列耗尽的原因。

 类似资料:
  • 我运行生产者,它生成N条消息,我在仪表板上看到它们。当我运行接收器时,它会接收来自队列的所有消息,并且队列为空。 我需要有多个生产者生成消息到同一个队列。多个客户从队列中接收消息。消息将被队列TTL删除。但是现在第一个接收者从队列中获取所有消息。我怎么能做到这一点?

  • 我有以下场景:有3个rabbitmq队列,生产者根据消息的优先级将消息推送到这些队列。(myqueue_high,myqueue_medium,myqueue_low)我希望有一个可以按顺序或优先级从这些队列中提取的单一使用者,即只要消息在那里,它就一直从高队列中提取。它是从介质中拉出来的。如果medium也是空的,它从Low拉出。 我如何实现这一点?我需要编写自定义组件吗?

  • 我刚刚开始玩弄《Spring-Cloud-Stream》中的Kafka活页夹。 我配置了一个简单的消费者: 但当我启动应用程序时,我看到在启动日志中创建了三个独立的消费者配置: 我发现这些配置之间唯一不同的是客户机。id。 除此之外,我不知道为什么只有一个消费者有三种配置。 是因为我也在运行吗? 这是我的:

  • 问题内容: 我有一个JMS客户端,它正在生成消息并通过JMS队列发送到其唯一的使用者。 我想要的是不止一个消费者收到这些消息。我想到的第一件事是将队列转换为主题,以便现有用户和新用户都可以订阅并将相同的消息传递给他们。 显然,这将涉及在生产者和消费者方面修改当前的客户代码。 我还要查看其他选项,例如创建第二个队列,这样就不必修改现有的使用者。我相信这种方法有很多优点,例如(如果我错了,请纠正我)在

  • TL;DR;我试图理解一个被分配了多个分区的单个使用者是如何处理reach分区的消费记录的。 例如: 在移动到下一个分区之前,会完全处理一个分区。 每次处理每个分区中的可用记录块。 从第一个可用分区处理一批N条记录 以循环旋转方式处理来自分区的N条记录 我找到了或分配程序的配置,但这只决定了使用者如何分配分区,而不是它如何从分配给它的分区中使用。 我开始深入研究KafkaConsumer源代码,#