Improve integration documentation

[adenhertog]

#56 indicates that the documentation is lacking around how to integrate all the components into a solution, especially around how to create a message-driven set of distributed services.

NServiceBus has a set of examples/tutorials that demonstrate some messaging core concepts and how to implement them with that library, which could be a useful reference to base this documentation off.

[Zehelein]

Sorry to add this here: is there some forum/discord/... to ask questions? Like for RabbitMQ transport, I see that a dead-letter queue is created and associated. But I could not figure out what would trigger a message to be moved here like some maximum number of retries or anything.

For a test I adjusted the rabbitmq-transport.ts with the following code to send a message to the dead letter queue after the first retry which would do that:

  async returnMessage(
    message: TransportMessage<RabbitMqMessage>,
  ): Promise<void> {
    if (message.raw.fields.redelivered) {
      // Message retries exhausted, send to DLQ
      this.logger.info(
        'Message retry limit exceeded, sending to dead letter queue',
        { rawMessage: message.raw },
      );
      this.channel.nack(message.raw, undefined, false);
    } else {
      this.logger.debug('Returning message', { rawMessage: message.raw });
      this.channel.nack(message.raw);
    }
  }

But are there any other configurations on the RabbitMQ setup level maybe to configure something?

[adenhertog]

Hey @Zehelein, I tried this today and confirmed there is no DLQ routing using RabbitMQ. I've just added this in now and have released it in @node-ts/bus-rabbitmq:0.5.0 so if you upgrade your local package you should see this failing dead letters after the default 10 attempts.

The number of attempts is configurable as per RabbitMqTransportConfiguration.maxRetries (https://node-ts.github.io/bus/packages/bus-rabbitmq/#configuration-options), and I've set up a quick demo of it working at node-ts/bus-starter#18 which is based of your other example.

Let me know how it goes, and thanks for letting me know!

[Zehelein]

Hi, thanks for looking so fast into it and providing a fix! But I am not sure - I fear this fix is not working. I also checked this message.raw.fields.deliveryTag but when I debugged/logged this tag then it would continue to increment. Meaning if I started with one message it was "1", and then the next failed had "2". It was not counting per message but per channel according to https://www.rabbitmq.com/confirms.html

This article seems to provide a few solutions - the ones with a bit of a delay in between would likely be nicer for transient errors.

[adenhertog]

I think you're right, that delivery tag is what you describe. I had more of a read into how rabbit/amqp deals with retries and they all seem to defer this to the client (ie: they don't track retries themselves). There has been ongoing work in rabbit to support this, but there's only been a small amount of progress in the last couple of years and looks like it's currently just a feature for quorum queues.

Adding this to rabbitmq-transport.ts means that a header should be incremented ("x-redelivery-count") on each receive, but this isn't possible without ack the original message and publish a new message with the amendments. It feels quite hacky, and sends the message to the back of the queue (this can be good or bad depending on the use case).

I'm working on this implementation on https://github.com/node-ts/bus/compare/rabbitmq-message-receives but this seems to break the DLQ routing.

[Zehelein]

Sorry I totally missed to provide the link to the article I was referring to:
https://engineering.nanit.com/rabbitmq-retries-the-full-story-ca4cc6c5b493

Maybe option 2 or 3 would be doable? Option 4 would depend on a plugin - not sure if that would be available in every setup.