Azure-sdk-for-js: [Service Bus] ServiceBusMessage to expose all AMQP details

[HarshaNalluru]

_Motivation for this change:_

A desire to allow reading and writing of the different parts of an AMQP message as described by the AMQP spec. This gives us the interopability feature when one uses a different client that does support writing to these parts, but then uses our libraries to read the message. See "Message Format" in section 3.2 of the AMQP spec

_Guiding factors for the proposal:_

• Existing users used to accessing either AMQP properties like messageId or AMQP message annotations like partitionKey or AMQP message header like deliveryCount should continue to have these as top level entities
• All inner details of AMQP message should be accessible to the advanced user. Since these do not matter to the most of the user segment, these details need not be at the top level. Therefore, we would like to hide the details (header, footer, message annotations, deliver annotations, properties) in a nested field. This model can be replicated in Event Hubs in the future
• We do not want to introduce any new AMQP message detail as a top level entity at this moment in time to avoid increasing the surface area and potential confusion due to introduction of new elements. If the service team determines that an AMQP message detail from header/footer/message annotations/delivery annotations deserves to get a spot on the top level, it should be do-able incrementally in a non breaking fashion

[ramya-rao-a]

Proposal 1: Add AMQPMessageDetails to both sendable message and the received message

// What is new? The `amqpMessageDetails` field
export interface ServiceBusMessage {
  body: any;
  contentType?: string;
  correlationId?: string | number | Buffer;
  subject?: string;
  messageId?: string | number | Buffer;
  partitionKey?: string;
  applicationProperties?: { [key: string]: any };
  replyTo?: string;
  replyToSessionId?: string;
  scheduledEnqueueTimeUtc?: Date;
  sessionId?: string;
  timeToLive?: number;
  to?: string;
  viaPartitionKey?: string;
  amqpMessageDetails: AMQPMessageDetails;
}

export interface AMQPMessageDetails {
  header?: AMQPMessageHeader;
  footer?: { [key: string]: any };
  messageAnnotations?: { [key: string]: any };
  deliveryAnnotations?: { [key: string]: any };
  properties: AMQPMessageProperties;
}

Proposal 2

• AMQPAnnotedMessage is the representation of the AMQP message as per the spec
• ServiceBusMessage is the "convenience layer" over the AMQPAnnotatedMessage pulling limited things to the top level that are familiar to SB users and ignoring other details
• Both ServiceBusMessage and AMQPAnnotedMessage are sendable messages. Most users will use the former, advanced users familiar with AMQP and wanting to tweak inner details can use the latter
• Received message will expose the AMQPAnnotatedMessage via a new field
• When wanting to resend a ReceivedMessage, one can send the entire message or the inner AMQPAnnotatedMessage

// What is new? nothing
export interface ServiceBusMessage {
  body: any;
  contentType?: string;
  correlationId?: string | number | Buffer;
  subject?: string;
  messageId?: string | number | Buffer;
  partitionKey?: string;
  applicationProperties?: { [key: string]: any };
  replyTo?: string;
  replyToSessionId?: string;
  scheduledEnqueueTimeUtc?: Date;
  sessionId?: string;
  timeToLive?: number;
  to?: string;
  viaPartitionKey?: string;
}

export interface AmqpAnnotatedMessage {
  header?: AMQPMessageHeader;
  footer?: { [key: string]: any };
  messageAnnotations?: { [key: string]: any };
  deliveryAnnotations?: { [key: string]: any };
  applicationProperties?: { [key: string]: any };
  properties?: AMQPMessageProperties;
  bodyType: "data" | "value" | "sequence",
  body: any;
}

// What is new? The field `amqpAnnotatedMessage`
export interface ReceivedMessage extends ServiceBusMessage {
  deadLetterReason?: string;
  deadLetterErrorDescription?: string;
  lockToken?: string;
  deliveryCount?: number;
  enqueuedTimeUtc?: Date;
  expiresAtUtc?: Date;
  lockedUntilUtc?: Date;
  enqueuedSequenceNumber?: number;
  sequenceNumber?: Long;
  deadLetterSource?: string;
  amqpAnnotatedMessage: AmqpAnnotatedMessage;
}


sendMessages(msgs: ServiceBusMessage | ServiceBusMessage[] | ServiceBusMessageBatch | AmqpAnnotatedMessage | AmqpAnnotatedMessage[], options)
export interface ServiceBusMessageBatch {
   (all the usual stuff)
   tryAdd(msg: ServiceBusMessage | AmqpAnnotatedMessage): boolean;
}

Interfaces common to both proposals:

export interface AMQPMessageHeader {
  firstAcquirer?: boolean;
  deliveryCount?: number;
  timeToLive?: number;
  durable?: boolean;
  priority?: number;
}

export interface AMQPMessageProperties {
  messageId?: string | number | Buffer;
  to?: string;
  correlationId?: string | number | Buffer;
  contentType?: string;
  contentEncoding?: string;
  absoluteExpiryTime?: Date;
  creationTime?: number;
  groupId?: string;
  groupSequence?: number;
  replyTo?: string;
  replyToGroupId?: string;
  subject?: string;
  userId?: string;
}

[HarshaNalluru]

Resending a received message

With proposal 1, there are attributes with the same value at multiple places.

export interface ServiceBusMessage {
 ... 
 amqpMessageDetails: AmqpAnnotatedMessage;
}

Consider the case while sending a receivedMessage using the sendMessages API, user might tweak the top level "body" property or the one on the amqpMessageDetails. Even if we set the precedence, it might be confusing for the users.

This is not a problem with proposal 2.

• the ServiceBusMessage and AmqpAnnotatedMessage are sendable messages.
• amqpMessageDetails on the received message is ignored, we only care about the fields on the ServiceBusMessage.
  send(receivedMessage);
• "advanced" users can modify and send receivedMessage.amqpMessageDetails using the same send API.
  send(receivedMessage.amqpMessageDetails);

Only other concern with a different first class AMQPMessage, is that we are implicitly assuming that the "advanced" users would know that partition key is meant to be set/read in the message annotations and that session id is actually groupid. We need to add proper docs for the props on AMQPMessageDetails for the same.

[hemanttanwar]

Proposal 1: Consider the case while sending a receivedMessage using the sendMessages API, user might tweak the top level "body" property or the one on the amqpMessageDetails. Even if we set the precedence, it might be confusing for the users.

Isn't we discussed that there is always one variable/source for a property 'value' and that gets changed whether user change at top level or in amqpMessageDetails ? User can change a value at any level and it will update one source.

95% (Whatever that number is) : A user will stay at top level and change values if needed. And Advance users can change any where but it will update same one variable for that property.

Proposal 2 : Like Connie mentioned, it will multiply overloads by two.

[ramya-rao-a]

Proposal 2 : Like Connie mentioned, it will multiply overloads by two.

Yes, in .Net & Java that would be the case. But Python & JS can use union of types

[HarshaNalluru]

Isn't we discussed that there is always one variable/source for a property 'value' and that gets changed whether user change at top level or in amqpMessageDetails ? User can change a value at any level and it will update one source.

@hemanttanwar The problem is.. we can't stop a user from constructing a message object with two different "body"s in JS, which is not a problem in Java or .NET.

[bterlson]

I'm a fan of approach 2, personally. Although, since ReceivedMessage is a subtype of ServiceBusMessage, it would be nice if their names are related similarly. ReceivedServiceBusMessage seems pedantically correct but maybe too long. Open to other suggestions, or keeping the current names if no alternatives are palatable.

[JoshLove-msft]

I'm a fan of approach 2, personally. Although, since ReceivedMessage is a subtype of ServiceBusMessage, it would be nice if their names are related similarly. ReceivedServiceBusMessage seems pedantically correct but maybe too long. Open to other suggestions, or keeping the current names if no alternatives are palatable.

We call it ServiceBusReceivedMessage in .NET.

[ramya-rao-a]

In #10957, we added a new field to hold all the inner AMQP details of the message when it is received. This allows someone using another client library to set such inner details to then use our libraries to receive the message and read these details.

Logged #11102 to track the need to have similar changes on the sending side.