Budgeting for Azure Service Bus

nservicebus
azureservicebus

#1

I’m looking for any basic info on typical usage cost of Azure Service Bus connection. I realize a lot will go into that regarding design, but I’m looking for defaults.

How much azure cost will NServiceBus transport run up when connected with default settings and processing NO messages. This will be a constant usage on.

Specifically I’m looking to see how many azure operations the default settings at idle will use.


(Sean Feldman) #2

Hi @robm,

Assuming you’re talking about Standard Tier, you’ll have a base charge of $10 for all of your namespaces.
Next step is to look at what topology will be used. For Forwarding topology only a single queue is polled. For EndpointOriented topology, input queue and all subscriptions are polled. With Forwarding topology there’s a cost of auto-forwarding from subscriptions to input queue, but I’ll omit that as you’re referring to a scenario with no messages.

If there are no messages, your cost will be limited to number of connections and queues polls. Note that ASB service Standard tier has first 13M ops/m free. After that you pay based on number of operations.

ASB operations and connections costs are:

Brokered connections (Number of AMQP connections or HTTP calls to Service Bus.)

First 1K/month Included
Next 99K (1K–100K)/month $0.03 per connection/month
Next 400K (100K–500K)/month $0.025 per connection/month
Over 500K/month $0.015 per connection/month

Messaging operations (An operation is any API call to the Service Bus service)

First 13M ops/month Included
Next 87M ops (13M–100M ops)/month $0.80 per million operations
Next 2,400M ops (100M–2,500M ops)/month $0.50 per million operations
Over 2,500M ops/month $0.20 per million operations

Number of operations (pollings) in this case will depend on number of connections you’ll have opened (controlled by number of messaging factories) and number of clients per entity you’re using. In case you’re planning to use multiple namespace (for HA or DR), you’ll have to take that into consideration as well.

Calculation is very different if plan to use Premium Tier with $22.26 fixed rate per message unit (per day).


#3

Thanks for the info… to dial it in a bit more…

With an idle always on endpoint topology of a single client in a single queue… what would the expected “Messaging operations” total be?


(Sean Feldman) #4

The transport is using ASB client that has no source code available.
I’d suggest to raise that as an question issue in Microsoft ASB old client repo to get the exact information.

Looking at the new ASB client, it has a wait time of one minute before client pulls for new message(s) again.


#5

here is some info…
A single client running this config…

            var endpointConfiguration = new EndpointConfiguration(_endpointName);
            endpointConfiguration.UseSerialization<NServiceBus.JsonSerializer>();

            endpointConfiguration.SendFailedMessagesTo("error");

            endpointConfiguration.AuditProcessedMessagesTo("audit");
            endpointConfiguration.DefineCriticalErrorAction(OnCriticalError);
            
            endpointConfiguration.UsePersistence<InMemoryPersistence>();
            endpointConfiguration.EnableInstallers();

            endpointConfiguration.UseSerialization<NServiceBus.JsonSerializer>();
            endpointConfiguration.UsePersistence<InMemoryPersistence>();

            var transport = endpointConfiguration.UseTransport<AzureServiceBusTransport>();
            transport.ConnectionString(connectionString);
            transport.UseForwardingTopology();

Produces 6.5 Thousand requests just sitting there doing nothing for about 12 hours. Meaning my solution, just sitting there, before a single message is sent is going to cost $300. Good to know at least.

asbrequests


(Brandon Ording) #6

Which version of NServiceBus are you testing with? If you’re using NSB 6, then you should also disable the legacy retries satellite queue if you aren’t upgrading an endpoint that was previously running NSB 5.

Without that API call, you’ve got two queues being consumed per endpoint instead of one, which would be making the request count much higher than it needs to be.


#7

NSBCore 6.x
NSBAzure 7.x

Ah HA!
For some reason it didn’t occur to me that the number of clients above 1 on an endpoint that is idle are not idle! So my chart above in this case is with 4 cores. This means my server side things that might need more could use them, but for my slower endpoints on the devices I could scale clients back without performance issue, and it would scale the operations.


(Sean Feldman) #8

Hi @robm,

I’m not quite following what you wanted to say in the last.
If you look at the initial comment I’ve pointed you to the transport documentation. Specifically, factories and clients. Concurrency plays a significant role as well and can be configured if needed. The snippet you’ve shared is using all the defaults. So for a 4 cores CPU concurrency would be max(Number of logical processors,2), eight.

Looking at the calculation, I’m not sure how you’d be paying $300 dollars. Assuming that with the given configuration that has 6.5K operations per 12 hours, it would be 13K operations per day, or 390K operations per month. Given you have 13 million free operations per month and low message flow, I’d imagine your bill would be substantially lower than $300. Even when you process messages, on average a message would be 3 billable operations. Something doesn’t add up. Unless you’re running a very big number of endpoints / scaled-out instances.

Also, like @bording pointed out, you should disable the legacy satellite. That with tuned performance to match your need would lead to a substantially lower bill.

And last, but not the list, if you find your solution indeed requiring such a significant resource allocation that your bill is getting over $300+, perhaps it’s worth looking at Premium tier. I’m not convinced it’s the case, but once you see your spend and it proves to be as you estimated, for ~$600/m you’ll have designated resources and no pay-per-operation.


(Sean Feldman) #9

More follow up.

Calculation you have going is based on assumption that any “incoming request” metric directly translates into “billable messaging operation”. That doesn’t seem to be the case.

I’ve looked into Azure Cost Management and Billing. Unfortunately, it doesn’t allow to isolate ASB cost of what you’ve already used. I assume you have more resources than just ASB namespaces to figure out the cost.

Saying that, there’s an Azure calculator to help you with estimates. Have you tried it? I’ve tried to see what an estimate would look like with the number of incoming requests as billable operations (event though it’s not correct to do as billable operations will be much lower) set to 1 million (roughly 2.5 times of what you’ve logged) and it was still $10 USD/m. After 14 million you start inuring some costs. And even then, for a million operations you’d be paying around 80c.

Hope this information helps.
Sean


#10

You were right above, I meant my test deployment scaled out. I’ve already racked up $300 in two weeks without a single message (based on # scaled). 200+ million.

That aside, it seems clear from what I’ve seen that incoming requests is 1:1 for what they consider billable operations… “For brokered entities (queues and topics or subscriptions), an operation is any API interaction with Service Bus service on any protocol.”

Using my MSDN account with free $150, I hit my usage cap around 250Million “Messaging operations” the “Incoming requests” metric was the same number as the operations on the bill.


(Sean Feldman) #11

With scale, w/o a single message going through all the credits sounds extreme.
May I suggest to get on a call?
Feel free to ping me at sean.feldman@particular.net


#12

Here is a snip of last months bill when I blew by my credit the first time. I’ll note that even with the spending cap at $150, it let me run over.
This was with the config posted above on several hundred 4 core systems, still without actually processing a single nsb message :slight_smile:

Also, ya, I’ve been assuming I’m in the market for premium, but i’ll have to come up with a strategy for name spaces and deployment since I was counting on more than 1000 endpoints.


(Sean Feldman) #13

Ok, then it’s a different story. With several hundreds of 4 core/8 logical processors deployment targetting 1K endpoints you definitely want to have concurrency and ASB settings under control based on number of messages you have. Premuim might be actually cheaper in your scenario. And more predictable.

W/o knowing your architecture, hard to understand what and how you deploy and if 1K endpoints is necessary or not. You’ve got my contact info if needed. Also, you can also raise a support issue if it comes to the point where you need it.

Sean


#14

Awesome thanks for the chat and info. At this point I’m more in the stage of just wanting to make sure I was understanding correctly.

I actually have two unrelated solutions that need implemented, one send only from thousands of devices to a single queue (or a small handful of them really) and a separate with the same several thousands of devices two way. Premium made sense until I saw the 1000 queue limit, but more importantly maybe the 1000 brokered connections at once. When I originally (naively) assumed that it wouldn’t cause a connection until reaching out with a message, it was a no brainer. Then I ran the above example and got concerned :wink:

Thanks again


#15

Latest Update:

Thanks again for the help and ideas.
I’ve got my solution running and processing millions of messages with far fewer total operations. Well within the expected budget now! Running on 2700 clients.


(Sean Feldman) #16

Hi @robm,

Thanks for the update. Glad to hear that our discussion was productive and resolved excessive billing.
I you’d be interested in a quick chat off this topic to talk about what are you doing with NServiceBus on Azure (if you won’t mind), I’ll be more than happy to do so. Ping me directly.

Sean