Limits & Scaling 📈
Hotsock is designed to scale infinitely. Whether you have 10, 100K, or 1M simultaneous connections, one or more installations can accommodate your needs. On the road to 100K concurrent connections though, there are a few things you may need to know.
You probably don't need to consider any of this until you're hitting tens of thousands of concurrent connections or hundreds of millions of messages per month.
DynamoDB​
DynamoDB Provisioned Billing​
Hotsock uses DynamoDB for its database. By default, DynamoDB's billing settings are configured in on-demand mode which bills entirely based on usage and automatically scales the underlying hardware to accommodate any scale.
As Hotsock database table usage increases, it typically also looks less spikey. A consistent and predictably high usage pattern can make it a candidate for provisioned capacity.
Enabling provisioned mode can be a meaningful cost savings measure at scale, but make sure you provision enough capacity to avoid request throttling!
Lambda​
Lambda Account Concurrency​
AWS accounts default to allowing 1,000 concurrent executions in each region. As your Hotsock usage increases, the overall quantity and concurrency of Lambda invocations in your account will increase as well. Learn more about Lambda quotas. You can and should monitor or set an alarm on the AWS/Lambda - ConcurrentExecutions CloudWatch metric to get a warning ahead of needing your account-level limit increased.
Brand new AWS accounts have a lower, undocumented concurrency limit that is automatically raised based on some combination usage and passage of time. If you created a new account for Hotsock and are being throttled or see that your concurrency Service Quota is too low, request a quota increase.
Like most account-level limits, AWS is willing to raise this limit upon request, but don't wait to request a limit increase until after you've hit the ceiling - approval can take a few days!
Lambda Provisioned Concurrency​
Lambda Provisioned Concurrency is not currently supported by Hotsock. Provisioned Concurrency requires the use of Lambda function versions, which unfortunately have stability/consistency issues when configuring with CloudFormation.
Thankfully, Hotsock functions do not suffer from notable cold-start performance issues, so you'd likely see little benefit to enabling Provisioned Concurrency for that purpose anyway.
API Gateway​
Requests Per Second​
AWS accounts have a default quota of 10,000 requests per second in each region across all API Gateway APIs in the region.
This quota is typically only hit unexpectedly when sharing a single AWS account with multiple services. When it does get hit, it may not be particularly obvious what caused it. You can monitor the sum of the AWS/ApiGateway - Count and AWS/ApiGateway - MessageCount CloudWatch metrics to track overall usage across APIs, but the metric is provided at 1-minute resolution - per-second granularity is not available.
AWS Support is hesitant to raise this quota upon request. You'll need to provide detailed use case information to get it raised and the back-and-forth can take a few days. Get it raised before you need it!
New Connections Per Second​
The AWS API Gateway service has no limit on concurrent connections, but does have a soft limit of 500 for the number of new WebSocket connections that can be initiated per second.
There's a AWS/ApiGateway - ConnectCount CloudWatch metric you can follow, but similar to the requests per second quota, per-second granularity is not available so hitting this quota can be tough to diagnose.
SQS​
SQS Requests Per Second​
Hotsock uses a high-throughput FIFO queue for all incoming messages before they are processed by Lambda and published to one or more WebSocket connections.
This queue can handle up to 70,000 incoming messages per second in the US East (N. Virginia), US East (Ohio), US West (Oregon), Europe (Frankfurt) and Europe (Ireland) regions with lower limits in other regions.
If you're nearing or hitting this limit, it may be time to consider splitting your application tenancy across multiple installations.
Multiple Installations​
If your application requires it, you can always create multiple Hotsock installations across multiple AWS accounts and/or AWS regions to accommodate for immense scaling requirements or tenant boundaries.
Starting with multiple installations for scaling reasons is likely a premature optimization with unnecessary overhead, but there's nothing wrong with an approach that shards tenants across multiple installations! Automating per-tenant installations is no problem either, since everything is configured from a single CloudFormation template.