Running PgBouncer on AWS ECS
PgBouncer sits between your application servers and Postgres, so the number of app connections no longer has to equal the number of database connections. On AWS ECS you can run it as a small, dedicated task and tune it to your workload. This guide walks through the container stack, task settings, networking choice, monitoring, and the PgBouncer settings that usually need attention. The figures cited below come from a published engineering write-up about one rollout; they are a reference point for reasoning about your own pool, not a universal recommendation and not a result this page claims to have reproduced.
Why put a pooler in front of Postgres
The scaling problem is arithmetic. When every web server keeps its own pool of database connections, adding servers multiplies connection counts, and Postgres or Aurora will eventually push back. Neither ships with a built-in pooler, so a dedicated layer such as PgBouncer centralizes connection management and decouples app-server count from database-connection count. It also offers pooling modes that recycle connections more efficiently than holding one per client session. Whether that helps depends on your workload, so treat the settings here as a starting configuration to measure against.
Container stack
A practical ECS task definition for PgBouncer uses three containers: the pooler itself, a Prometheus exporter for PgBouncer metrics, and Metricbeat for shipping those metrics. The pooler image in the captured setup was Edoburu's minimal PgBouncer image. It is unofficial, but it exposes PgBouncer settings through environment variables and its code is easy to follow. The other two containers exist for monitoring and metrics delivery. Keeping the exporter and shipper as sidecars means each PgBouncer task carries its own metrics path, so scaling the pooler scales observability with it.
Task settings and ulimits
The captured deployment ran on AWS Fargate with 1 vCPU and 2 GB of memory per task, and CPU usage rarely exceeded 30% including the metrics sidecars. PgBouncer is written in C and uses libuv for asynchronous I/O, which makes it efficient but not multithreaded. If you need more than one core, run multiple processes and reflect that in your configuration. Because of the async I/O model, file-descriptor limits can be hit unless you raise them in the container definition. The exact ulimit block from the captured task definition is supplied as a protected configuration module and should be applied as-is. Sizing a task around one vCPU is therefore a deliberate choice: extra cores do not help a single PgBouncer process, so scale by adding tasks rather than enlarging them.
Network Load Balancer vs ECS Service Discovery
The choice depends on where your clients live. A Network Load Balancer is the option to use when clients connect from outside an AWS VPC, but an AWS ELB incurs bandwidth charges even for traffic inside the same region. If every client and the downstream Postgres server are inside the VPC, ECS Service Discovery avoids those charges: an internal hostname resolves to the local IP of one PgBouncer task, and each DNS query returns a randomized set of A records so connections spread across tasks. In the captured setup, two PgBouncer tasks ran in two availability zones with connections distributed between them. The trade-off is operational: an NLB gives you a stable endpoint and health-checked targets, while Service Discovery relies on DNS behavior and client-side re-resolution.
Monitoring PgBouncer
PgBouncer exposes metrics through a pseudo-database that you connect to and query. To get them into a standard format, the captured setup used an open-source Prometheus exporter that connects to that database and serves Prometheus metrics over HTTP. There was no Prometheus server in that environment, so Metricbeat ingested the metrics into an Elasticsearch cluster used mainly for logs. Once metrics are flowing, a dashboard can track performance. The most useful signals were the total number of connections to the backing database, split by state, and the average and 95th percentile wait time for clients acquiring a connection. Pairing those two views is what tells you whether pressure is on the client side or the database side.
Pooling mode
Pooling mode is the setting with the largest effect on how connections are shared. Transaction pooling recycles a connection between transactions instead of holding it for a whole client session, which improves efficiency. It comes with restrictions on session-level features, so check PgBouncer's documented constraints before switching. The captured deployment uses transaction pooling. If your application depends on prepared statements, session variables, or advisory locks that span statements, verify those behaviors against the documented restrictions before adopting this mode.
Connection limits
Two limits usually need changing from the defaults: how many clients may connect to PgBouncer, and how many connections PgBouncer may open to the downstream server. The captured setup adjusted max_client_conn, default_pool_size and min_pool_size. The default max_client_conn of 100 was far too low for that workload. After tuning, roughly 1200 concurrent application connections were reduced to about 100 connections from PgBouncer to the RDS cluster. That ratio is an illustration of pool math for one workload, not a target to copy. Set max_client_conn from expected client concurrency, then size default_pool_size and min_pool_size against the database's connection budget.
TCP keepalive behind a Network Load Balancer
When PgBouncer sits behind an AWS Network Load Balancer, the load balancer's connection idle timeout can be more aggressive than PgBouncer's default keepalive behavior. The captured setup lowered tcp_keepalive so the pooler stayed compatible with the load balancer. If you use ECS Service Discovery instead, this particular mismatch does not apply in the same way, but idle-connection behavior is still worth watching. The symptom to recognize is pooled connections dropping after quiet periods, which shows up as reconnect churn rather than pool exhaustion.
Decision checklist before you deploy
Use this list to turn the captured setup into a plan for your own environment:
- Are your clients outside the VPC? If yes, plan for a Network Load Balancer and its bandwidth charges. If no, ECS Service Discovery is the lower-cost path.
- Does your application rely on session-level Postgres features? If yes, transaction pooling may not fit without changes.
- Have you raised the nofile ulimit in the PgBouncer container definition?
- Do you need more than one core? If so, plan for multiple PgBouncer processes rather than assuming threads.
- Have you set max_client_conn above the default of 100 to match expected client concurrency?
- Have you sized default_pool_size and min_pool_size against the downstream database's connection budget?
- Is there a metrics path from the exporter to whatever you use for dashboards?
- If you use an NLB, have you checked tcp_keepalive against its idle timeout?
Telling apart the common failure modes
Three symptoms look similar but point to different fixes. If clients wait a long time for a connection while downstream connection counts are low, the limit to inspect is max_client_conn or the pool's client-side capacity. If downstream connection counts sit at the pool ceiling and client wait times rise, default_pool_size is the constraint and the database may also be saturated. If connections drop after idle periods behind an NLB, the keepalive setting is the likely cause rather than pool sizing. Watching both the client wait-time percentiles and the downstream connection states together is what makes these distinguishable.
Settings and task fields at a glance
The table below maps the settings discussed above to what they control and when to revisit them. It is a reading aid, not a substitute for PgBouncer's own documentation.
A worked example of pool math
Suppose your app tier can open 1,200 concurrent connections and your database budget is around 100. With transaction pooling and a default_pool_size near 100, PgBouncer can multiplex the 1,200 clients onto roughly 100 backend connections, provided the app does not depend on session state. The captured deployment reached that shape after raising max_client_conn above the default and setting default_pool_size and min_pool_size deliberately. Your own ratio will differ with query duration and concurrency; measure wait-time percentiles before and after the change rather than assuming the same numbers apply.
Alternatives worth considering
If your Postgres runs on AWS RDS, AWS RDS Proxy is an alternative pooler. The published write-up says the team did not choose it because it was new at the time and did not work with their Heroku setup, where the proxy could only be reached from inside a VPC. That is a dated constraint from the captured post, not a current statement about RDS Proxy; check current AWS documentation before ruling it in or out. For a deeper explanation of how PgBouncer works, a talk by one of the project maintainers is a useful starting point.
| Setting or field | What it controls | When to revisit |
|---|---|---|
| max_client_conn | How many clients may connect to PgBouncer | Client wait times rise while downstream connections stay low |
| default_pool_size | Limit on connections PgBouncer opens to the downstream server | Downstream connections sit at the ceiling |
| min_pool_size | Minimum number of downstream connections PgBouncer maintains | You want warm connections available at low load |
| tcp_keepalive | TCP keepalive interval for pooled connections | Connections drop after idle periods behind an NLB |
| pool_mode (transaction) | Recycles connections between transactions rather than per session | The app relies on session-level Postgres features |
| nofile ulimit | File-descriptor limit for the PgBouncer container | You see file-descriptor errors under load |
Values reflect one captured production rollout and are illustrative; confirm against current PgBouncer documentation and your own workload.
Frequently asked questions
Why use PgBouncer instead of connecting each app server directly to Postgres?
Direct connections scale with the number of app servers, which can push the database toward its connection ceiling. A pooler centralizes connections so app servers can be added without a proportional rise in database connections, and pooling modes let connections be reused more efficiently. The benefit depends on your workload.
Should I use a Network Load Balancer or ECS Service Discovery in front of PgBouncer?
Use a Network Load Balancer if clients connect from outside the VPC, accepting that ELB bandwidth charges apply even within the same region. If all clients and the Postgres server are inside the VPC, ECS Service Discovery avoids those charges and still spreads connections across tasks through randomized DNS answers.
What are the trade-offs of transaction pooling?
Transaction pooling recycles a connection between transactions rather than holding it for a whole session, which improves efficiency. It restricts session-level features, so check PgBouncer's documented constraints before switching.
Why does PgBouncer on ECS need a raised nofile ulimit?
PgBouncer uses libuv for asynchronous I/O, so it can hit file-descriptor limits under load. Raising the nofile soft and hard limits in the container definition prevents that failure mode.
Why lower tcp_keepalive when using a Network Load Balancer?
The load balancer's connection idle timeout can be more aggressive than PgBouncer's default keepalive behavior, which can drop idle pooled connections. Lowering tcp_keepalive keeps the pooler compatible with the load balancer.