Spring Boot - Limit on number of connections creat

Question

I developed a microservice using Spring Boot. I was performance testing the service by stubbing the backend calls. When I looked at the thread count , I see that the maximum number of threads that created to the service is 20 at any point in time even though the number of calls made is much higher. Are there any limitations with respect to number of calls that can be made to a microservice developed using Spring Boot. Please can you guide in what steps I need to follow to troubleshoot / increase the number of connections accepted by the service?

Answer 1

Tomcat - maxThreads vs maxConnections

Try to set maxConnections property to be more than 10000.

From the doc:

The maximum number of connections that the server will accept and process at any given time. When this number has been reached, the server will accept, but not process, one further connection. This additional connection be blocked until the number of connections being processed falls below maxConnections at which point the server will start accepting and processing new connections again. Note that once the limit has been reached, the operating system may still accept connections based on the acceptCount setting. The default value varies by connector type. For BIO the default is the value of maxThreads unless an Executor is used in which case the default will be the value of maxThreads from the executor. For NIO the default is 10000. For APR/native, the default is 8192.

Note that for APR/native on Windows, the configured value will be reduced to the highest multiple of 1024 that is less than or equal to maxConnections. This is done for performance reasons. If set to a value of -1, the maxConnections feature is disabled and connections are not counted.

There is a properties for spring boot, tomcat max-connection, which needs to be set in application.properties file

server.tomcat.max-connections= # Maximum number of connections that the server will accept and process at any given time.

More info visit here , here and here

Answer 2

This setting is derived from the embedded container (tomcat, jetty...).

Tomcat's number of threads

You may specify this property in your application.properties

server.tomcat.max-threads=400

You say you counted 20 threads, however according to this other stackoverflow question/answer, the default number of thread should be 200 with tomcat, since server.tomcat.max-threads's default value is 0. See tomcat's documentation:

The maximum number of request processing threads to be created by this Connector, which therefore determines the maximum number of simultaneous requests that can be handled. If not specified, this attribute is set to 200. If an executor is associated with this connector, this attribute is ignored as the connector will execute tasks using the executor rather than an internal thread pool.

Also, the property for:

• undertow: server.undertow.worker-threads

• jetty: server.jetty.acceptors

You'll find the list of properties in Spring's documentation

Answer 3

While the accepted answer is very useful, I recently experienced what I believe to be the same problem as the original poster. This is the only search result I could find that directly correlated with my experience, so I thought I'd add my solution in case it helps someone.

In my case, the observed concurrency limit of 20 was imposed by the default setting of 20 for themaxConcurrentStreamExecution property in org.apache.coyote.http2.Http2Protocol.

If you're experiencing this problem and you're using HTTP/2, there's a good chance that increasing maxConcurrentStreamExecution will help.

You can find more info in the Tomcat Configuration Reference, which actually states that this should be set to 200 by default (not 20). You can definitely see the default setting of 20 in org.apache.coyote.http2.Http2Protocol, though, so I'm not sure if this is a typo or just something that presents itself differently in the embedded version of Tomcat.

Answer 4

Increase maxConcurrentStreamExecution (set 200) for HTTP/2 in Spring Boot 2:

@Bean
public WebServerFactoryCustomizer<TomcatServletWebServerFactory> containerCustomizer() {
    return new WebServerFactoryCustomizer<TomcatServletWebServerFactory>() {
        @Override
        public void customize(TomcatServletWebServerFactory factory) {
            factory.addConnectorCustomizers(new TomcatConnectorCustomizer() {
                @Override
                public void customize(Connector connector) {
                    Arrays.stream(connector.getProtocolHandler().findUpgradeProtocols())
                        .filter(upgradeProtocol -> upgradeProtocol instanceof Http2Protocol)
                        .map(upgradeProtocol -> (Http2Protocol) upgradeProtocol)
                        .forEach(http2Protocol -> http2Protocol.setMaxConcurrentStreamExecution(200));
                }
            });
        }
    };
}