“Connection reset by peer” when benchmarking a sim

Question

I'm trying to write an extremely simple concurrent server in Rust to play with the language's concurrency primitives and its threading model. Here's my code:

use std::io::prelude::*;
use std::io::Result;
use std::net::{TcpListener, TcpStream, Shutdown};
use std::sync::{Arc, Mutex};
use std::thread;

fn handle_client(mut stream: TcpStream) -> Result<()> {
    try!(stream.write(b"HTTP/1.0 200 OK\r\nContent-Type: text/plain\r\nContent-Length: 5\r\n\r\nPong!\r\n"));
    // try!(stream.shutdown(Shutdown::Both));

    Ok(())
}

fn main() {
    let listener = TcpListener::bind("127.0.0.1:1337").unwrap();
    // let count = Arc::new(Mutex::new(0));

    for stream in listener.incoming() {
        match stream {
            Ok(stream) => {
                // let count = count.clone();
                thread::spawn(move || {
                    let _ = handle_client(stream);

                    // let mut count = count.lock().unwrap();
                    // *count += 1;
                    // println!("{:?}", *count);
                });
            }
            Err(e) => {
                println!("Error: {}", e);
            }
        }
    }

    drop(listener);
}

When I run ab -c 100 -n 100 http://127.0.0.1:1337/ with the program running as listed above, I get apr_socket_recv: Connection reset by peer (104) almost immediately. Why?

When I add try!(stream.shutdown(Shutdown::Both)); (commented out near the top, above) I no longer get the apr_socket_recv error like before, but apachebench gives me results that say 199 failed requests due to exceptions. Why? What am I doing wrong?

Concurrency Level:      100
Time taken for tests:   0.008 seconds
Complete requests:      100
Failed requests:        199
   (Connect: 0, Receive: 0, Length: 0, Exceptions: 199)
Total transferred:      500 bytes

Answer 1

I believe the problem is that you are not fully reading the data sent from the client, so the client never has a chance to transition to reading the response. When it tries to write more data, it notices the socket has been closed and fails.

I've augmented your example to read all the HTTP headers before replying, ignoring any request body. I'm punting on pretty error handling and just panicking if there are errors:

use std::io::prelude::*;
use std::io::BufReader;
use std::net::{TcpListener, TcpStream};
use std::thread;

fn handle_client(mut stream: TcpStream) {
    // Read all the headers
    for header in BufReader::new(&mut stream).lines() {
        let header = header.unwrap();
        if header == "\r" { break }
    }

    // Write our response
    stream.write_all(b"HTTP/1.0 200 OK\r\n\r\n").unwrap();
}

fn main() {
    let listener = TcpListener::bind("127.0.0.1:8080").unwrap();

    for stream in listener.incoming() {
        let stream = stream.unwrap();
        thread::spawn(|| {
            handle_client(stream);
        });
    }
}

This works with ab -c 50 -n 5000 http://127.0.0.1:8080/:

Benchmarking 127.0.0.1 (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests


Server Software:
Server Hostname:        127.0.0.1
Server Port:            8080

Document Path:          /
Document Length:        0 bytes

Concurrency Level:      50
Time taken for tests:   1.293 seconds
Complete requests:      5000
Failed requests:        0
Total transferred:      95000 bytes
HTML transferred:       0 bytes
Requests per second:    3868.22 [#/sec] (mean)
Time per request:       12.926 [ms] (mean)
Time per request:       0.259 [ms] (mean, across all concurrent requests)
Transfer rate:          71.77 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        1    6   1.7      6      14
Processing:     1    6   1.7      6      14
Waiting:        1    6   1.7      6      14
Total:          5   13   2.6     12      23

Percentage of the requests served within a certain time (ms)
  50%     12
  66%     13
  75%     13
  80%     14
  90%     17
  95%     19
  98%     21
  99%     22
 100%     23 (longest request)

Answer 2

The documentation of TcpStream states that

The socket will be closed when the value is dropped.

Since your function ends without properly shutting down the TCP-stream but simply by closing the socket, you get the Connection reset by peer error. A full TCP-shutdown requires multiple messages to be sent in both directions. This obviously can't happen anymore since the socket was closed.

I don't have an answer for your second question. Also, on stackoverflow you should only ask one question per question.