How to set timeout in *os.File/io.Read in golang

问题:

I know there is a function called SetReadDeadline that can set a timeout in socket(conn.net) reading, while io.Read not. There is a way that starts another routine as a timer to solve this problem, but it brings another problem that the reader routine(io.Read) still block:

func (self *TimeoutReader) Read(buf []byte) (n int, err error) { 
    ch := make(chan bool) 
    n = 0 
    err = nil 
    go func() { // this goroutime still exist even when timeout
            n, err = self.reader.Read(buf) 
            ch <- true 
    }() 
    select { 
    case <-ch: 
            return 
    case <-time.After(self.timeout): 
            return 0, errors.New("Timeout") 
    } 
    return 
} 

This question is similar in this post, but the answer is unclear. Do you guys have any good idea to solve this problem?

回答1:

Your mistake here is something different:

When you read from the reader you just read one time and that is wrong:

go func() { 
        n, err = self.reader.Read(buf) // this Read needs to be in a loop
        ch <- true 
}() 

Here is a simple example (https://play.golang.org/p/2AnhrbrhLrv)

buf := bytes.NewBufferString("0123456789")
r := make([]byte, 3)
n, err := buf.Read(r)
fmt.Println(string(r), n, err)
// Output: 012 3 <nil>

The size of the given slice is used when using the io.Reader. If you would log the n variable in your code you would see, that not the whole file is read. The select statement outside of your goroutine is at the wrong place.

go func() {
    a := make([]byte, 1024)
    for {
        select {
        case <-quit:
            result <- []byte{}
            return
        default:
            _, err = self.reader.Read(buf)
            if err == io.EOF {
                result <- a
                return
            }
        }
    }
}()

But there is something more! You want to implement the io.Reader interface. After the Read() method is called until the file ends you should not start a goroutine in here, because you just read chunks of the file. Also the timeout inside the Read() method doesn't help, because that timeout works for each call and not for the whole file.

回答2:

In addition to @apxp's point about looping over Read, you could use a buffer size of 1 byte so that you never block as long is there is data to read.

When interacting with external resources anything can happen. It is possible for any given io.Reader implementation to simply block forever. Here, I'll write one for you...

type BlockingReader struct{}

func (BlockingReader) Read(b []byte) (int, error) {
    <-make(chan struct{})
    return 0, nil
}

Remember anyone can implement an interface, so you can't make any assumptions that it will behave like *os.File or any other standard library io.Reader. In addition to asinine coding like mine above, an io.Reader could legitimately connect to a resources that can block forever.

You cannot kill gorountines, so if an io.Reader truly blocks forever the blocked goroutine will continue to consume resources until your application terminates. However, this shouldn't be a problem, a blocked goroutine does not consume much in the way of resources, and should be fine as long as you don't blindly retry blocked Reads by spawning more gorountines.

回答3:

Instead of setting a timeout directly on the read, you can close the os.File after a timeout. As written in https://golang.org/pkg/os/#File.Close

Close closes the File, rendering it unusable for I/O. On files that support SetDeadline, any pending I/O operations will be canceled and return immediately with an error.

This should cause your read to fail immediately.