Docker Debian apt Error Reading from Server

Question

It would seem that apt-get is having issues connecting with the repository servers. I suppose it is likely compatibility issues, as mentioned here, however the proposed solution of apt-get clean does not work for me. Also I am surprised, if this is the case, that there are not more people having my issue.

MWE

Dockerfile

FROM debian:jessie
RUN apt-get clean && apt-get update && apt-get install -y --no-install-recommends \
    git

$ docker build .
docker build .
Sending build context to Docker daemon 2.048 kB
Step 0 : FROM debian:jessie
---> 4a5e6db8c069
Step 1 : RUN apt-get clean && apt-get update && apt-get install -y --no-install-recommends     git
---> Running in 43b93e93feab
Get:1 http://security.debian.org jessie/updates InRelease [63.1 kB]
... some omitted ...
Get:6 http://httpredir.debian.org jessie-updates/main amd64 Packages [3614 B]
Fetched 9552 kB in 7s (1346 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
The following extra packages will be installed:
... some omitted ...
0 upgraded, 26 newly installed, 0 to remove and 0 not upgraded.
Need to get 13.2 MB of archives.
After this operation, 64.0 MB of additional disk space will be used.
Get:1 http://security.debian.org/ jessie/updates/main libgnutls-deb0-28 amd64 3.3.8-6+deb8u2 [694 kB]
... some omitted ...
Get:5 http://httpredir.debian.org/debian/ jessie/main libnettle4 amd64 2.7.1-5 [176 kB]
Err http://httpredir.debian.org/debian/ jessie/main libffi6 amd64 3.1-2+b2
  Error reading from server. Remote end closed connection [IP: 176.9.184.93 80]
... some omitted ...
Get:25 http://httpredir.debian.org/debian/ jessie/main git amd64 1:2.1.4-2.1 [3624 kB]
Fetched 13.2 MB in 10s (1307 kB/s)
E: Failed to fetch http://httpredir.debian.org/debian/pool/main/libf/libffi/libffi6_3.1-2+b2_amd64.deb  Error reading from server. Remote end closed connection [IP: 176.9.184.93 80]

E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?
The command '/bin/sh -c apt-get clean && apt-get update && apt-get install -y --no-install-recommends     git' returned a non-zero code: 100

Please note that I also posted here with a different issue. I believe it to be unrelated, but it may well actually be.

Answer 1

For whoever is having an issue with this, this is my attempt to "fix" the issue by swapping out httpredir with a single working domain whenever the Dockerfile is being built:

FROM debian:je...

# Insert this line before "RUN apt-get update" to dynamically
# replace httpredir.debian.org with a single working domain
# in attempt to "prevent" the "Error reading from server" error.
RUN sed -i "s/httpredir.debian.org/`curl -s -D - http://httpredir.debian.org/demo/debian/ | awk '/^Link:/ { print $2 }' | sed -e 's@<http://\(.*\)/debian/>;@\1@g'`/" /etc/apt/sources.list

# Continue with your apt-get update...
RUN apt-get update...

What this command does is:

1. Curl the http://httpredir.debian.org/demo/debian/ from the building machine to get the headers from debian demo page (-s is silent, don't output. -D is to dump headers)
2. Extract the headers, find the Link header fragment. This contains the best route as recommended by httpredir.
3. The last sed -e ... is to extract out the domain name of the link in step 2.
4. Then finally, the domain found in step 3 is being feed into the global sed command, and replace the domain httpredir.debian.org found in /etc/apt/sources.list.

This is not a fix, but rather a simple hack to (greatly) reduce the chances of failed build. And... pardon me if it looks weird, as it's my virgin sed & piping attempt.

Edit

On a side note, if the domain that it picks simply too slow or not responding as it should, you may want to do it manually by

1. Visit http://httpredir.debian.org/demo.html, and you should see a link there like http://......./debian/. For example, at the point of writing, I saw http://mirrors.tuna.tsinghua.edu.cn/debian/

2. Instead of the long RUN sed -i.... command, use this instead:

   RUN sed -i "s/httpredir.debian.org/mirrors.tuna.tsinghua.edu.cn/" /etc/apt/sources.list
   

Answer 2

Not enough reputation to comment on previous answers, so I will (confusingly) add a new answer:

• I don't think hardcoding a single mirror is really a viable solution, since as for example seen here, there's a reason debian implemented the whole httpredir thing -- mirrors go down or out of date.
• I've dealt with this issue a lot of times, and the logs always indicate that docker's actually running the apt-get command, which means --no-cache is unlikely to be fixing it -- it's just that if you rebuild, httpredir is likely to pick a different mirror, even if you don't change anything in your docker file, and the build will work.

Answer 3

For those visiting with similar issues, using the --no-cache flag in the docker build may help. Similar issues (though not this exact one) can occur if the apt-get update is old and not being recalled sue to caching.

Answer 4

The httpredir.debian.org mirror is "magic" in that it will load-balance and geo-ip you to transparent increase performance and availability. I would therefore immediately suspect it of causing your problem, or at least be the first thing to rule out.

I would check if you could:

• Still reproduce the problem; httpredir.debian.org will throw out "bad" mirrors from its internal lists so your issue may have been temporary.

• Reproduce the problem with a different, non-httpredir.debian.org mirror. Try something like ftp.de.debian.org. If it then works with this mirror, do please contact the httpredir.debian.org maintainer and report the issue to them. They are quite responsive and open to bug reports.

Answer 5

I added apt-get clean to my dockerfile before the apt-get update line, it seems to have done the trick.

I guess I have no way of knowing whether or not it was the extra command or if it was luck that fixed my build, but I took the advice from https://github.com/CGAL/cgal-testsuite-dockerfiles/issues/19