Certbot not creating acme-challenge folder

Question

I had working Let's encrypt certificates some months ago (with the old letsencrypt client). The server I am using is nginx.

Certbot is creating the .well-known folder, but not the acme-challenge folder

Now I tried to create new certificates via ~/certbot-auto certonly --webroot -w /var/www/webroot -d domain.com -d www.domain.com -d git.domain.com

But I always get errors like this:

IMPORTANT NOTES:
   - The following errors were reported by the server:

   Domain: git.domain.com
   Type:   unauthorized
   Detail: Invalid response from
   http://git.domain.com/.well-known/acme-challenge/ZLsZwCsBU5LQn6mnzDBaD6MHHlhV3FP7ozenxaw4fow:
   "<.!DOCTYPE html>
   <.html lang='en'>
   <.head prefix='og: http://ogp.me/ns#'>
   <.meta charset='utf-8'>
   <.meta content='IE=edge' http-equiv"

   Domain: www.domain.com
   Type:   unauthorized
   Detail: Invalid response from
   http://www.domain.com/.well-known/acme-challenge/7vHwDXstyiY0wgECcR5zuS2jE57m8I3utszEkwj_mWw:
   "<.html>
   <.head><.title>404 Not Found</title></head>
   <.body bgcolor="white">
   <.center><.h1>404 Not Found</h1></center>

(Of course the dots inside the HTML tags are not really there)

I have looked for a solution, but didn't found one yet. Does anybody know why certbot is not creating the folders?

Thanks in advance!

Answer 1

The problem was the nginx configuration. I replaced my long configuration files with the simplest config possible:

server {
    listen 80;
    server_name domain.com www.domain.com git.domain.com;
    root /var/www/domain/;
}

Then I was able to issue new certificates.

The problem with my long configuration files was (as far as I can tell) that I had the these lines:

location ~ /.well-known {
    allow all;
}

But they should be:

location ~ /.well-known/acme-challenge/ {
    allow all;
}

Now the renewal works, too.

Answer 2

I had a similar issue. My problem was, that I had this rule:

 location ~ /\. {
    access_log off;
    log_not_found off;
    deny all;
 }

these lines where canceling every acces to any directory starting with a "." (point)

Answer 3

For some strange reason (I think the certbot script changed in some way), I was not able in any way to renew the certificates. I found this thread that finally helped me after almost 4 hours of research:

https://community.letsencrypt.org/t/solved-invalid-response-403-forbidden/64170/13

hope it helps somebody else.

The trick is to add this in the apache config :

DocumentRoot /var/lib/letsencrypt/http_challenges
    <Directory /var/lib/letsencrypt/http_challenges>
            Allow from All
    </Directory>

Hope it works for someone else!