Dealing with Bad request

Question

I’m getting: 'HTTP/1.1 400 Bad Request\r\n' and I don’t get why. It looks like it authenticates and then there is a redirection and then it now doesn’t work. Why is this happening?

I had thought it was the header and that it was missing content type, but even adding that produced the same outcome..

headers = {
    'basic_auth': 'brofewfefwefewef:EKAXsWkdt5H6yJEmtexN',
    'Content-Type': 'application/json'
}
client = Client(ClientConfig(), headers=headers, refresh=True)


class FileDownloader(object):
    ...Line 152...
    def _get_http_pool(self, secure=True):
        if secure:
            _http = urllib3.PoolManager(cert_reqs=str('CERT_REQUIRED'),
                                        ca_certs=certifi.where())
        else:
            _http = urllib3.PoolManager()

        if self.headers:
            content_type = self.headers.get('Content-Type')
            if 'Content-Type' in self.headers:
                del self.headers['Content-Type']
            _headers = urllib3.util.make_headers(**self.headers)
            _http.headers.update(_headers)
            if content_type:
                _http.headers['content-type'] = content_type
        print(_http.headers)
        return _http

https://github.com/JMSwag/PyUpdater/blob/master/pyupdater/client/downloader.py Line 366, is where the download itself starts. This is perplexing to say the least.

Error:

DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): api.bitbucket.org
send: b'GET /2.0/repositories/Anexampleuser/repo/downloads/keys.gz HTTP/1.1\r\nHost: api.bitbucket.org\r\nAccept-Encoding: identity\r\nauthorization: Basic YnJvZmV3ZmVmd2VmZXdlZjpFS0FYc1drZHQ1SDZ5SkVtdGV4Tg==\r\n\r\n'
reply: 'HTTP/1.1 302 Found\r\n'
DEBUG:urllib3.connectionpool:https://api.bitbucket.org:443 "GET/2.0/repositories/Anexampleuser/repo/downloads/keys.gz HTTP/1.1" 302 0
DEBUG:urllib3.util.retry:Incremented Retry for (url='https://api.bitbucket.org/2.0/repositories/Anexampleuser/repo/downloads/keys.gz'): Retry(total=2, connect=None, read=None, redirect=None, status=None)
INFO:urllib3.poolmanager:Redirecting https://api.bitbucket.org/2.0/repositories/Anexampleuser/repo/downloads/keys.gz -> https://bbuseruploads.s3.amazonaws.com/a0e395b6-0c54-4efb-9074-57ec4190020b/downloads/3fc0be6d-ca69-42d3-9711-fbb5cfd2bc38/keys.gz?Signature=ZQxeUTvYC3Q%2Fo1aaS1CSuzyit0Q%3D&Expires=1515976464&AWSAccessKeyId=AKIAIQWXW6WLXMB5QZAQ&versionId=n.ymY11KRkq36Xozy25aChvfUT.YzTf5&response-content-disposition=attachment%3B%20filename%3D%22keys.gz%22
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): bbuseruploads.s3.amazonaws.com
header: Server header: Vary header: Content-Type header: X-OAuth-Scopes header: Strict-Transport-Security header: Date header: Location header: X-Served-By header: ETag header: X-Static-Version header: X-Content-Type-Options header: X-Accepted-OAuth-Scopes header: X-Credential-Type header: X-Render-Time header: Connection header: X-Request-Count header: X-Frame-Options header: X-Version header: Content-Length send: b'GET /a0e395b6-0c54-4efb-9074-57ec4190020b/downloads/3fc0be6d-ca69-42d3-9711-fbb5cfd2bc38/keys.gz?Signature=ZQxeUTvYC3Q%2Fo1aaS1CSuzyit0Q%3D&Expires=1515976464&AWSAccessKeyId=AKIAIQWXW6WLXMB5QZAQ&versionId=n.ymY11KRkq36Xozy25aChvfUT.YzTf5&response-content-disposition=attachment%3B%20filename%3D%22keys.gz%22 HTTP/1.1\r\nHost: bbuseruploads.s3.amazonaws.com\r\nAccept-Encoding: identity\r\nauthorization: Basic YnJvZmV3ZmVmd2VmZXdlZjpFS0FYc1drZHQ1SDZ5SkVtdGV4Tg==\r\n\r\n'
reply: 'HTTP/1.1 400 Bad Request\r\n'

Answer 1

As this is a 400, I would guess the request to Amazon’s S3 service doesn’t like and shouldn’t have the Authorization header, but requests passes it anyway when redirecting.

What you should do is using allow_redirects=False and then doing the redirect yourself, extracting the location from the response’s Location header.

Answer 2

The BitBucket API returns 302 for the /downloads/ endpoint, so the Authorization header is carried out to the next request, while Amazon does not like that header, so it returns 400. A workaround is recreating the redirected request manually. For example: (error checking omitted)

import urllib3

http_pool = urllib3.PoolManager()
req = http_pool.urlopen(
    'GET', 'https://api.bitbucket.org/2.0/repositories/brofewfefwefewef/eee/downloads/keys.gz',
    redirect=False, headers={'Authorization': 'Basic YnJvZmV3ZmVmd2VmZXdlZjpFS0FYc1drZHQ1SDZ5SkVtdGV4Tg=='})
redirected_req = http_pool.urlopen('GET', req.headers['Location'], preload_content=False)
with open('keys.gz', 'wb') as f:
    f.write(redirected_req.read())

By the way, your access token is still usable.

Answer 3

You need to give them access to the repository by creating a team inside bitbucket. Then you can use git --export to download files

git archive --remote=ssh://[e-mail]/<your-username>/<reponame>.git <branchname> <filename> --output output.tar

Of course, you need to have git authed eg. using an ssh key or similar

I don't think there is a way to make direct download links with auth in bitbucket, then you need to set that up outside bitbucket.