OAuth2 access to Cosmos' WebHDFS in FIWARE Lab

Question

I've recently seen the access to Cosmos' WebHDFS in FIWARE Lab has been protected with OAuth2. I know I have to add a OAuth2 token to the request in order to continue using WebHDFS, but:

• How can I get the token?
• How the token is added to the request?

Without the token, the API always returns:

$ curl -X GET "http://cosmos.lab.fi-ware.org:14000/webhdfs/v1/user/gtorodelvalle?op=liststatus&user.name=gtorodelvalle"
Auth-token not found in request header

Answer 1

The right request must be:

curl -X POST "https://cosmos.lab.fi-ware.org:13000/cosmos-auth/v1/token" -H "Content-Type: application/x-www-form-urlencoded" -d "grant_type=password&username=[e-mail]&password=yourpassword" -k

The url was incorrect, the correct is https://cosmos.lab.fi-ware.org:13000

-k is for turn off certificate verification

Answer 2

Yes, now WebHDFS access is protected with OAuth2. This is part of the general mechanism for pretecting REST APIs in FIWARE, which performs authentication and authorization. You can find more details here.

First of all, you must request an OAuth2 token to the Cosmos tokens generator. This is a service running in cosmos.lab.fiware.org:13000. You can do this using any REST client, the easiest way is using the curl command:

$ curl -k -X POST "https://cosmos.lab.fiware.org:13000/cosmos-auth/v1/token" -H "Content-Type: application/x-www-form-urlencoded" -d "grant_type=password&username=[e-mail]&password=xxxxxxxx"
{"access_token": "qjHPUcnW6leYAqr3Xw34DWLQlja0Ix", "token_type": "Bearer", "expires_in": 3600, "refresh_token": "V2Wlk7aFCnElKlW9BOmRzGhBtqgR2z"}

As you can see, your FIWARE Lab credentials are required in the payload, in the form of a password-based grant type.

Once the access token is got (in the example above, it is qjHPUcnW6leYAqr3Xw34DWLQlja0Ix), simply add it to the same WebHDFS request you were performing in the past. The token is added by using the X-Auth-Token header:

$ curl -X GET "http://cosmos.lab.fiware.org:14000/webhdfs/v1/user/frb/path/to/the/data?op=liststatus&user.name=frb" -H "X-Auth-Token: qjHPUcnW6leYAqr3Xw34DWLQlja0Ix"
{"FileStatuses":{"FileStatus":[...]}}

If you try the above request with a random token the server will return the token is not valid; that's because you have not authenticated properly:

$ curl -X GET "http://cosmos.lab.fiware.org:14000/webhdfs/v1/user/frb/path/tp/the/data?op=liststatus&user.name=frb" -H "X-Auth-Token: randomtoken93487345"
User token not authorized

The same way, if using a valid token but trying to access another HDFS userspace, you will get the same answer; that's because you are not authorized to access any HDFS userspace but the one owned by you:

$ curl -X GET "http://cosmos.lab.fiware.org:14000/webhdfs/v1/user/fgalan/path/tp/the/data?op=liststatus&user.name=fgalan" -H "X-Auth-Token: qjHPUcnW6leYAqr3Xw34DWLQlja0Ix"
User token not authorized

IMPORTANT UPDATE:

From summer 2016, cosmos.lab.fiware.org is not workin anymore. Instead, a pair of clusters, storage.cosmos.lab.fiware.org and computing.cosmos.lab.fiware.org have been setup. Regarding the auth server of Cosmos, it currently run in computing.cosmos.lab.fiware.org, port TCP/13000.