Hive - Thrift - Missing version in readMessageBegi

Question

Hello I'm trying to build a nodejs client for querying my hive database using thrift but I'm facing a strange issue ... I've generated my nodejs client API with thrift (thrift -r --gen js:node TCLIService.thrift with TCLIService being the thrift file defining the Hive services) and now I try to connect to Hive but my OpenSession is pending ... Maybe I'me not doing the right call but I don't find anything recent on the net (every thrift / node / hive project is 4 or 5 years old). Can you take a look and tell me if I'm doing it wrong ? Thanks

TCLIService.thrift :

// OpenSession()
//
// Open a session (connection) on the server against
// which operations may be executed.
struct TOpenSessionReq {
    // The version of the HiveServer2 protocol that the client is using.
    1: required TProtocolVersion client_protocol = TProtocolVersion.HIVE_CLI_SERVICE_PROTOCOL_V8

    // Username and password for authentication.
    // Depending on the authentication scheme being used,
    // this information may instead be provided by a lower
    // protocol layer, in which case these fields may be
    // left unset.
    2: optional string username
    3: optional string password

    // Configuration overlay which is applied when the session is
    // first created.
    4: optional map<string, string> configuration
}

service TCLIService {
    TOpenSessionResp OpenSession(1:TOpenSessionReq req);
}

Nodejs code : var sessionHandle = '';

function openSession(config) {
    openSessReq = new ttypes.TOpenSessionReq();
    openSessReq.client_protocol = ttypes.TProtocolVersion.HIVE_CLI_SERVICE_PROTOCOL_V8;
    openSessReq.username = config.hiveUser;
    openSessReq.password = config.hivePassword;
    console.log("openSessReq = " + JSON.stringify(openSessReq));
    console.log("Before OpenSession : sessionHandle = " + sessionHandle);
    sessionHandle = client.OpenSession(openSessReq, function (err, result){
        console.log('handler fired ... ');
        if (err) {
            console.error("error : " + err);
        } else {
            console.log("result : " + result);
        }
    });
    console.log("After OpenSession : sessionHandle = " + sessionHandle);
}

connection.on('error', function(err) {
    /*Error detected, print the error*/
    console.error(err);
    /*Close the program with error code*/
    process.exit(1)
});

console.log('Error handler registered ...')

connection.on('connect', function(){
    console.log('Connected ...');
    /*Init session*/
    console.log(client);
    openSession(config);
}

My output is the following :

CreateConnection DONE ...
Error handler registered ...
Connect handler registered ...
Connected ...
{ output:
   TBufferedTransport {
     defaultReadBufferSize: 1024,
     writeBufferSize: 512,
     inBuf: <Buffer e8 19 2b 03 00 00 00 00 b0 1a 2b 03 00 00 00 00 e8 18 2b 03 00 00 00 00 f0 18 2b 03 00 00 00 00 b0 19 2b 03 00 00 00 00 78 1a 2b 03 00 00 00 00 70 1d ... >,
     readCursor: 0,
     writeCursor: 0,
     outBuffers: [],
     outCount: 0,
     onFlush: [Function],
     client: [Circular] },
  pClass: [Function: TBinaryProtocol],
  _seqid: 0,
  _reqs: {} }
openSessReq = {"client_protocol":7,"username":"root","password":"root","configuration":null}
Before OpenSession : sessionHandle =
After OpenSession : sessionHandle = undefined
^C

The script is running indefinitely ... If I change the port or shutdown HiveServer I'm facing an error so the connection is working but I can't find out why the openSession isn't ! Thanks for your help

EDIT : I've checked my logs and the NOSASL identification was the issue ... I've fixed (I think) this issue and now getting the following error :

Client :

{ [Error: write EPIPE]
 code: 'EPIPE',
 errno: 'EPIPE',
 syscall: 'write',
 address: undefined }

Server :

2016-02-17 13:36:37,152 ERROR org.apache.thrift.server.TThreadPoolServer: [HiveServer2-Handler-Pool: Thread-24]: Thrift error occurred during processing of message.
    org.apache.thrift.protocol.TProtocolException: Missing version in readMessageBegin, old client?
        at org.apache.thrift.protocol.TBinaryProtocol.readMessageBegin(TBinaryProtocol.java:228)
        at org.apache.thrift.TBaseProcessor.process(TBaseProcessor.java:27)
        at org.apache.hive.service.auth.TSetIpAddressProcessor.process(TSetIpAddressProcessor.java:56)
        at org.apache.thrift.server.TThreadPoolServer$WorkerProcess.run(TThreadPoolServer.java:285)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:745)

I've read that the issue must be raised by using the wrong protocol layer ... I'm using TBinaryProtocol is it a problem ?

Answer 1

Try to check three part of your environments, all of the three are in the same config file(hive-site.xml) by default:

1. transportMode check hive.server2.transport.mode, you must use the same mode in serve and client.

2. ssl enable check hive.server2.use.SSL, you can try to close to make sure there has nothing to do with this configuration.

3. auth check hive.server2.authentication, since there has several kinds(NONE,NOSASL,KERBEROS,LDAP,CUSTOM) of auth mode, make sure you use the proper one.

PS. You can try to connect to your hs2 by console and see if there is anything wrong with server.

Answer 2

I've fixed my problem by doing the following :

Check the protocol version of my HiveServer2 by using beewax and query my HiveServer2 (with Hue) and checking HiveServer2 logs (I found it was HIVE_CLI_SERVICE_PROTOCOL_V7 for Hive 0.15.0)

Enable NOSASL authentication on my HiveServer2 by modifying the hive-site.xml