HBase region over region servers load not balanced

Question

I'm running a small cluster with two region servers of HBase 0.94.7. I find that the load request over region servers is very unbalanced. From the Web UI, I got:

Region1:  numberOfOnlineRegions=1, usedHeapMB=26, maxHeapMB=3983
Region2:  numberOfOnlineRegions=22, usedHeapMB=44, maxHeapMB=3983

The region2 is servered as master. I checked that the load balancer is on. And I find some logs in the master log:

INFO org.apache.hadoop.hbase.master.LoadBalancer: Skipping load balancing because balanced cluster; servers=2 regions=1 average=0.5 mostloaded=1 leastloaded=0
DEBUG org.apache.hadoop.hbase.master.LoadBalancer: Balance parameter: numRegions=10, numServers=2, max=5, min=5
INFO org.apache.hadoop.hbase.master.LoadBalancer: Calculated a load balance in 12ms. Moving 5 regions off of 1 overloaded servers onto 1 less loaded servers
DEBUG org.apache.hadoop.hbase.master.LoadBalancer: Balance parameter: numRegions=8, numServers=2, max=4, min=4
INFO org.apache.hadoop.hbase.master.LoadBalancer: Calculated a load balance in 0ms. Moving 4 regions off of 1 overloaded servers onto 1 less loaded servers
INFO org.apache.hadoop.hbase.master.LoadBalancer: Skipping load balancing because balanced cluster; servers=2 regions=1 average=0.5 mostloaded=1 leastloaded=0
INFO org.apache.hadoop.hbase.master.HMaster: balance hri=LogTable,\x00\x00\x01\xE8\x00\x00\x01@\x09\xB2\xBA4$\xC3Oe,1374591174086.65391b7a54e9c8e85a3d94bf7627fd20., src=region2,60020,1374587851008, dest=region1,60020,1374587851018
DEBUG org.apache.hadoop.hbase.master.AssignmentManager: Starting unassignment of region LogTable,\x00\x00\x01\xE8\x00\x00\x01@\x09\xB2\xBA4$\xC3Oe,1374591174086.65391b7a54e9c8e85a3d94bf7627fd20. (offlining)

It seems that the load cannot be balanced from the region2 to region1. I don't know if it's a configuration problem? What parameter should I check on region1?

Thanks

Answer 1

if your rowkey is composed of ID, a date and a hash value, you could make the rowkey :hashvalue+date.

Answer 2

Are you using sequential rowkeys, like timestamp?If that is the case you might end up with RegionServer Hotspotting, putting uneven load on the servers. Avoid using sequential keys, if you can. If it is not possible create pre-splitted tables.