當前位置:編程學習大全網 - 源碼下載 - 如何解決 hbase regiontoobusyexception

如何解決 hbase regiontoobusyexception

Hbase掃描時拋出ScannerTimeoutException處理

在對Hbase進行Scan的時候有時候會拋出ScannerTimeoutException,場景如下:

2012 15:28:52 (14mins, 41sec)

org.apache.hadoop.hbase.client.ScannerTimeoutException: 60622ms passed since the last invocation, timeout is currently set to 60000

at org.apache.hadoop.hbase.client.HTable$ClientScanner.next(HTable.java:1196)

at org.apache.hadoop.hbase.mapreduce.TableRecordReaderImpl.nextKeyValue(TableRecordReaderImpl.java:133)

at org.apache.hadoop.hbase.mapreduce.TableRecordReader.nextKeyValue(TableRecordReader.java:142)

at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.nextKeyValue(MapTask.java:532)

at org.apache.hadoop.mapreduce.MapContext.nextKeyValue(MapContext.java:67)

at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:143)

at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)

at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)

at org.apache.hadoop.mapred.Child$4.run(Child.java:255)

at java.security.AccessController.doPrivileged(Native Method)

at javax.security.auth.Subject.doAs(Subject.java:396)

at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1093)

at org.apache.hadoop.mapred.Child.main(Child.java:249)

原因查看步驟:

查找源碼如下:

long timeout = lastNext + scannerTimeout;

// If we are over the timeout, throw this exception to the client

// Else, it's because the region moved and we used the old id

// against the new region server; reset the scanner.

if (timeout < System.currentTimeMillis()) {

long elapsed = System.currentTimeMillis() - lastNext;

ScannerTimeoutException ex = new ScannerTimeoutException(

elapsed + "ms passed since the last invocation, " +

"timeout is currently set to " + scannerTimeout);

ex.initCause(e);

throw ex;

}

核實下如下代碼

long timeout = lastNext + scannerTimeout;

得出:ClientScanner.lastNext和HTable.scannerTimeout

跟蹤:HTable.scannerTimeout

this.scannerTimeout =

(int) conf.getLong(HConstants.HBASE_REGIONSERVER_LEASE_PERIOD_KEY, HConstants.DEFAULT_HBASE_REGIONSERVER_LEASE_PERIOD);

查看HConstants.HBASE_REGIONSERVER_LEASE_PERIOD_KEY和HConstants.DEFAULT_HBASE_REGIONSERVER_LEASE_PERIOD

我們可以得知如果沒有設置HConstants.HBASE_REGIONSERVER_LEASE_PERIOD_KEY那麽他們采用默認值即

public static long DEFAULT_HBASE_REGIONSERVER_LEASE_PERIOD = 60000;

跟蹤:ClientScanner.lastNext發現此為上次訪問時間

解決方案:

可以通過設置HConstants.HBASE_REGIONSERVER_LEASE_PERIOD_KEY解決

代碼如下:

config.setLong(HConstants.HBASE_REGIONSERVER_LEASE_PERIOD_KEY, 120000);

  • 上一篇:如何登陸淘寶子賬號
  • 下一篇:java初學者,如何學習java
  • copyright 2024編程學習大全網