首页 新闻 会员 周边

无法插入hbase数据“waiting for some tasks to finish. Expected max=0, tasksInProgress=14”

0
[已解决问题] 解决于 2019-09-18 09:37
  1. 系统中有两个插入hbase数据库的方法,船舶轨迹数据、轨迹异常数据; 插入船舶轨迹数据到hbase中正常,添加轨迹异常数据无法插入(最近一直 报 错,hbase集群正常); 在别的服务器上测试该方法,没有报错信息,且添加hbase数据正常
  2. 代码链接:

2019-08-22 01:52:00.519 INFO 15971 --- [duler_Worker-14] c.g.modules.job.utils.ScheduleJob : 任务执行完毕,任务ID:1113279906432684033 总共耗时:510 毫秒
2019-08-22 01:52:10.052 INFO 15971 --- [TaskExecutor-10] o.a.hadoop.hbase.client.AsyncProcess : #5663, waiting for some tasks to finish. Expected max=0, tasksInProgress=10
2019-08-22 01:52:20.047 INFO 15971 --- [red--pool1-t773] o.a.hadoop.hbase.client.AsyncProcess : #5663, table=trajectory_exception_201908, attempt=10/35 failed=1ops, last exception: org.apache.hadoop.net.ConnectTimeoutException: 10000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=slave7/59.212.148.118:60020] on slave7,60020,1566195248672, tracking started null, retrying after=10002ms, replay=1ops
2019-08-22 01:52:30.006 INFO 15971 --- [eduler_Worker-3] c.g.modules.job.utils.ScheduleJob : 开始执行定时任务
2019-08-22 01:52:30.007 INFO 15971 --- [eduler_Worker-3] c.g.modules.job.utils.ScheduleJob : 任务准备执行,任务ID:1113279906432684033
2019-08-22 01:52:30.007 INFO 15971 --- [eduler_Worker-3] c.g.modules.reptile.task.ShipWarnTask : 开始抓取船舶异常事件数据...
2019-08-22 01:52:30.086 INFO 15971 --- [TaskExecutor-10] o.a.hadoop.hbase.client.AsyncProcess : #5663, waiting for some tasks to finish. Expected max=0, tasksInProgress=11
2019-08-22 01:52:30.116 INFO 15971 --- [eduler_Worker-3] c.g.modules.reptile.task.ShipWarnTask : 船舶异常事件数据抓取完成!
2019-08-22 01:52:30.116 INFO 15971 --- [eduler_Worker-3] c.g.modules.job.utils.ScheduleJob : 任务执行完毕,任务ID:1113279906432684033 总共耗时:109 毫秒
2019-08-22 01:52:40.063 INFO 15971 --- [red--pool1-t773] o.a.hadoop.hbase.client.AsyncProcess : #5663, table=trajectory_exception_201908, attempt=11/35 failed=1ops, last exception: org.apache.hadoop.net.ConnectTimeoutException: 10000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=slave7/59.212.148.118:60020] on slave7,60020,1566195248672, tracking started null, retrying after=10004ms, replay=1ops
2019-08-22 01:52:50.119 INFO 15971 --- [TaskExecutor-10] o.a.hadoop.hbase.client.AsyncProcess : #5663, waiting for some tasks to finish. Expected max=0, tasksInProgress=12
2019-08-22 01:53:00.007 INFO 15971 --- [duler_Worker-15] c.g.modules.job.utils.ScheduleJob : 开始执行定时任务
2019-08-22 01:53:00.007 INFO 15971 --- [duler_Worker-15] c.g.modules.job.utils.ScheduleJob : 任务准备执行,任务ID:1113279906432684033
2019-08-22 01:53:00.007 INFO 15971 --- [duler_Worker-15] c.g.modules.reptile.task.ShipWarnTask : 开始抓取船舶异常事件数据...
2019-08-22 01:53:00.082 INFO 15971 --- [red--pool1-t773] o.a.hadoop.hbase.client.AsyncProcess : #5663, table=trajectory_exception_201908, attempt=12/35 failed=1ops, last exception: org.apache.hadoop.net.ConnectTimeoutException: 10000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=slave7/59.212.148.118:60020] on slave7,60020,1566195248672, tracking started null, retrying after=20000ms, replay=1ops
2019-08-22 01:53:00.116 INFO 15971 --- [duler_Worker-15] c.g.modules.reptile.task.ShipWarnTask : 船舶异常事件数据抓取完成!
2019-08-22 01:53:00.116 INFO 15971 --- [duler_Worker-15] c.g.modules.job.utils.ScheduleJob : 任务执行完毕,任务ID:1113279906432684033 总共耗时:109 毫秒
2019-08-22 01:53:20.168 INFO 15971 --- [TaskExecutor-10] o.a.hadoop.hbase.client.AsyncProcess : #5663, waiting for some tasks to finish. Expected max=0, tasksInProgress=13
2019-08-22 01:53:30.006 INFO 15971 --- [duler_Worker-16] c.g.modules.job.utils.ScheduleJob : 开始执行定时任务
2019-08-22 01:53:30.006 INFO 15971 --- [duler_Worker-16] c.g.modules.job.utils.ScheduleJob : 任务准备执行,任务ID:1113279906432684033
2019-08-22 01:53:30.006 INFO 15971 --- [duler_Worker-16] c.g.modules.reptile.task.ShipWarnTask : 开始抓取船舶异常事件数据...
2019-08-22 01:53:30.097 INFO 15971 --- [red--pool1-t773] o.a.hadoop.hbase.client.AsyncProcess : #5663, table=trajectory_exception_201908, attempt=13/35 failed=1ops, last exception: org.apache.hadoop.net.ConnectTimeoutException: 10000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=slave7/59.212.148.118:60020] on slave7,60020,1566195248672, tracking started null, retrying after=20186ms, replay=1ops
2019-08-22 01:53:30.124 INFO 15971 --- [duler_Worker-16] c.g.modules.reptile.task.ShipWarnTask : 船舶异常事件数据抓取完成!
2019-08-22 01:53:30.124 INFO 15971 --- [duler_Worker-16] c.g.modules.job.utils.ScheduleJob : 任务执行完毕,任务ID:1113279906432684033 总共耗时:118 毫秒
2019-08-22 01:53:50.323 INFO 15971 --- [TaskExecutor-10] o.a.hadoop.hbase.client.AsyncProcess : #5663, waiting for some tasks to finish. Expected max=0, tasksInProgress=14
2019-08-22 01:54:00.007 INFO 15971 --- [duler_Worker-17] c.g.modules.job.utils.ScheduleJob : 开始执行定时任务
2019-08-22 01:54:00.008 INFO 15971 --- [duler_Worker-17] c.g.modules.job.utils.ScheduleJob : 任务准备执行,任务ID:1113279906432684033
2019-08-22 01:54:00.008 INFO 15971 --- [duler_Worker-17] c.g.modules.reptile.task.ShipWarnTask : 开始抓取船舶异常事件数据...
2019-08-22 01:54:00.116 INFO 15971 --- [duler_Worker-17] c.g.modules.reptile.task.ShipWarnTask : 船舶异常事件数据抓取完成!
2019-08-22 01:54:00.116 INFO 15971 --- [duler_Worker-17] c.g.modules.job.utils.ScheduleJob : 任务执行完毕,任务ID:1113279906432684033 总共耗时:108 毫秒
2019-08-22 01:54:00.297 INFO 15971 --- [red--pool1-t773] o.a.hadoop.hbase.client.AsyncProcess : #5663, table=trajectory_exception_201908, attempt=14/35 failed=1ops, last exception: org.apache.hadoop.net.ConnectTimeoutException: 10000 millis timeout while waiting for channel to be ready for connect. ch : java.nio.channels.SocketChannel[connection-pending remote=slave7/59.212.148.118:60020] on slave7,60020,1566195248672, tracking started null, retrying after=20155ms, replay=1ops
2019-08-22 01:54:20.474 INFO 15971 --- [TaskExecutor-10] o.a.hadoop.hbase.client.AsyncProcess : #5663, waiting for some tasks to finish. Expected max=0, tasksInProgress=15
2019-08-22 01:54:30.007 INFO 15971 --- [duler_Worker-18] c.g.modules.job.utils.ScheduleJob : 开始执行定时任务
2019-08-22 01:54:30.007 INFO 15971 --- [duler_Worker-18] c.g.modules.job.utils.ScheduleJob : 任务准备执行,任务ID:1113279906432684033
有的说是 客户端提交速度过快引起的,使用rx2java,限制插入速度也不行 https://blog.csdn.net/xiefu5hh/article/details/53056113

lypsoft的主页 lypsoft | 菜鸟二级 | 园豆:204
提问于:2019-08-23 10:01
< >
分享
最佳答案
0

尝试了各种方法:最终重启了hbase集群两次 可以了(有点郁闷,不知道问题出在哪里)。

lypsoft | 菜鸟二级 |园豆:204 | 2019-09-18 09:32
清除回答草稿
   您需要登录以后才能回答,未注册用户请先注册