com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:

Mr_xushuai
Hi yan
I got the following exception, I find then that the error I getting is intermittent.
In that case, I'm guessing the problem is caused by a timeout that is configured in the connection pool.
From the following stack trace, I get the impression that you are keeping the database connection open while the user browses your Web page and sometimes the user browses for such a long time that you hit the connection pool timeout.
Obtain a database connection immediately prior to performing a database operation and release the connection as soon as the operation has completed.
I try to change the glu-console-webapp.groovy ,set dataSource.autoReconnect=true,but it's seem not right, I check the "http://www.grails.org/doc/2.2.1/guide/conf.html",The datasource not have the attribute(wait_timeout, autoReconnect).

In my database(mysql my.conf),
wait_timeout = 3600
interactive_timeout = 3600

Any idea on how to resolve this problem?


com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: The last packet successfully received from the server was 7,567,037 milliseconds ago.  The last packet sent successfully to the server was 7,567,038 milliseconds ago. is longer than the server configured value of 'wait_timeout'. You should consider either expiring and/or testing connection validity before use in your application, increasing the server configured values for client timeouts, or using the Connector/J connection property 'autoReconnect=true' to avoid this problem.
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:1121)
        at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3938)
        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2551)
        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2731)
        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2809)
        at com.mysql.jdbc.ConnectionImpl.setAutoCommit(ConnectionImpl.java:5333)
        at org.apache.commons.dbcp.DelegatingConnection.setAutoCommit(DelegatingConnection.java:371)
        at org.apache.commons.dbcp.PoolingDataSource$PoolGuardConnectionWrapper.setAutoCommit(PoolingDataSource.java:328)
        at org.grails.datastore.gorm.GormStaticApi.withTransaction(GormStaticApi.groovy:687)
        at org.linkedin.glu.console.provisioner.services.storage.DeploymentStorageImpl.endDeployment(DeploymentStorageImpl.groovy:134)
        at org.linkedin.glu.orchestration.engine.deployment.ProgressTracker.onPlanEnd(ProgressTracker.groovy:71)
        at org.linkedin.util.reflect.ObjectProxyInvocationHandler.invoke(ObjectProxyInvocationHandler.java:56)
        at org.linkedin.glu.provisioner.plan.impl.AsyncPlanExecutionProgressTrackerProxy.doInvoke(AsyncPlanExecutionProgressTrackerProxy.java:73)
        at org.linkedin.glu.provisioner.plan.impl.AsyncPlanExecutionProgressTrackerProxy.access$000(AsyncPlanExecutionProgressTrackerProxy.java:40)
        at org.linkedin.glu.provisioner.plan.impl.AsyncPlanExecutionProgressTrackerProxy$1.run(AsyncPlanExecutionProgressTrackerProxy.java:62)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
        at java.util.concurrent.FutureTask.run(FutureTask.java:262)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
        at java.lang.Thread.run(Thread.java:744)
Caused by: java.net.SocketException: Broken pipe
        at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:113)
        at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
        at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
        at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
        at com.mysql.jdbc.MysqlIO.send(MysqlIO.java:3919)
        ... 18 more
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:

frenchyan
Administrator
There was a thread relating to a similar issue: http://glu.977617.n3.nabble.com/Database-Connection-Pool-Configuration-for-Console-td4025869.html


in the "Note" section it discusses your issue and ways to address it.

Yan
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:

Mr_xushuai
Many thanks, in advance, for your help.

I still have another question, when I click the "View agents fabric", It's doesn’t open.
because return 504 Gateway timeout.I have set the timeout to 10 minutes.I think it is incredible.So I guess it's an issue with too many fabric and agent.We currently have 500+ fabric and 5000+ agent, what should I do  for solve the problem?


the url :http://glu/console/fabric/listAgentFabrics?fabric=osp-zjrs-up
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException:

frenchyan
Administrator
Hmmm..

The code being executed is the following:


I could imagine that line 73 which gets all the agents could be costly as it calls there:


which essentially does:

for each fabric
  get all children nodes of zookeeper node: /org/glu/agents/names
  for each node:
   test if node exists and if it does
     read content of that node (which contains the fabric)


Since you have 500 fabrics, that would represent 500 access to ZooKeeper + 2 access per agents (one for test, one for read content) => 10K access => 10500 accesses to ZooKeeper to render this screen.

I am not sure if that is the problem but it certainly could be. So here what I would suggest you to do:

a) check the console log files, especially the GC log files. In the past when people ran into performance issues it was due to excessive GCing. If that is the case then you need to increase the memory size allocated to the console

b) if this is not a GC issue, then you would need to investigate the #access to ZooKeeper. I would suggest writing a little separate program that simply does the loop I described and see how long it takes.

Hope this helps
Yan

Loading...