fileContent tries to variable expand file names

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

fileContent tries to variable expand file names

sodul
We have a bug in one of our scripts that ended up creating a directory that looks like a groovy variable:
/some/path/${some_variable}/

The groovy code tries to expand the 'some_variable' value and we get this error:
Error 500: Internal Server Error
URI
/console/agents/fileContent/hadoop-scheduler00
Class
org.linkedin.glu.agent.rest.client.RecoverableAgentException
Message
Internal Connector Error (1002) - The connector encountered an unexpected condition which prevented it from fulfilling the request

 Line | Method
->>  653 | handleRecoverableError in org.linkedin.glu.agent.rest.client.AgentRestClient
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 
|    626 | handleError            in     ''
|    599 | handleResponse . . . . in     ''
|    275 | getFileContent         in     ''
|    202 | doCall . . . . . . . . in org.linkedin.glu.orchestration.engine.agents.AgentsServiceImpl$_streamFileContent_closure10
|    394 | doCall                 in org.linkedin.glu.orchestration.engine.agents.AgentsServiceImpl$_withRemoteAgent_closure20
|     64 | doCall . . . . . . . . in org.linkedin.glu.agent.rest.client.AgentFactoryImpl$_withRemoteAgent_closure2
|     85 | withRestClient         in org.linkedin.glu.agent.rest.client.RestClientFactoryImpl
|     55 | withRemoteAgent . . .  in org.linkedin.glu.agent.rest.client.AgentFactoryImpl
|    393 | withRemoteAgent        in org.linkedin.glu.orchestration.engine.agents.AgentsServiceImpl
|    201 | streamFileContent . .  in     ''
|     84 | streamFileContent      in org.linkedin.glu.orchestration.engine.agents.AuditedAgentsService
|    413 | doCall . . . . . . . . in org.linkedin.glu.console.controllers.AgentsController$_closure19_closure50
|    718 | handleNoAgent          in org.linkedin.glu.console.controllers.AgentsController
|    405 | doCall . . . . . . . . in org.linkedin.glu.console.controllers.AgentsController$_closure19
|    669 | handle                 in org.eclipse.jetty.servlet.ServletHolder
|   1448 | doFilter . . . . . . . in org.eclipse.jetty.servlet.ServletHandler$CachedChain
|    195 | doFilter               in grails.plugin.cache.web.filter.PageFragmentCachingFilter
|     63 | doFilter . . . . . . . in grails.plugin.cache.web.filter.AbstractFilter
|   1419 | doFilter               in org.eclipse.jetty.servlet.ServletHandler$CachedChain
|    455 | doHandle . . . . . . . in org.eclipse.jetty.servlet.ServletHandler
|    137 | handle                 in org.eclipse.jetty.server.handler.ScopedHandler
|    575 | handle . . . . . . . . in org.eclipse.jetty.security.SecurityHandler
|    231 | doHandle               in org.eclipse.jetty.server.session.SessionHandler
|   1075 | doHandle . . . . . . . in org.eclipse.jetty.server.handler.ContextHandler
|    384 | doScope                in org.eclipse.jetty.servlet.ServletHandler
|    193 | doScope . . . . . . .  in org.eclipse.jetty.server.session.SessionHandler
|   1009 | doScope                in org.eclipse.jetty.server.handler.ContextHandler
|    135 | handle . . . . . . . . in org.eclipse.jetty.server.handler.ScopedHandler
|    276 | forward                in org.eclipse.jetty.server.Dispatcher
|    103 | forward . . . . . . .  in     ''
|   1419 | doFilter               in org.eclipse.jetty.servlet.ServletHandler$CachedChain
|     55 | doFilter . . . . . . . in org.apache.shiro.grails.SavedRequestFilter
|   1419 | doFilter               in org.eclipse.jetty.servlet.ServletHandler$CachedChain
|    449 | executeChain . . . . . in org.apache.shiro.web.servlet.AbstractShiroFilter
|    365 | call                   in org.apache.shiro.web.servlet.AbstractShiroFilter$1
|     90 | doCall . . . . . . . . in org.apache.shiro.subject.support.SubjectCallable
|     83 | call                   in     ''
|    383 | execute . . . . . . .  in org.apache.shiro.subject.support.DelegatingSubject
|    362 | doFilterInternal       in org.apache.shiro.web.servlet.AbstractShiroFilter
|    125 | doFilter . . . . . . . in org.apache.shiro.web.servlet.OncePerRequestFilter
|   1419 | doFilter               in org.eclipse.jetty.servlet.ServletHandler$CachedChain
|    455 | doHandle . . . . . . . in org.eclipse.jetty.servlet.ServletHandler
|    137 | handle                 in org.eclipse.jetty.server.handler.ScopedHandler
|    533 | handle . . . . . . . . in org.eclipse.jetty.security.SecurityHandler
|    231 | doHandle               in org.eclipse.jetty.server.session.SessionHandler
|   1075 | doHandle . . . . . . . in org.eclipse.jetty.server.handler.ContextHandler
|    384 | doScope                in org.eclipse.jetty.servlet.ServletHandler
|    193 | doScope . . . . . . .  in org.eclipse.jetty.server.session.SessionHandler
|   1009 | doScope                in org.eclipse.jetty.server.handler.ContextHandler
|    135 | handle . . . . . . . . in org.eclipse.jetty.server.handler.ScopedHandler
|    255 | handle                 in org.eclipse.jetty.server.handler.ContextHandlerCollection
|    154 | handle . . . . . . . . in org.eclipse.jetty.server.handler.HandlerCollection
|    116 | handle                 in org.eclipse.jetty.server.handler.HandlerWrapper
|    368 | handle . . . . . . . . in org.eclipse.jetty.server.Server
|    489 | handleRequest          in org.eclipse.jetty.server.AbstractHttpConnection
|    942 | headerComplete . . . . in     ''
|   1004 | headerComplete         in org.eclipse.jetty.server.AbstractHttpConnection$RequestHandler
|    640 | parseNext . . . . . .  in org.eclipse.jetty.http.HttpParser
|    235 | parseAvailable         in     ''
|     82 | handle . . . . . . . . in org.eclipse.jetty.server.AsyncHttpConnection
|    628 | handle                 in org.eclipse.jetty.io.nio.SelectChannelEndPoint
|     52 | run . . . . . . . . .  in org.eclipse.jetty.io.nio.SelectChannelEndPoint$1
|    608 | runJob                 in org.eclipse.jetty.util.thread.QueuedThreadPool
|    543 | run . . . . . . . . .  in org.eclipse.jetty.util.thread.QueuedThreadPool$3
^    722 | run                    in java.lang.Thread
Reply | Threaded
Open this post in threaded view
|

Re: fileContent tries to variable expand file names

frenchyan
Administrator
Can you please file a ticket for it on github with steps to reproduce? Is it a blocker (I was not planning to release a new version until I come back)? Or it is not an issue after fixing your glu script?

Thanks
Yan
Reply | Threaded
Open this post in threaded view
|

Re: fileContent tries to variable expand file names

sodul
https://github.com/pongasoft/glu/issues/247

This is not a blocker for us, that directory name was created by error and it has been fixed on our side.

Very easy to reproduce.

My main concern is that since groovy can execute code as part of the variable expansion, this is possible: http://xkcd.com/327/
Reply | Threaded
Open this post in threaded view
|

Re: fileContent tries to variable expand file names

frenchyan
Administrator
I totally agree with you in terms of the concern. 

That being said the stack trace you posted does not show a variable expansion issue. I am seeing a RecoverableAgentException with no indication whatsoever that a variable has been expanded. What am I missing? Do you have something else (on the agent side) that shows this specific issue?

I will try to reproduce in any case.

Yan


Reply | Threaded
Open this post in threaded view
|

Re: fileContent tries to variable expand file names

sodul
From the agent's log when going to a /tmp/${var} directory:
Exception in thread "Thread-31819" groovy.lang.GroovyRuntimeException: exception while reading process stream
	at org.codehaus.groovy.runtime.ProcessGroovyMethods$TextDumper.run(ProcessGroovyMethods.java:482)
	at java.lang.Thread.run(Thread.java:722)
Caused by: java.io.IOException: Stream closed
	at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:162)
	at java.io.BufferedInputStream.read1(BufferedInputStream.java:272)
	at java.io.BufferedInputStream.read(BufferedInputStream.java:334)
	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:283)
	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:325)
	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:177)
	at java.io.InputStreamReader.read(InputStreamReader.java:184)
	at java.io.BufferedReader.fill(BufferedReader.java:154)
	at java.io.BufferedReader.readLine(BufferedReader.java:317)
	at java.io.BufferedReader.readLine(BufferedReader.java:382)
	at org.codehaus.groovy.runtime.ProcessGroovyMethods$TextDumper.run(ProcessGroovyMethods.java:475)
	... 1 more
2013/10/07 18:27:47.344 INFO [AgentImpl] getFileContent: [location:tmp]
Reply | Threaded
Open this post in threaded view
|

Re: fileContent tries to variable expand file names

frenchyan
Administrator
This does not show that the string is being "evalled". It simply shows that /${var} is "lost". But I will investigate.

Yan
Reply | Threaded
Open this post in threaded view
|

Re: fileContent tries to variable expand file names

frenchyan
Administrator
In reply to this post by sodul
I can reproduce the issue (as in, it fails), but I really don't think it is evaluating the string.

The issue is that the API requires a "location" which needs to be properly URI encoded (/$%7Bvar%7D) but returns canonicalPath (coming from file.getCanonicalPath()) which are encoded differently...

Kind of a mess :( At least the name is explicit (canonicalPath) but I wished I had noticed that earlier to make the API more symmetric.

Yan


On Mon, Oct 7, 2013 at 9:24 AM, Yan (pongasoft) <[hidden email]> wrote:
This does not show that the string is being "evalled". It simply shows that /${var} is "lost". But I will investigate.

Yan

Reply | Threaded
Open this post in threaded view
|

Re: fileContent tries to variable expand file names

sodul
Good to know it is not a security issue. No need to rush on a fix.