Set dependency Sequence for Parallel deployment Plans

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Set dependency Sequence for Parallel deployment Plans

chaoyang ma
Hi,

I want to use parallel plans, but my project have dependency.
Project A, B, C, D, E

I want a plan that deploy A first, then B, then C, D at the same time, then E.

all agents should do this in parallel.

How to make this plan?

Reply | Threaded
Open this post in threaded view
|

Re: Set dependency Sequence for Parallel deployment Plans

frenchyan
Administrator
In general it is not a good think to have dependencies between components. At the moment glu only supports parent/child as a dependency model. The parent/child dependency is for the "typical" use case: webapp container (the parent), webapp (the child) and need to be deployed on the same agent. 

I don't believe this is your use case. It is strongly recommended that you remove those dependencies in general, because components can come and go and relying on a component to be always up (and prior to another one being up) will lead to unreliable setups. It is also not very obvious what should happen in the event that component C needs to be upgraded to a new version: for some period of time it will not be available so how is E going to behave? Should E also be redeployed? etc... not a trivial problem.

That being said, I do understand that you currently have some dependencies that you have to work around. 

The best you can do at this stage is issue multiple parallel deployments from the command line / rest api. I would suggest adding tags to your entries (like project-a, project-b, etc...), then issue something like (pseudo code!):

deploy tag project-a in parallel
deploy tag project-b in parallel
deploy tag project-c and project-d in parallel
deploy tag project-e in parallel

which will essentially achieve what you want to do. The use of tags allows you to not have to worry about which agents are involved or how many. It is also very easy to wrap this is a small shell script that will accomplish all this in one single call.

Yan

Reply | Threaded
Open this post in threaded view
|

Re: Set dependency Sequence for Parallel deployment Plans

sodul
The way I've done it in the past is to have the start script of each component handle the dependency just before launching the actual process.

I do that with a start script written in python, it knows where the 'parent' service is and wait for serverA:port to listen. It checks every second, then fail to start after 30s (or 2m, depends on your apps).

With this implementation we can tell Glu to deploy all apps in parallel, and they all actually start in the required order. It has worked well for us in a setup with over a dozen interdependent services. Glu itself is never aware of the startup dependencies btw.

Note that this technique only for the Start step. I do not recommend to bind your stop steps in the same manner since you would not be able to redeploy individual components anymore.

This works well in practice, but should not prevent you from telling your developers to fix their applications to work as services. Your developers should write their code so that servers are expected to crash (hardware failure, etc…). Netflix has pushed this as far as to destroy EC2 instances randomly every day: http://techblog.netflix.com/2012/07/chaos-monkey-released-into-wild.html


Reply | Threaded
Open this post in threaded view
|

Re: Set dependency Sequence for Parallel deployment Plans

sabhinandan
In reply to this post by chaoyang ma
Hi,
We are having the same use case an the way we are handling is
in the json model of each service we are having the mountpoint of other service on which it depends
For example.
JSON model of E contains functional depdency to C,D
and JSON model of D contains, functional depdency to B
and JSON model of B contains, functional depdency to A
And there are two ways you can solve this
First is :
now when we try to start E, E checks if C and D is in running state , if not then start them
now D will check, it depends on B and B is in running state or not, if not then start B, and same for B...
Second is:
From the application level we know this depdency chain, and when user try to start E, it automatically first gives call to services on which it depends, here it is C and D,
Same way D will give first call to B and so on...

I hope it should help !!!
Reply | Threaded
Open this post in threaded view
|

Re: Set dependency Sequence for Parallel deployment Plans

chaoyang ma
In reply to this post by chaoyang ma
Hi, I have added a tag in the entry and hope to use the filter to seperate the deployment.

      "tags": [
        "projectA"
      ]

And I try to use command line to drive the deployment.

./console-cli.py -f dev -u admin -x admin -s "tags='projectA'" -n deploy

I expected to see only projectA entries in the plan. But this command still lists all the entries. Do I need to do something else? or the command I am using is wrong?

Reply | Threaded
Open this post in threaded view
|

Re: Set dependency Sequence for Parallel deployment Plans

frenchyan
Administrator
Are you sure you have only 1 entry with this tag? Does it work in the UI (console)? If you go to the dashboard and click on your tag it should filter the model by the tag. And then Plans subtab/deploy should also should a filtered out version.

I just tried with the tutorial and it works:

./bin/console-cli.sh -f glu-dev-1 -u admin -x admin -s "tags='backend'" -n deploy
<?xml version="1.0"?>
<plan fabric="glu-dev-1" systemId="0072cd5616ea327bd3673542608b3cdfe16317ef" id="8c9d704b-d7ba-4a42-b92d-f8e6b05e4a90" origin="rest" planType="deploy" filter="tags='backend'" name="origin=rest - fabric=glu-dev-1 - planType=deploy - filter=tags='backend' - SEQUENTIAL" savedTime="1372095840329">
  <sequential origin="rest" fabric="glu-dev-1" planType="deploy" filter="tags='backend'">
    <sequential agent="agent-1" mountPoint="/sample/i003">
      <leaf agent="agent-1" fabric="glu-dev-1" initParameters="{metadata={cluster=c2, container={name=sample}, product=product1, version=1.0.0}, port=9002, skeleton=http://localhost:8080/glu/repository/tgzs/jetty-distribution-8.1.10.v20130312.tar.gz, tags=[backend, osx, webapp], webapps=[{war=http://localhost:8080/glu/repository/wars/org.linkedin.glu.samples.sample-webapp-5.0.0.war, contextPath=/cp4, monitor=/monitor}]}" mountPoint="/sample/i003" name="Install script for [/sample/i003] on [agent-1]" script="http://localhost:8080/glu/repository/scripts/org.linkedin.glu.script-jetty-5.0.0/JettyGluScript.groovy" scriptLifecycle="installScript" />
      <leaf agent="agent-1" fabric="glu-dev-1" mountPoint="/sample/i003" name="Run [install] phase for [/sample/i003] on [agent-1]" scriptAction="install" toState="installed" />
      <leaf agent="agent-1" fabric="glu-dev-1" mountPoint="/sample/i003" name="Run [configure] phase for [/sample/i003] on [agent-1]" scriptAction="configure" toState="stopped" />
      <leaf agent="agent-1" fabric="glu-dev-1" mountPoint="/sample/i003" name="Run [start] phase for [/sample/i003] on [agent-1]" scriptAction="start" toState="running" />
    </sequential>
  </sequential>
</plan>

Only /sample/i003 has the backend tag

Yan
Reply | Threaded
Open this post in threaded view
|

Re: Set dependency Sequence for Parallel deployment Plans

chaoyang ma
Hi, I found the reason. In the console, the tag filter is working. However in the command line, the filter only works for "status" command, not "deploy" command according to the help information below.

  -s FILTER, --systemFilter=FILTER
                        Filter in DSL sytax for filtering the model.
                        Applicable only with "status" command. See "Filter
                        Syntax" section here:
                        https://github.com/linkedin/glu/wiki/Console

I am using glu 4.6.2. Does this improved in later version?
Reply | Threaded
Open this post in threaded view
|

Re: Set dependency Sequence for Parallel deployment Plans

frenchyan
Administrator
It is a known bug in 4.6.2 that has been fixed in later version. That being said you can fix it yourself as the fix is trivial (python):

Simply edit the file console-cli/bin/console.py

to change system_filter into systemFilter at line 149 (as shown in the git commit https://github.com/pongasoft/glu/commit/8ef27d4e1500fa1d65e0e81811de85fad509c227)

Or you can upgrade to 4.7.2 

Yan