How does upgrading software deployed on an Agent work?

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

How does upgrading software deployed on an Agent work?

oldgluuser
Specifically I am interested in software deployed to an agent, not Glu itself.

So just looking at the tutorial, the specific version of Jetty is defined in the "Static Model".  However, this changes with every Jetty release.  Let's say two point releases go by (from say 8.1.10 to 8.3.2) and I want to upgrade this.  But since this is just one of huge number of version upgrades, can I get Glu to tell me about this?  I read that the info is used for "deltas", but is there some automatic (or semi-automatic) way find out about such updates?

thanks!
Reply | Threaded
Open this post in threaded view
|

Re: How does upgrading software deployed on an Agent work?

frenchyan
Administrator
I am not sure I understand the question correctly. But if you are asking whether glu monitors that there is a more recent version of jetty that has been made available on the jetty web site, then clearly the answer is no. Actually glu does not know much about software or what you do with it. It is very opaque:

=> you essentially provide a map (initParameters) of key value pairs (nesting is ok). The meaning of the keys and value is up to you and is made available in your glu script for you to do whatever you want with it. The only thing you know is that glu will use this map to determine the delta. But that is where glu stops.

Yan
Reply | Threaded
Open this post in threaded view
|

Re: How does upgrading software deployed on an Agent work?

oldgluuser
I think you partially understand my question, which is a start!  So first, maybe Jetty is a bad example because it is, well "Jetty"!   In the tutorial, the example initParam is called "skeleton" which points to a local web site.   If I were using Glu to deploy one of my own items, let's say a war file, could it point to a local Nexus or Artifactory repository and detect that a new version was available?  

If not, what if I augmented the console to include a groovy script that performed such and updated the "static model", what steps would need to proceed to get the new version downloaded to the appropriate agents?

thanks!
Reply | Threaded
Open this post in threaded view
|

Re: How does upgrading software deployed on an Agent work?

sodul
I use Artifactory to store my binaries and deploy them through Glu, but I still only use static versions.

I do not recommend that you go with the approach that you mentioned but you could achieve that by abusing the monitor step of the groovy script. The way you would do it is setup your model so that the skeleton has enough information to point to artifactory and specify the binary name and extension. Duting the install step your groovy script is smart enough to query the artifactory rest api to get the latest release version of that binary and install it. Store that version information under the mount point. During the monitor step of the groovy script read that stored version, redo a query to artifactory, and if different store that to an exported variable such as scriptDelta (groovy scripts export their vars to zookeeper by default). In the console dashboard add the new scriptDelta column. Whenever there is a difference, it will show up in that column.

Again … I do not really recommend doing this since you will no longer have the advantage of the console's delta engine and you will have to manually filter what needs to be deployed.
Reply | Threaded
Open this post in threaded view
|

Re: How does upgrading software deployed on an Agent work?

frenchyan
Administrator
I second Stephane in not recommending this approach. The way I would recommend doing it is the following:

- define the json model that describes your system to contain static versions (ex: war: "http://artifactory/a/b/c/1.0.0/my.war"), exactly like in the tutorial model/example
- write an external program that uses the rest api (of the console) to query the static model (/model/static).
- parse the json model and for each war, "ping" artifactory to see if there is a new version for this war
- if any new version, then regenerate/replace in the model the ones that have changed
- upload the new model to glu  (rest api)
- trigger a deploy (rest api) => glu will redeploy ONLY the ones that have changed

Once you have this program written, then it is up to you to decide when to run it:

- you could simply run it in a forever loop (while(true) { sleep 30mn, run program }) or as a cron job
- you could also invoke it part of a ci build => if successful run this program (usually the ci build is the one which deploys the result of your build in artifactory on successful build/test runs)

Yan
Reply | Threaded
Open this post in threaded view
|

Re: How does upgrading software deployed on an Agent work?

oldgluuser
Here is what I understand thus far.  There is a basic set of steps that can be followed to update/regenerate/replace the model.  Then a deploy can be initiated (manually by the User if desired), to update the agents.

What I don't understand is that both of you recommend not doing this "in Glu" - which for me I think that would mean in a groovy script somewhere.  The only reason I can think of why this would be undesirable is that there is no periodic way to execute such - is that correct?  Doing it as an external program gives me more to release, more to deploy, more to monitor, more fault tolerance issues/failure scenarios, just seems like more headache!

As for the CI build suggestion, that would not apply.  The use case I am considering would be to deploy an instance at a Client site - it would "reach back" to a private Artifactory site to see if any new releases have been issued and would then apply those either automatically or upon demand (depending upon the option selected during initial install).
Reply | Threaded
Open this post in threaded view
|

Re: How does upgrading software deployed on an Agent work?

frenchyan
Administrator
These are recommendations not what you have to do. At the end of the day glu is a platform and is very flexible and let you build whatever you want on top of it. It is not a full blown turn key application and has never been designed to be. For example, it was a conscious design choice that glu would never do anything on its own. Even if you upload a new model with changes in glu, glu will tell you there is a mismatch (a delta) but will not deploy anything until you instruct it to do so. That being said, because of those "primitive" building blocks (+ rest api, plugins, etc...) it is very easy to "build" a solution where the deployment is automatic. I emphasize "build", because you still need to write some code in some form or another to use those building blocks.

If your concern is the fact that you need to have this external program that needs to be released and deployed, monitored, etc,... there are several ways you can go about it:
- one possible way, is that you could actually package this external program as a glu script, use the "timers" feature to have the main loop run on a regular basis... glu itself will then deploy/monitor this glu script like any other
- another way would be to 'enhance' the console webapp to add the logic (grails has the quartz plugin (http://grails.org/plugin/quartz2) that would let you run your code on a regular basis). => Since it is not supported natively, you would have to fork glu to add your own code. I can also look at a way to enhance the  console to let you "plugin" your own (any kind of) jobs. That would obviously mean a change in glu, but I can see other users needing the ability to have scheduled jobs running on a regular basis.

Yan
Reply | Threaded
Open this post in threaded view
|

Re: How does upgrading software deployed on an Agent work?

oldgluuser
As Glu is still quite new to me, I am just "trying to do the right thing".   I have no concern whatsoever about having to write a little (or even a lot!) of code to make it functional for what is needed.  I just want to leverage expertise to avoid doing something wrong as one of my first steps!  

My concern was what you stated about the external program.  What I was thinking/hoping was to parallel the timer usage in the Jetty example script, but to have it run some groovy code (in a separate file/class) that would do the check.  From the initial responses, I thought this direction was advised against and was confused about the exact semantics of that.  BTW, I was able to "manually" do the steps I envision via the console UI against the tutorial and it all seems quite clean!
Reply | Threaded
Open this post in threaded view
|

Re: How does upgrading software deployed on an Agent work?

sodul
In reply to this post by oldgluuser
Actually I've done this in the past for full CI by driving this in Jenkins.

Here is the workflow:
 - developer commit code
 - Jenkins detect the change and trigger a build
 - binary get published to artifacts repository and trigger next job
 - next job is to create a list of artifacts parable by other tools but mainly the Extended Parameters Plugin for Jenkins
 - CI deployment job is triggered by the updated artifacts list
 - CI deployment job take latest versions of binaries from the list
 - CI deployment job calls a custom glu_model.py script feeding latest versions and target fabric to update
 - glu_model.py reads the old model from the console, generate a new one with the new versions and push it to the console
 - glu_model.py trigger the deploy action on the console (optional)
 - CI environment is updated

The only manual step here was for the developer to commit his change. The deployment jobs do not need to be full CI and can be triggered on demand, with the option to do rolling deployments, full redeploy or simply update the model. I cannot share glu_model.py since I have not finished writing it at my current workplace and the logic is always very specific to your own Glu implementation (the model entries are very custom in our case).

I recommend that you explore using Jenkins to do your builds but also trigger anything that need to be automated. Jenkins has hundreds of plugins, can publish to artifactory (even without maven),

Consider Glu a Platform for you to add your own custom workflow. Using Glu as a deployment platform is working well, link it to an automation platform such as Jenkins and it will make wonders.

You will have to write code. Something like blu_model.py took about 700 lines, the groovy script (I've shared that last year on this forum) about 200 lines, glu.py (custom logic called by the groovy script) about 700 lines (can do file searches on artifactory), the list artifacts script a few hundred lines. So overall 2000 to 3000 lines of script to have a full unattended Continuous Deployment solution.
Reply | Threaded
Open this post in threaded view
|

Re: How does upgrading software deployed on an Agent work?

frenchyan
Administrator
In reply to this post by oldgluuser
"BTW, I was able to "manually" do the steps I envision via the console UI against the tutorial and it all seems quite clean! "

Can you describe what you did? I am just curious what it entails.

Thanks
Yan