Standard practice for Glu to manage multiple apps and environments

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Standard practice for Glu to manage multiple apps and environments

Ed Tretriluxana
I don't know if this question has been asked. I couldn't find an answer to it so I am posting it here.

We're looking into Glu right now to see how we can use it to manage multiple apps deployment to multiple environments. Basically we have N number of apps each of which has multiple environments (e.g. DEV, IT, UAT, QA, PROD). Each environment of each app usually resides on its own machine but it may also reside on the same machine as other environments. For example, App A and App B may share the same machine M for their IT environments. They just run on a different port. In prod, App A and B have their own set of machines.

Given that rough sketch, after reading through the docs and play around with tutorial, I'm not sure/clear how Glu works with it. Glu has its own notion of Fabric which seems closest related to a group of machine. However, an agent can belong to only one Fabric. The same restriction of one-Fabric-only goes with the static model if I'm not mistaken.

That said, to apply Glu, are we saying that we need to create a different Fabric for each and every App/Env combination. For example, let's say we have 10 apps each of which has 5 environments. Are we to create 50 different Fabrics? Also, for the case where a machine hosts more than one environment, does it mean we need to install more than one agent on that machine?

Please shed some light how Glu is intended to work in this scenario. Thanks.
Reply | Threaded
Open this post in threaded view
|

Re: Standard practice for Glu to manage multiple apps and environments

frenchyan
Administrator

I am currently traveling and have limited Internet access. The short answer is, no you would not need 50 fabrics but 5: 1 per environment.

I will post more details tomorrow after I am back.

Yan

On Sep 9, 2013 10:55 AM, "Ed Tretriluxana [via glu]" <[hidden email]> wrote:
I don't know if this question has been asked. I couldn't find an answer to it so I am posting it here.

We're looking into Glu right now to see how we can use it to manage multiple apps deployment to multiple environments. Basically we have N number of apps each of which has multiple environments (e.g. DEV, IT, UAT, QA, PROD). Each environment of each app usually resides on its own machine but it may also reside on the same machine as other environments. For example, App A and App B may share the same machine M for their IT environments. They just run on a different port. In prod, App A and B have their own set of machines.

Given that rough sketch, after reading through the docs and play around with tutorial, I'm not sure/clear how Glu works with it. Glu has its own notion of Fabric which seems closest related to a group of machine. However, an agent can belong to only one Fabric. The same restriction of one-Fabric-only goes with the static model if I'm not mistaken.

That said, to apply Glu, are we saying that we need to create a different Fabric for each and every App/Env combination. For example, let's say we have 10 apps each of which has 5 environments. Are we to create 50 different Fabrics? Also, for the case where a machine hosts more than one environment, does it mean we need to install more than one agent on that machine?

Please shed some light how Glu is intended to work in this scenario. Thanks.


If you reply to this email, your message will be added to the discussion below:
http://glu.977617.n3.nabble.com/Standard-practice-for-Glu-to-manage-multiple-apps-and-environments-tp4026061.html
To start a new topic under glu, email [hidden email]
To unsubscribe from glu, click here.
NAML
Reply | Threaded
Open this post in threaded view
|

Re: Standard practice for Glu to manage multiple apps and environments

sodul
In reply to this post by Ed Tretriluxana
Hi Ed,

It depends a little bit of the control and flexibility you want/require. Something to keep in mind is that you 'should' have one host user per agent, but it is not a hard requirement (sudo can work around most of that).

An agent is also restricted to a single fabric indeed, but you can run multiple agents on a single machine.

Personally the way I would implement it in your case is to have 5 fabrics, and probably 5 different users for the agents of each fabric to run on: the DEV fabric would run the agent as the agent_dev user, the QA fabric would have agents running as the agent_qa user, etc... This would ensure that your apps from one fabric would not interfere on each other (or at least limit potential impact on each other). From there you define in your model what app goes on what machines, and provide a way during the configuration step to not overlap on the ports (maybe define ranges dedicated to each fabric: Dev get 10000-19999, QA get 20000-29999, etc...). You can from there easily define a model for each fabric and state where each app goes in each fabric, including apps that go on multiple agents, without having to worry about the other fabrics. They can all share the same zookeeper instances but you will have to make sure each agent name is unique across all fabrics. Use something like '<fabric>-<hostname>'.

To build your models, say you have fabrics A and B on hosts h1, h2, h3 for apps x, y, z.

The model for fabric A could be defined as:
{"entries": [
    {"agent": "A-h1", "mountpoint": "/x"},
    {"agent": "A-h1", "mountpoint": "/y"},
    {"agent": "A-h1", "mountpoint": "/z"},
    {"agent": "A-h2", "mountpoint": "/y"},
    {"agent": "A-h3", "mountpoint": "/z"},
]}
Where app x only runs on host h1, app y is running on h1 and h2 while app z is running on h1 and h3.

For Fabric B it could be:
{"entries": [
    {"agent": "B-h1", "mountpoint": "/x"},
    {"agent": "B-h1", "mountpoint": "/y"},
    {"agent": "B-h1", "mountpoint": "/z"},
    {"agent": "B-h2", "mountpoint": "/y"},
    {"agent": "B-h3", "mountpoint": "/x"},
]}
Where app x runs on host h1 and h2, app y is running on h1 and h2 while app z is running only on h1.

This gives you great flexibility while the console's dashboard show you at a glance where all the apps for a given environment (fabric) run.

Now I actually recommend that you use virtualization (XEN is free to use, and works great) so that you can actually assign all machines in a fabric to virtual hosts dedicated to that fabric. I also recommend that you write a custom script to help generate/update your model. Something that would tie with your CI system.
Reply | Threaded
Open this post in threaded view
|

Re: Standard practice for Glu to manage multiple apps and environments

Ed Tretriluxana
Hi Sodul,

Your response is extremely helpful. Now I can see how this thing will be done in Glu and feel much less intimidated with the thought of fabric proliferation.

I was also thinking of the need to build a layer on top of Glu to maintain the models and other stuff. What you were suggesting exactly confirms that I wasn't misunderstanding the concepts.

This is really great! Highly appreciate your response.
Reply | Threaded
Open this post in threaded view
|

Re: Standard practice for Glu to manage multiple apps and environments

Ed Tretriluxana
In reply to this post by frenchyan
Yan, thanks so much for dropping a message. I'll wait to hear more from you.

Reply | Threaded
Open this post in threaded view
|

Re: Standard practice for Glu to manage multiple apps and environments

sodul
In reply to this post by Ed Tretriluxana
Reply | Threaded
Open this post in threaded view
|

Re: Standard practice for Glu to manage multiple apps and environments

frenchyan
Administrator
In reply to this post by Ed Tretriluxana
Hi Ed

I think in the end Stephane summarized it pretty well. You simply define 1 fabric per environment as they are usually segregated and also have different requirements (like you were describing in IT App A and App B run on the same host but in prod they would be on different hosts).

Each fabric (IT, PROD, etc...) has its own model describing where each app needs to be deployed... so a given app (the same) can be deployed on different hosts or the same host depending on the fabric.

It is highly recommended to model your own environment outside of glu with your own concepts and then map it to the glu json model. For the mapping you can either write a small script that process your model in whatever format you want (xml, yaml, json, etc...) and generate the correct json format for glu or if you are familiar with groovy you can use the newly introduced json groovy syntax which one way to think about it let you describe your own model in your own term and then use groovy logic to generate the json... An example of this second option can be seen here: https://github.com/pongasoft/glu/blob/master/console/org.linkedin.glu.console-server/src/cmdline/resources/glu/repository/systems/sample-webapp-system.json.groovy where you can see concepts outside of glu (like instance, cluster, products,... ) that gets then mapped in the json format through groovy dsl.

Hope this helps
Yan
Reply | Threaded
Open this post in threaded view
|

Re: Standard practice for Glu to manage multiple apps and environments

Ed Tretriluxana
Thanks so much Jan. We are now building a prototype using the recommendations from both of you. Really appreciate quick and helpful responses. Will post more questions later if I have any.