ZooKeeper question

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

ZooKeeper question

gfliker
Hi,

We started using the basic zookeeper setup bundled with the glu package.

Now that were on a more production stage i want to set up a better zookeeper cluster.

So my question are:

- Is there anything specific in the zookeeper bundle in the download or i can use the Apache one.

- What will we lost if we start using the new/fresh zookeeper cluster ?

Thanks

Gil

Outbrain Ops
 
Reply | Threaded
Open this post in threaded view
|

Re: ZooKeeper question

frenchyan
Administrator
Hi Gil

Regarding your first question, it is actually recommended to run zookeeper in a cluster (and for that I would strongly suggest to use the one directly from apache and follow their instructions on how to set it up). From my experience (and after discussing with the ZooKeeper authors), it is best/recommended to set it up with 3 servers in the cluster (and this is something pretty easy to set up). This is the way it is setup at LinkedIn.

There is nothing special coming with the one bundled with glu except that it has a shell script wrapper around it, but that's it. So plain, old vanilla ZooKeeper will work. Make sure though that it is at least v 3.3.1

I am not sure I understand the second question entirely. What do you mean by new/fresh zookeeper ? As opposed to what ?

I can see 2 scenarios:
1) In this scenario you are reusing the one that you were using so far.
You currently have 1 zookeeper and you are going to add 2 to the cluster then I think nothing will be lost: you just add the 2 new zookeeper and configure them to see the first one, then you need to stop the first one and configure it to see the 2 new ones. And restart it.

2) In this scenario you are starting from scratch (not reusing the one you were using so far). You will need to load the (agent) configuration in zookeeper.

-----
In any case, you will have to tell the agents about the cluster and you usually start them with the cluster on the command line: -z zkhost1:2181;zkhost2:2181;zkhost3:2181

Hope this helps
Yan


Reply | Threaded
Open this post in threaded view
|

Re: ZooKeeper question

rantav
Hi Yan, what Gil was asking is what you answered at 2, we have a live deployments of a few dozens of agents and now we want to wipe out ZK and start fresh. So your answer is that all we have to do is reload all the agent's configuration, that's all. 
Thanks

On Tue, Jan 25, 2011 at 4:40 AM, frenchyan [via glu] <[hidden email]> wrote:
Hi Gil

Regarding your first question, it is actually recommended to run zookeeper in a cluster (and for that I would strongly suggest to use the one directly from apache and follow their instructions on how to set it up). From my experience (and after discussing with the ZooKeeper authors), it is best/recommended to set it up with 3 servers in the cluster (and this is something pretty easy to set up). This is the way it is setup at LinkedIn.

There is nothing special coming with the one bundled with glu except that it has a shell script wrapper around it, but that's it. So plain, old vanilla ZooKeeper will work. Make sure though that it is at least v 3.3.1

I am not sure I understand the second question entirely. What do you mean by new/fresh zookeeper ? As opposed to what ?

I can see 2 scenarios:
1) In this scenario you are reusing the one that you were using so far.
You currently have 1 zookeeper and you are going to add 2 to the cluster then I think nothing will be lost: you just add the 2 new zookeeper and configure them to see the first one, then you need to stop the first one and configure it to see the 2 new ones. And restart it.

2) In this scenario you are starting from scratch (not reusing the one you were using so far). You will need to load the (agent) configuration in zookeeper.

-----
In any case, you will have to tell the agents about the cluster and you usually start them with the cluster on the command line: -z zkhost1:2181;zkhost2:2181;zkhost3:2181

Hope this helps
Yan





If you reply to this email, your message will be added to the discussion below:
http://glu.977617.n3.nabble.com/ZooKeeper-question-tp2322812p2326110.html
To start a new topic under glu, email [hidden email]
To unsubscribe from glu, click here.



--
/Ran

Gil
Reply | Threaded
Open this post in threaded view
|

Re: ZooKeeper question

Gil
Hi,

Thx for the prompt response.
And im referring scenario number 2 so i got my answer.

Gil
Reply | Threaded
Open this post in threaded view
|

Re: ZooKeeper question

frenchyan
Administrator
Cool. I am glad I answered it. By the way, once you have the agents in production they can be upgraded to a later version directly from the console. I will start 'publishing' the agent upgrade package as well so that you don't have to build it yourself.

Yan
Gil
Reply | Threaded
Open this post in threaded view
|

Re: ZooKeeper question

Gil
Nice feature,

We will use it for sure.

Gil
Reply | Threaded
Open this post in threaded view
|

Re: ZooKeeper question

chaoyang ma
In reply to this post by rantav
Hi Yan:

Can you explain a bit more on how to do this "make the new zookeeper to see another one" in the below paragraph. Actually I am trying to setup 3 zookeeper as a cluster, but did not know where to start. If you can explain a bit more on the steps, that would be great.

"I can see 2 scenarios:
1) In this scenario you are reusing the one that you were using so far.
You currently have 1 zookeeper and you are going to add 2 to the cluster then I think nothing will be lost: you just add the 2 new zookeeper and configure them to see the first one, then you need to stop the first one and configure it to see the 2 new ones. And restart it. "
Reply | Threaded
Open this post in threaded view
|

Re: ZooKeeper question

frenchyan
Administrator
This is explained in the documentation: http://pongasoft.github.io/glu/docs/latest/html/production-setup.html#step-2-setup-zookeeper which links to the official documentation for ZooKeeper. But in a nutshell if you have 3 hosts running ZooKeeper (h1, h2 and h3), then you configure it this way:

on h1:
data/myid => contains 1
conf/zoo.cfg => at the end you need:
server.1=h1:2888:3888
server.2=h2:2888:3888
server.3=h3:2888:3888

on h2:
data/myid => contains 2
conf/zoo.cfg => at the end you need:
server.1=h1:2888:3888
server.2=h2:2888:3888
server.3=h3:2888:3888

on h3:
data/myid => contains 3
conf/zoo.cfg => at the end you need:
server.1=h1:2888:3888
server.2=h2:2888:3888
server.3=h3:2888:3888

Yan