GlusterFS 설치하기 (v3.1.2)
출처 : http://redmine.nehome.net/redmine/activity?from=2011-05-04
Install – GlusterFS v3.1.2-1 (by RPMs)
User documentation
2011-04-26
테스트 환경
CentOS 5.4(x86_64) 3대 + CentOS 5.4(i386) 3대 = 총 6-VM으로 테스트 진행
Hostname = IP
GFS1 = 192.168.100.31 (x86_64) GFS2 = 192.168.100.32 (x86_64) GFS3 = 192.168.100.33 (x86_64) GFS4 = 192.168.100.34 (i386) GFS5 = 192.168.100.35 (i386) GFS6 = 192.168.100.36 (i386)
6대 모두 하나의 스위치에 연결
GlusterFS로 사용될 Physicial HDD
200GB
/dev/sdb -> "fdisk" -> /dev/sdb1 -> "mount" -> /data
본 문서의 기본 작업 디렉토리는 /usr/local/src 로 함.
처음 테스트에는 4개 Node만 이용하고, 2개 노드는 운영중 확장/추가 테스트에 이용.
6-VM 공통 OS환경
편의상 /etc/hosts에 모든 GlusterFS Node들의 host정보 등록
192.168.100.31 GFS1 192.168.100.32 GFS2 192.168.100.33 GFS3 192.168.100.34 GFS4 192.168.100.35 GFS5 192.168.100.36 GFS6
iptables off # chkconfig iptables off # service iptables stop portmap on # chkconfig portmap on # service portmap start nfs-server off # chkconfig nfs off # chkconfig nfslock off # service nfs stop # service nfslock stop mount point 생성 # mkdir /data /mnt/nfs /mnt/glusterfs fuse 모듈 로딩 처리 # modprobe fuse # echo "modprobe fuse" >> /etc/rc.local Install GlusterFS (on all GlusterFS Nodes) Download SourceRPM http://download.gluster.com/pub/gluster/glusterfs/3.1/LATEST/ # wget http://download.gluster.com/pub/gluster/glusterfs/3.1/3.1.2/CentOS/glusterfs-core-3.1.2-1.x86_64.rpm # wget http://download.gluster.com/pub/gluster/glusterfs/3.1/3.1.2/CentOS/glusterfs-debuginfo-3.1.2-1.x86_64.rpm # wget http://download.gluster.com/pub/gluster/glusterfs/3.1/3.1.2/CentOS/glusterfs-fuse-3.1.2-1.x86_64.rpm # wget http://download.gluster.com/pub/gluster/glusterfs/3.1/3.1.2/CentOS/glusterfs-rdma-3.1.2-1.x86_64.rpm
Install Dependant RPM # yum -y install libibverbs-devel Install GlusterFS RPMs on x86_64 : # rpm -Uvh glusterfs-* Run GlusterFS Daemon # service glusterd start Probe Peer Node (On GFS1 Node) # gluster peer status No peers present # gluster peer probe gfs1 Probe on localhost not needed # gluster peer probe gfs2 Probe successful # gluster peer probe gfs3 Probe successful # gluster peer probe gfs4 Probe successful # gluster peer status Number of Peers: 3 Hostname: gfs2 Uuid: 8c5f1b10-4f15-4305-86b9-260e0dc2b8dc State: Peer in Cluster (Connected) Hostname: gfs3 Uuid: ba864454-e73e-4317-b90c-7f7c6142eef3 State: Peer in Cluster (Connected) Hostname: gfs4 Uuid: 9c01a6b7-c5c0-4bcd-af75-1b7fd1f9e6c3 State: Peer in Cluster (Connected) Create GlusterFS Volume and Start Volume Create a 4-Node(GFS1, GFS2, GFS3, GFS4) distributed (replicated) volume with a two-way mirror (Run On Any Nodes(4-Nodes)) # gluster volume info No volume present # gluster volume create test-vol replica 2 gfs1:/data/ gfs2:/data/ gfs3:/data/ gfs4:/data/ Creation of volume test-vol has been successful. Please start the volume to access data. # gluster volume info Volume Name: test-vol Type: Distributed-Replicate Status: Created Number of Bricks: 2 x 2 = 4 Transport-type: tcp Bricks: Brick1: gfs1:/data Brick2: gfs2:/data Brick3: gfs3:/data Brick4: gfs4:/data # gluster volume start test-vol Starting volume test-vol has been successful * test-vol : Volume Name Mount by NFS/GlusterFS Befor Mount # df -Th Filesystem Type Size Used Avail Use% Mounted on /dev/sda3 ext3 61G 4.8G 53G 9% / /dev/sda1 ext3 996M 46M 899M 5% /boot tmpfs tmpfs 187M 0 187M 0% /dev/shm /dev/sdb1 ext3 197G 188M 187G 1% /data Mount by NFS (run on any...) # mount -t nfs gfs1:/test-vol /mnt/nfs (gfs1 = gfs2 or gfs3 gfs4...) df -Th Filesystem Type Size Used Avail Use% Mounted on /dev/sda3 ext3 61G 4.8G 53G 9% / /dev/sda1 ext3 996M 46M 899M 5% /boot tmpfs tmpfs 187M 0 187M 0% /dev/shm /dev/sdb1 ext3 197G 188M 187G 1% /data gfs1:/test-vol nfs 394G 375M 374G 1% /mnt/nfs Mount by GlusterFS (run on any...) # lsmod | grep fuse fuse 83057 0 # mount -t glusterfs gfs1:/test-vol /mnt/glusterfs (gfs1 = gfs2 or gfs3 gfs4...) # df -Th Filesystem Type Size Used Avail Use% Mounted on /dev/sda3 ext3 61G 4.8G 53G 9% / /dev/sda1 ext3 996M 46M 899M 5% /boot tmpfs tmpfs 187M 0 187M 0% /dev/shm /dev/sdb1 ext3 197G 188M 187G 1% /data gfs1:/test-vol nfs 394G 375M 374G 1% /mnt/nfs glusterfs#gfs1:/test-vol fuse 394G 375M 374G 1% /mnt/glusterfs Add-Node (or Brick) GFS5, GFS6에 각각 존재하는 200GB 용량을 Online상테에서 확장 # gluster peer probe gfs5 Probe successful # gluster peer probe gfs6 Probe successful # gluster peer status Number of Peers: 5 Hostname: gfs2 Uuid: 8c5f1b10-4f15-4305-86b9-260e0dc2b8dc State: Peer in Cluster (Connected) Hostname: gfs3 Uuid: ba864454-e73e-4317-b90c-7f7c6142eef3 State: Peer in Cluster (Connected) Hostname: gfs4 Uuid: 9c01a6b7-c5c0-4bcd-af75-1b7fd1f9e6c3 State: Peer in Cluster (Connected) Hostname: gfs5 Uuid: cd9a8813-f9e4-4613-a24f-43ac907f2908 State: Peer in Cluster (Connected) Hostname: gfs6 Uuid: fc3b6b09-2143-459a-9886-92b271a779b5 State: Peer in Cluster (Connected) # df -Th Filesystem Type Size Used Avail Use% Mounted on /dev/sda3 ext3 61G 4.8G 53G 9% / /dev/sda1 ext3 996M 46M 899M 5% /boot tmpfs tmpfs 187M 0 187M 0% /dev/shm /dev/sdb1 ext3 197G 188M 187G 1% /data gfs1:/test-vol nfs 394G 375M 374G 1% /mnt/nfs glusterfs#gfs1:/test-vol fuse 394G 375M 374G 1% /mnt/glusterfs # gluster volume add-brick test-vol gfs5:/data/ gfs6:/data/ Add Brick successful # df -Th Filesystem Type Size Used Avail Use% Mounted on /dev/sda3 ext3 61G 4.8G 53G 9% / /dev/sda1 ext3 996M 46M 899M 5% /boot tmpfs tmpfs 187M 0 187M 0% /dev/shm /dev/sdb1 ext3 197G 188M 187G 1% /data gfs1:/test-vol nfs - - - - /mnt/nfs glusterfs#gfs1:/test-vol fuse 394G 375M 374G 1% /mnt/glusterfs # df -Th Filesystem Type Size Used Avail Use% Mounted on /dev/sda3 ext3 61G 4.8G 53G 9% / /dev/sda1 ext3 996M 46M 899M 5% /boot tmpfs tmpfs 187M 0 187M 0% /dev/shm /dev/sdb1 ext3 197G 188M 187G 1% /data gfs1:/test-vol nfs 591G 563M 561G 1% /mnt/nfs glusterfs#gfs1:/test-vol fuse 394G 375M 374G 1% /mnt/glusterfs Man Page of gluster # man gluster 참조 권장
Reading your article helped me a lot and I agree with you. But I still have some doubts, can you clarify for me? I’ll keep an eye out for your answers.
Can you be more specific about the content of your article? After reading it, I still have some doubts. Hope you can help me.