Lustre LVM Snopshot 구현 하기
Lustre LVM Snopshot 구현하기
작성일 : 2016-06-01
작성자 : 서진우 (alang@clunix.com
– lustre 구성 요소 확인
mdt : /dev/sdb
ost1: /dev/sdc
ost2: /dev/sdd
ost3: /dev/sde
– mdt, ost 해당 device 를 lvm 으로 구성
// mdt lvm 생성
# pvcreate /dev/sdb
# pvdisplay
# vgcreate mdtgrp /dev/sdb
# vgdisplay
# lvcreate -L1.5T -nMDT0 mdtgrp
# lvdisplay
// ost lvm 생성
# pvcreate /dev/sdc /dev/sdd /dev/sde
vgcreate ost1grp /dev/sdc
vgcreate ost2grp /dev/sdd
vgcreate ost3grp /dev/sde
lvcreate -L1.5T -nOST1 ost1grp
lvcreate -L1.5T -nOST2 ost2grp
lvcreate -L1.5T -nOST3 ost3grp
mkfs.lustre –reformat –writeconf –fsname=gcfs –mgs –mdt –servicenode=master01-ib@o2ib –mgsnode=master01-ib@o2ib –index=0 /dev/mdtgrp/MDT0
mkfs.lustre –reformat –writeconf –fsname=gcfs –ost –servicenode=master01-ib@o2ib –mgsnode=master01-ib@o2ib –index=1 /dev/ost1grp/OST1
mkfs.lustre –reformat –writeconf –fsname=gcfs –ost –servicenode=master01-ib@o2ib –mgsnode=master01-ib@o2ib –index=2 /dev/ost2grp/OST2
mkfs.lustre –reformat –writeconf –fsname=gcfs –ost –servicenode=master01-ib@o2ib –mgsnode=master01-ib@o2ib –index=3 /dev/ost3grp/OST3
vi /etc/ldev.conf
master01 master01 gcfs-MDT0000 /dev/mdtgrp/MDT0
master01 master01 gcfs-OST0001 /dev/ost1grp/OST1
master01 master01 gcfs-OST0002 /dev/ost2grp/OST2
master01 master01 gcfs-OST0003 /dev/ost3grp/OST3
/etc/rc.d/init.d/lustre start
lustre client 서버에서 gcfs 볼륨에 mount 하고, 데이터를 저장한다.
– Backing up New/Changed Files to the Backup File System
modprobe dm-snapshot
// 스냅샵을 찍는다. 이 시점까지 저장된 데이터가 아래 snapshot lv 에 보관된다.
lvcreate -L5G -s -n MDT0-1 /dev/mdtgrp/MDT0
lvcreate -L5G -s -n OST1-1 /dev/ost1grp/OST1
lvcreate -L5G -s -n OST2-1 /dev/ost2grp/OST2
lvcreate -L5G -s -n OST3-1 /dev/ost3grp/OST3
만일 용량이 부족하면 lvextend 명령으로 용량을 늘려준다.
lvextend -L +5G /dev/mdtgrp/MDT0b
lvdisplay
lvs
– Restoring the File System From a Snapshot
일단 lustre client 에서 gcfs volume 에 저장된 모든 데이를 지워버린다.
그런 후..snapshot lv 에 lustre fs 를 만든다.
tunefs.lustre –reformat –fsname=gcfsback –writeconf /dev/mdtgrp/MDT0-1
tunefs.lustre –reformat –fsname=gcfsback –writeconf /dev/ost1grp/OST1-1
tunefs.lustre –reformat –fsname=gcfsback –writeconf /dev/ost2grp/OST2-1
tunefs.lustre –reformat –fsname=gcfsback –writeconf /dev/ost3grp/OST3-1
[root@master01 ~]# mkdir -p /mnt/lustre/snapshot/mdt
[root@master01 ~]# mkdir -p /mnt/lustre/snapshot/ost1
[root@master01 ~]# mkdir -p /mnt/lustre/snapshot/ost2
[root@master01 ~]# mkdir -p /mnt/lustre/snapshot/ost3
mount -t ldiskfs /dev/mdtgrp/MDT0b /mnt/lustre/snapshot/mdt
mount -t ldiskfs /dev/ost1grp/OST1b /mnt/lustre/snapshot/ost1
mount -t ldiskfs /dev/ost2grp/OST2b /mnt/lustre/snapshot/ost2
mount -t ldiskfs /dev/ost3grp/OST3b /mnt/lustre/snapshot/ost3
각 mount 볼륨에서 last_rcvd 디렉토리를 제거한다.
rm -f /mnt/lustre/snapshot/mdt/last_rcvd
rm -f /mnt/lustre/snapshot/ost1/last_rcvd
rm -f /mnt/lustre/snapshot/ost2/last_rcvd
rm -f /mnt/lustre/snapshot/ost3/last_rcvd
umount /mnt/lustre/snapshot/mdt
umount /mnt/lustre/snapshot/ost1
umount /mnt/lustre/snapshot/ost2
umount /mnt/lustre/snapshot/ost3
mount -t lustre /dev/mdtgrp/MDT0-1 /mnt/lustre/snapshot/mdt
mount -t lustre /dev/ost1grp/OST1-1 /mnt/lustre/snapshot/ost1
mount -t lustre /dev/ost2grp/OST2-1 /mnt/lustre/snapshot/ost2
mount -t lustre /dev/ost3grp/OST3-1 /mnt/lustre/snapshot/ost3
lustre client mount
# mount -t lustre master01-ib@o2ib:/gcfsback /home
/home 밑에 데이터가 보관되어 있는 것을 확인할 수 있을 것이다.
lvconvert 를 통한 복구 ..
복구시에는 반드시 해당 volume에 mount 상태가 아니여만 함.
lvconvert –merge /dev/mdtgrp/MDT0-1
lvconvert –merge /dev/ost1grp/OST1-1
lvconvert –merge /dev/ost2grp/OST2-1
lvconvert –merge /dev/ost3grp/OST3-1
간혹 origin volume 이 mount 상태에서 merge 를 하면
Can’t merge over open origin volume 가 뜸..
즉 다음번 LV 활성 시 자동으로 스냅샷 복구가 진행되도록 예약됨
이때는 ..LV를 비활성화 후 활성화를 시킴.
# lvchange -a n /dev/mdtgrp/MDT0
# lvchange -a y /dev/mdtgrp/MDT0
