[클러스터] globus 3.95 설치 하기
Install_Globus3.9.5
작성자 : 김형진 (qanii@kisti.re.kr)
최종 수정일 : 2005-03-28
이 문서는 Globus4.0 Beta 버전인 Globus3.9.5의 설치방법과 간단한 사용 방법을 담고 있습니다.
——————————————————————————–
globus 유저 생성
각각의 노드에서 :
root#groupadd -gid 504 globus
root#useradd -m -uid 504 -g globus globus
참고1. : gid와 uid값은 임의로 정하며, redhat경우 일반 유저 생성시 500번부터 순차적으로 만들어진다.
참고2. : 각각의 노드간의 uid와 gid는 꼭 일치시켜야 한다.
JDK 설치
Version : 1.4.2+
다운로드 : http://javashoplm.sun.com/ECom/docs/Welcome.jsp?StoreId=22&PartDetailId=j2sdk-1.4.2_07-oth-JPR&SiteId=JSC&TransactionId=noreg
설치
root권한으로 다음과 같은 명령어 실행한다.
root# ./j2sdk-1_4_2_07-linux-i586.bin
root# mkdir /usr/java
root# mv j2sdk1.4.2_07 /usr/java
환경변수 설정
환경 변수 설정을 위해, globus를 사용할 계정마다 다음과 같이 실행한다.
user$ cat >> ~/.bashrc
export JAVA_HOME=/usr/java/j2sdk1.4.2_04
export PATH=$JAVA_HOME/bin:$PATH
user$ . ~/.bashrc
Ant 설치
Version : 1.5.1+
다운로드 : http://ant.apache.org/bindownload.cgi
참고 : Fedora Core2에서 제공하는 Ant는 사용하지 말 것.
설치
root권한으로 다음과 같은 명령어 실행한다.
root# tar zxvf apache-ant-1.6.2-bin.tar.gz
root# cd apache-ant-1.6.2
root# mkdir /usr/java/ant
root# mv bin lib /usr/java/ant
환경설정
마찬가지로, 다음과 같이 입력하여 환경변수를 설정한다.
user$ cat >> ~/.bashrc
export ANT_HOME=/usr/local/ant
export PATH=/usr/local/ant/bin:$PATH
user$ . ~/.bashrc
postgresql 설치
Version : 7.1+
다운로드 : http://www.postgresql.org/ftp/
설치
root# rpm -ivh postgresql-libs-7.2.3-1PGDG.i386.rpm
root# rpm -ivh postgresql-7.2.3-1PGDG.i386.rpm
root# rpm -ivh postgresql-server-7.2.3-1PGDG.i386.rpm
root# rpm -ivh postgresql-jdbc-7.2.3-1PGDG.i386.rpm
참고 : 위의 예제는 OS(Redhat 7.3)와의 호환성을 위해 7.2.3을 설치한 것이다.
설정
/var/lib/pgsql/data/pg_hba.conf 파일의 127.0.0.1 부분을 현제 아이피로 바꿔준다.
root# diff -Naur pg_hba.conf_orig pg_hba.conf
— pg_hba.conf_orig Thu Mar 3 15:34:34 2005
+++ pg_hba.conf Fri Mar 4 13:52:33 2005
@@ -242,4 +242,4 @@
# TYPE DATABASE IP_ADDRESS MASK AUTH_TYPE AUTH_ARGUMENT
local all trust
-host all 127.0.0.1 255.255.255.255 trust
+host all 150.183.24.17 255.255.255.255 trust
/var/lib/pgsql/data/postgresql.conf 에서 tcp 통신이 가능하도록 설정한다.
root# diff -Naur postgresql.conf_orig postgresql.conf
— postgresql.conf_orig Fri Mar 4 13:58:49 2005
+++ postgresql.conf Fri Mar 4 13:59:32 2005
@@ -23,7 +23,7 @@
#
# Connection Parameters
#
-#tcpip_socket = true
+tcpip_socket = true
#ssl = false
#max_connections = 32
Globus 설치
다운로드 : http://www-unix.globus.org/toolkit/survey/index.php?download=gt3.9.5-all-source-installer.tar.gz
설치할 디렉토리 생성
root# mkdir /usr/local/globus-3.9.5
root# chown globus:globus /usr/local/globus-3.9.5
Globus 설치
globus$ tar zxvf gt3.9.5-all-source-installer.tar.gz
globus$ cd gt3.9.5-all-source-installer
globus$ export GLOBUS_LOCATION=/usr/local/globus-3.9.5
globus$ ./configure –prefix=$GLOBUS_LOCATION
globus$ make # 로그 파일을 만들고 싶으면 : make 2 > &1 | tee build.log
환경 설정
globus$ cat >> ~/.bashrc
export GLOBUS_LOCATION=/usr/local/gt3.9.5
. $GLOBUS_LOCATION/etc/globus-user-env.sh
gridca.gridcenter.or.kr를 통해 GridCA Certification의 발급 및 설정
기본 설정 사항
기본 디렉토리 생성
/etc/디렉토리 밑에 grid-security와 grid-security/certificates 디렉토리를 만든다.
root# mkdir -p /etc/grid-security/certificates
기본 설정파일 생성
http://gridca.gridcenter.or.kr/에서 다음과 같은 Trusted CA 파일 다운로드하여 /etc/grid-security/certificates 디렉토리 안에 넣는다.
f93666d2.0(http://gridca.gridcenter.or.kr/certificates/f93666d2.0)
f93666d2.signing_policy(http://gridca.gridcenter.or.kr/certificates/f93666d2.signing_policy)
grid-cert-request configuration 파일 생성
/etc/grid-security/certificates 디렉토리 안에, 다음과 같은 내용의 globus-user-ssl.conf.f93666d2 파일을 생성한다.
#
# SSLeay example configuration file.
# This is mostly being used for generation of certificate requests.
#
RANDFILE = $ENV::HOME/.rnd
####################################################################
[ ca ]
default_ca = CA_default # The default ca section
####################################################################
[ CA_default ]
dir = ./demoCA # Where everything is kept
certs = $dir/certs # Where the issued certs are kept
crl_dir = $dir/crl # Where the issued crl are kept
database = $dir/index.txt # database index file.
new_certs_dir = $dir/newcerts # default place for new certs.
certificate = $dir/cacert.pem # The CA certificate
serial = $dir/serial # The current serial number
crl = $dir/crl.pem # The current CRL
private_key = $dir/private/cakey.pem# The private key
RANDFILE = $dir/private/.rand # private random number file
x509_extensions = x509v3_extensions # The extentions to add to the cert
default_days = 365 # how long to certify for
default_crl_days= 365 # DEE 30 # how long before next CRL
default_md = md5 # which md to use.
preserve = no # keep passed DN ordering
# A few difference way of specifying how similar the request should look
# For type CA, the listed attributes must be the same, and the optional
# and supplied fields are just that 🙂
policy = policy_match
# For the CA policy
[ policy_match ]
countryName = optional
stateOrProvinceName = optional
organizationName = match
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
# For the ‘anything’ policy
# At this point in time, you must list all acceptable ‘object’
# types.
[ policy_anything ]
countryName = optional
stateOrProvinceName = optional
localityName = optional
organizationName = optional
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
####################################################################
[ req ]
default_bits = 1024
default_keyfile = privkey.pem
distinguished_name = req_distinguished_name
req_extensions = v3_req
[ req_distinguished_name ]
# BEGIN CONFIG
countryName = Country Name (2 letter code)
countryName_default = KR
countryName_min = 2
countryName_max = 2
#stateOrProvinceName = State or Province Name (full name)
#localityName = Locality Name (e.g., city)
0.organizationName = Main Organization
0.organizationName_default = Globus
0.organizationalUnitName = Level 0 Organizational Unit
0.organizationalUnitName_default = gridcenter.or.kr
commonName = Name (e.g., John M. Smith)
commonName_max = 64
#emailAddress = Email Address
#emailAddress_max = 40
# END CONFIG
[ v3_req ]
nsCertType = objsign,email,server,client
basicConstraints = critical,CA:false
똑같이, /etc/grid-security/certificates 디렉토리 안에 다음 내용의 globus-host-ssl.conf.f93666d2 파일을 생성한다.
#
# SSLeay example configuration file.
# This is mostly being used for generation of certificate requests.
#
RANDFILE = $ENV::HOME/.rnd
####################################################################
[ ca ]
default_ca = CA_default # The default ca section
####################################################################
[ CA_default ]
dir = ./demoCA # Where everything is kept
certs = $dir/certs # Where the issued certs are kept
crl_dir = $dir/crl # Where the issued crl are kept
database = $dir/index.txt # database index file.
new_certs_dir = $dir/newcerts # default place for new certs.
certificate = $dir/cacert.pem # The CA certificate
serial = $dir/serial # The current serial number
crl = $dir/crl.pem # The current CRL
private_key = $dir/private/cakey.pem# The private key
RANDFILE = $dir/private/.rand # private random number file
x509_extensions = x509v3_extensions # The extentions to add to the cert
default_days = 365 # how long to certify for
default_crl_days= 365 # DEE 30 # how long before next CRL
default_md = md5 # which md to use.
preserve = no # keep passed DN ordering
# A few difference way of specifying how similar the request should look
# For type CA, the listed attributes must be the same, and the optional
# and supplied fields are just that 🙂
policy = policy_match
# For the CA policy
[ policy_match ]
countryName = optional
stateOrProvinceName = optional
organizationName = match
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
# For the ‘anything’ policy
# At this point in time, you must list all acceptable ‘object’
# types.
[ policy_anything ]
countryName = optional
stateOrProvinceName = optional
localityName = optional
organizationName = optional
organizationalUnitName = optional
commonName = supplied
emailAddress = optional
####################################################################
[ req ]
default_bits = 1024
default_keyfile = privkey.pem
distinguished_name = req_distinguished_name
req_extensions = v3_req
[ req_distinguished_name ]
# BEGIN CONFIG
countryName = Country Name (2 letter code)
countryName_default = KR
countryName_min = 2
countryName_max = 2
#stateOrProvinceName = State or Province Name (full name)
#localityName = Locality Name (e.g., city)
0.organizationName = Main Organization
0.organizationName_default = Globus
commonName = Name (e.g., John M. Smith)
commonName_max = 64
#emailAddress = Email Address
#emailAddress_max = 40
# END CONFIG
[ v3_req ]
nsCertType = objsign,email,server,client
basicConstraints = critical,CA:false
마지막으로, /etc/grid-security/certificates 디렉토리 안에, 다음 내용의 grid-security.conf.f93666d2 파일을 생성한다.
#################################################################
#
# File: grid-security.conf
#
# Purpose: This file contains the configuration information
# for the Grid Security Infrastructure
#
#################################################################
# These values are set by globus-setup
SETUP_GSI_HOST_BASE_DN=”c=KR, o=Globus”
SETUP_GSI_USER_BASE_DN=”o=Grid, o=Globus”
SETUP_GSI_CA_NAME=”KISTI Supercomputing Center CA2″
SETUP_GSI_CA_EMAIL_ADDR=”sangwan@kisti.re.kr”
_domain=`globus-domainname`
DEFAULT_GSI_HOST_BASE_DN=”c=KR, o=Globus”
DEFAULT_GSI_USER_BASE_DN=”ou=${_domain}, c=KR, o=Globus”
DEFAULT_GSI_CA_NAME=”KISTI Supercomputing Center CA2″
DEFAULT_GSI_CA_NAME=”KISTI Supercomputing Center CA2″
DEFAULT_GSI_CA_EMAIL_ADDR=”sangwan@kisti.re.kr”
# Distinguish Name (DN) of the Host
GSI_HOST_BASE_DN=”${SETUP_GSI_HOST_BASE_DN:-${DEFAULT_GSI_HOST_BASE_DN}}”
# Distinguish Name (DN) of the User
GSI_USER_BASE_DN=”${SETUP_GSI_USER_BASE_DN:-${DEFAULT_GSI_USER_BASE_DN}}”
# CA Name for the organization
GSI_CA_NAME=”${SETUP_GSI_CA_NAME:-${DEFAULT_GSI_CA_NAME}}”
# CA Email address for the organization
GSI_CA_EMAIL_ADDR=”${SETUP_GSI_CA_EMAIL_ADDR:-${DEFAULT_GSI_CA_EMAIL_ADDR}}”
export GSI_HOST_BASE_DN
export GSI_USER_BASE_DN
export GSI_CA_NAME
export GSI_CA_EMAIL_ADDR
링크 생성 : 위에서 생성한 파일들을, /etc/grid-security 디렉토리 안에, 다음과 같이 링크를 걸어 준다.
root# ln -s /etc/grid-security/certificates/globus-user-ssl.conf.f93666d2 /etc/grid-security/globus-user-ssl.conf
root# ln -s /etc/grid-security/certificates/globus-host-ssl.conf.f93666d2 /etc/grid-security/globus-host-ssl.conf
root# ln -s /etc/grid-security/certificates/grid-security.conf.f93666d2 /etc/grid-security/grid-security.conf
Host Certificates 생성
root 권한으로 다음을 입력하여 Host Certificates를 만든다.
root# grid-cert-request -host ‘solar16.gridcenter.or.kr’
생성된 /etc/grid-security/hostcert_request.pem파일을 http://gridca.gridcenter.or.kr/GridCA/ (필요시 계정 생성)의 “Upload a new CSR for a host”를 통해서 업로드 후 Cert 요청을 한다.
http://gridca.gridcenter.or.kr/GridCA/ 에서 생성된 host certificates 파일을 /etc/grid-security/hostcert.pem에 저장한다.
/etc/grid-security/ 의 hostcert.pem 과 hostkey.pem파일을 각각 containercert.pem containerkey.pem으로 copy 후 globus계정으로 퍼미션을 바꿔준다.
root# pwd
/etc/grid-security/
root# cp hostcert.pem containercert.pem
root# cp hostkey.pem containerkey.pem
root# chown globus:globus container*
User Certificates 생성(globus계정 포함)
user$ grid-cert-request
생성된 ~/.globus/usercert_request.pem파일을 http://gridca.gridcenter.or.kr/GridCA/ 의 “Upload a new CSR for a user”를 통해서 업로드 후 Cert 요청한다.
홈페이지에서 생성된 user certificates 파일을 ~/.globus/usercert.pem에 저장한다.
certification을 다음과 같이 host에 등록한다.
user$ grid-cert-info -subject
/O=Grid/O=Globus/OU=KISTI/CN=s0lar16
user$ whoami
user01
user$ su
password:
root# grid-mapfile-add-entry -dn “/O=Grid/O=Globus/OU=KISTI/CN=s0lar16” -ln “user01”
root# cat /etc/grid-security/grid-mapfile
“/O=Grid/O=Globus/OU=KISTI/CN=s0lar16” user01
GridFTP 설정(xinetd 이용)
다음 내용의 /etc/xinet.d/디렉토리 안에 gsiftp 파일을 생성한다.
service gsiftp
{
disable = no
instances = 100
socket_type = stream
wait = no
user = root
env += GLOBUS_LOCATION=/usr/local/gt3.9.5
env += LD_LIBRARY_PATH=/usr/local/gt3.9.5/lib
server = /usr/local/gt3.9.5/sbin/globus-gridftp-server
server_args = -i
log_on_success += DURATION
nice = 10
}
/etc/services 에 다음과 같은 내용 추가한다.
gsiftp 2811/tcp gsiftp # GridFtp
xinetd 데몬 제시작
root# chkconfig gsiftp on
root# /etc/init.d/xinetd restart
RFT 설정
postgres 페스워드 설정 및 로그인
root# passwd postgres
Changing password for user postgres.
New password:
Retype new password:
passwd: all authentication tokens updated successfully.
root# su postgres
b. postgresql DB설정
postgres$ createdb rftDatabase
postgres$ psql -d rftDatabase -f $GLOBUS_LOCATION/share/globus_wsrf_rft/rft_schema.sql
c. postgresql 시작
root# /etc/init.d/postgresql start #(root 계정으로)
or
postgres$ /usr/bin/pg_ctl start -D /var/lib/pgsql/data -l /var/log/pgsql
$GLOBUS_LOCATION/etc/globus_wsrf_rft/jndi-config.xml 파일의 “dbConfiguration” 하위에 있는 connectionString과 userName 그리고 password를 알맞게 설정한다.
globus$ cat $GLOBUS_LOCATION/etc/globus_wsrf_rft/jndi-config.xml
…
<parameter>
<name>
connectionString
</name>
<value>
jdbc:postgresql://solar16.gridcenter.or.kr/rftDatabase
</value>
</parameter>
<parameter>
<name>
userName
</name>
<value>
postgres
</value>
</parameter>
<parameter>
<name>
password
</name>
<value>
foo.bar.
</value>
</parameter>
…
설치 확인
globus$ grid-proxy-init -verify -debug
User Cert File: /home/globus/.globus/usercert.pem
User Key File: /home/globus/.globus/userkey.pem
Trusted CA Cert Dir: /etc/grid-security/certificates
Output File: /tmp/x509up_u504
Your identity: /O=Grid/O=Globus/OU=KISTI/CN=s0lar16
Enter GRID pass phrase for this identity:
Creating proxy ……++++++++++++
..++++++++++++
Done
Proxy Verify OK
Your proxy is valid until: Mon Mar 7 05:36:04 2005
globus$ globus-start-container
Starting SOAP server at: https://150.183.24.17:8443/wsrf/services/
With the following services:
[1]: https://150.183.24.17:8443/wsrf/services/TriggerFactoryService
[2]: https://150.183.24.17:8443/wsrf/services/DelegationTestService
[3]: https://150.183.24.17:8443/wsrf/services/SecureCounterService
[4]: https://150.183.24.17:8443/wsrf/services/IndexServiceEntry
…
PBS 설정
PBS 설치 : http://testbed.gridcenter.or.kr/kor/technical_doc/pbs/pbs-install.html 참조.
GT4에서 PBS 플러그인 설치
globus$ cd $Globus_install_dir
globus$ make gt4-gram-pbs postinstall
globus$ gpt-postinstall
globus$ cd $GLOBUS_LOCATION/setup/globus
globus$ ./setup-globus-job-manager-pbs –remote-shell=ssh
실행 예제
간단한 작업 예제
globus$ cat > simple_job.xml
<?xml version=”1.0″ encoding=”UTF-8″?>
<job>
<executable>/bin/echo</executable>
<argument>this is an example_string </argument>
<argument>Globus was here</argument>
<stdout>${GLOBUS_USER_HOME}/stdout</stdout>
<stderr>${GLOBUS_USER_HOME}/stderr</stderr>
</job>
globus$ $GLOBUS_LOCATION/bin/globusrun-ws -submit -f simple_job.xml
Submitting job…Done.
Job ID: uuid:9a2326cc-8fb2-11d9-9103-0020ed62bb2b
Termination time: 03/09/2005 09:15 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job…Done.
PBS 작업의 예제
globus$ $GLOBUS_LOCATION/bin/globusrun-ws -submit -factory-type PBS -f simple_job.xml
staging 작업 예제
globus$ cat > staging_job.xml
<job>
<executable>my_echo</executable>
<directory>${GLOBUS_USER_HOME}</directory>
<argument>Hello</argument>
<argument>World!</argument>
<stdout>${GLOBUS_USER_HOME}/stdout</stdout>
<stderr>${GLOBUS_USER_HOME}/stderr</stderr>
<fileStageIn>
<transfer>
<sourceUrl>gsiftp://XXX.XXX.XXX.XXX:2811/bin/echo</sourceUrl>
<destinationUrl>file:///${GLOBUS_USER_HOME}/my_echo</destinationUrl>
</transfer>
</fileStageIn>
<fileStageOut>
<transfer>
<sourceUrl>file://${GLOBUS_USER_HOME}/stdout</sourceUrl>
<destinationUrl>gsiftp://XXX.XXX.XXX.XXX:2811/tmp/stdout</destinationUrl>
</transfer>
</fileStageOut>
<fileCleanUp>
<deletion>
<file>file://${GLOBUS_USER_HOME}/my_echo</file>
</deletion>
</fileCleanUp>
</job>
globus$ $GLOBUS_LOCATION/bin/globusrun-ws -submit -f staging_job.xml
stagingCredentialEndpoint 관련 애러가 날 경우
globus$ $GLOBUS_LOCATION/bin/globusrun-ws -submit -staging-delegate -f staging_job.xml
Multyjob 작업 예제
globus$ cat > multyjob.xml
<?xml version=”1.0″ encoding=”UTF-8″?>
<multiJob xmlns:gram=”http://www.globus.org/namespaces/2004/10/gram/job”
xmlns:wsa=”http://schemas.xmlsoap.org/ws/2004/03/addressing”>
<factoryEndpoint>
<wsa:Address>
https://XXX.XXX.XXX.XXX:8443/wsrf/services/ManagedJobFactoryService
</wsa:Address>
<wsa:ReferenceProperties>
<gram:ResourceID>Multi</gram:ResourceID>
</wsa:ReferenceProperties>
</factoryEndpoint>
<directory>${GLOBUS_LOCATION}</directory>
<count>1</count>
<job>
<factoryEndpoint>
<wsa:Address>https://XXX.XXX.XXX.XXX:8443/wsrf/services/ManagedJobFactoryService</wsa:Address>
<wsa:ReferenceProperties>
<gram:ResourceID>Fork</gram:ResourceID>
</wsa:ReferenceProperties>
</factoryEndpoint>
<executable>/bin/date</executable>
<stdout>${GLOBUS_USER_HOME}/stdout.p1</stdout>
<stderr>${GLOBUS_USER_HOME}/stderr.p1</stderr>
<count>2</count>
</job>
<job>
<factoryEndpoint>
<wsa:Address>https://XXX.XXX.XXX.XXX:8443/wsrf/services/ManagedJobFactoryService</wsa:Address>
<wsa:ReferenceProperties>
<gram:ResourceID>Fork</gram:ResourceID>
</wsa:ReferenceProperties>
</factoryEndpoint>
<executable>/bin/echo</executable>
<argument>Hello World!</argument>
<stdout>${GLOBUS_USER_HOME}/stdout.p2</stdout>
<stderr>${GLOBUS_USER_HOME}/stderr.p2</stderr>
<count>1</count>
</job>
</multiJob>
globus$ $GLOBUS_LOCATION/bin/globusrun-ws -submit -f multyjob.xml
WS Gram의 Sudo 설정
목적 : 요청된 특정 서비스의 실행을, 서비스 계정(본 예제에서는 Globus계정)이 아닌 다른 특정 계정으로 실행 시키고자 할 때.
예 : “/O=Grid/O=Globus/OU=KISTI/CN=s0lar16″라는 proxy로 요청된 모든 Job들은 특정 계정 test_sudo로 실행되도록 할 때.
root권한으로, 명령어 “visudo”를 이용하여 /etc/sudoers에 다음과 같은 내용을 첨가한다.
# Globus GRAM entries
globus ALL=(test_sudo) \\
NOPASSWD: /usr/local/globus-3.9.5/libexec/globus-gridmap-and-execute \\
/usr/local/globus-3.9.5/libexec/globus-job-manager-script.pl *
globus ALL=(test_sudo) \\
NOPASSWD: /usr/local/globus-3.9.5/libexec/globus-gridmap-and-execute \\
/usr/local/globus-3.9.5/libexec/globus-gram-local-proxy-tool *
참고 1. test_sudo가 있는 곳에 Job을 수행할 계정을 알맞게 수정한다. (예 : “을 제외한, “username01,username02” , “ALL” 등등 )
참고 2. “/usr/local/globus-3.9.5″는 $GLOBUS_LOCATION이 가리키고 있는 디렉토리로 알맞게 수정한다.
/etc/grid-security 디렉토리 안에 있는 grid-mapfile을 알맞게 고친다.
“/O=Grid/O=Globus/OU=KISTI/CN=s0lar16” test_sudo
작업 예제 : 위의 실행 예제 1번을 돌려 보자.
globus$ whoami
globus
globus$ $GLOBUS_LOCATION/bin/globusrun-ws -submit -f simple_job.xml
Submitting job…Done.
Job ID: uuid:9a2326cc-8fb2-11d9-9103-0020ed62bb2b
Termination time: 03/28/2005 23:75 GMT
Current job state: Active
Current job state: CleanUp
Current job state: Done
Destroying job…Done.
globus$ ls ~/
globus$ ls -l /home/test_sudo
-rw-r–r– 1 test_sudo test_sudo 0 Mar 28 23:75 stderr
-rw-r–r– 1 test_sudo test_sudo 43 Mar 28 23:75 stdout
——————————————————————————–
