文档控制​

修改记录​

分发者​

审阅记录​

1 安装目标与规划​

两台华为2288 H V5服务器将安装Oracle11gR2 RAC。​

1.1 安装介质版本​

1.2 安装目录规划​

1.3 网络规划​

1.4 共享存储规划​

2 安装环境检查​

2.1 软件需求检查​

l 操作系统需求​

要求:Oracle Enterprise Linux 6.10 (Update 1 or later)​

[root@shyorc01 ~]# at /etc/redhat-release​

Red Hat Enterprise Linux Server release 6.10 (Santiago)​

[root@shyorc02 ~]# at /etc/redhat-release​

Red Hat Enterprise Linux Server release 6.10 (Santiago)​

l 系统软件包需求​

binutils-2.20.51.0.2-5.28.el6 (x86_64)​

compat-libcap1-1.10-1 (x86_64)​

compat-libstdc++-33-3.2.3-69.el6 (x86_64)​

compat-libstdc++-33-3.2.3-69.el6 (i686)​

gcc-4.4.6-3.el6 (x86_64)​

gcc-c++-4.4.6-3.el6 (x86_64)​

glibc-2.12-1.47.el6 (x86_64)​

glibc-2.12-1.47.el6 (i686)​

glibc-devel-2.12-1.47.el6 (x86_64)​

glibc-devel-2.12-1.47.el6 (i686)​

glibc-headers-2.12-1.149.el6（x86_64）​

ksh-20100621-12.el6 (x86_64)​

libgcc-4.4.6-3.el6 (x86_64)​

libgcc-4.4.6-3.el6 (i686)​

libstdc++-4.4.6-3.el6 (x86_64)​

libstdc++-4.4.6-3.el6 (i686)​

libstdc++-devel-4.4.6-3.el6 (x86_64)​

libstdc++-devel-4.4.6-3.el6 (i686)​

libaio-0.3.107-10.el6 (x86_64)​

libaio-0.3.107-10.el6 (i686)​

libaio-devel-0.3.107-10.el6 (x86_64)​

libaio-devel-0.3.107-10.el6 (i686)​

make-3.81-19.el6 (x86_64)​

sysstat-9.0.4-18.el6 (x86_64)​

oracleasm-support(x86_64)​

elfutils-libelf-devel-0.97(x86_64)​

l 挂载yum​

mount –o loop /oracle/ora_linux_610.iso /mnt​

l 检查命令：​

yum list binutils compat-libcap1 compat-libstdc++-33 compat-libstdc++-33.i686 cpp gcc gcc-c++ glibc glibc.i686 glibc-devel glibc-devel.i686 glibc-headers ksh libaio libaio.i686 libaio-devel libaio-devel.i686 libgcc libgcc.i686 libstdc++ libstdc++.i686 libstdc++-devel libstdc++-devel.i686 make mpfr.x86_64 sysstat oracleasm-support elfutils-libelf-devel​

检查结果​

l 补安装软件包：​

#cd /etc/yum.repos.d​

#vim /etc/yum.repos.d/oel6.repo​

[base]​

name=iso​

baseurl=file:///mnt​

gpgcheck=1​

enabled=1​

gpgkey=file:///mnt/RPM-GPG-KEY-oracle​

l 安装命令​

yum -y install binutils compat-libcap1 compat-libstdc++-33 compat-libstdc++-33.i686 cpp gcc gcc-c++ glibc glibc.i686 glibc-devel glibc-devel.i686 glibc-headers ksh libaio libaio.i686 libaio-devel libaio-devel.i686 libgcc libgcc.i686 libstdc++ libstdc++.i686 libstdc++-devel libstdc++-devel.i686 make mpfr.x86_64 sysstat oracleasm-support elfutils-libelf-devel​

l JAVA 版本需求：1.4.2​

[root@shyorc01 ~]# java -version​

java version "1.7.0_181"​

OpenJDK Runtime Environment (rhel-2.6.14.10.0.1.el6-x86_64 u181-b00)​

OpenJDK 64-Bit Server VM (build 24.181-b00, mixed mode)​

[root@shyorc02 software]# java -version​

java version "1.7.0_181"​

OpenJDK Runtime Environment (rhel-2.6.14.10.0.1.el6-x86_64 u181-b00)​

OpenJDK 64-Bit Server VM (build 24.181-b00, mixed mode)​

3 安装前准备工作​

3.1 创建安装目录​

在所有节点上执行以下​

mkdir -p /oracle/app/grid​

mkdir -p /oracle/app/11.2.0/grid​

mkdir -p /oracle/app/oraInventory​

mkdir -p /oracle/app/oracle/product/11.2.0/db_1​

3.2 建立用户​

安装oracle11g rac软件需要手工创建oracle用户和grid用户，前者用于管理数据库，后者用于管理Grid。​

Ø 建立oracle和grid用户​

groupadd -g 50001 oinstall​

groupadd -g 50002 dba​

groupadd -g 50003 oper​

groupadd -g 50004 asmadmin​

groupadd -g 50005 asmoper​

groupadd -g 50006 asmdba​

useradd -u 60001 -g oinstall -G dba,asmdba,oper oracle​

useradd -u 60002 -g oinstall -G asmadmin,asmdba,asmoper,oper,dba grid​

id -a oracle​

id -a grid​

Ø 修改密码​

echo "P@sswordgrid" | passwd --stdin grid​

echo "P@sswordoracle" | passwd --stdin oracle​

Ø 安装目录授权​

将以上目录及子目录的权限授予相关用户​

chown -R grid:oinstall /oracle​

chown -R grid:oinstall /oracle/app/oraInventory​

chown -R oracle:oinstall /oracle/app/oracle​

chmod -R 775 /oracle​

3.3 配置和修改参数​

Ø 配置英文环境​

echo "export LANG=en_US.UTF-8" >> ~/.bash_profile​

source ~/.bash_profile​

Ø 关闭透明页​

cat /sys/kernel/mm/transparent_hugepage/defrag​

cat /sys/kernel/mm/transparent_hugepage/enabled​

cat >> /etc/rc.d/rc.local <<-EOF​

if test -f /sys/kernel/mm/transparent_hugepage/enabled; then​

echo never > /sys/kernel/mm/transparent_hugepage/enabled​

fi​

if test -f /sys/kernel/mm/transparent_hugepage/defrag; then​

echo never > /sys/kernel/mm/transparent_hugepage/defrag​

fi​

EOF​

chmod +x /etc/rc.d/rc.local​

Ø 关闭NUMA功能、需要重启服务器​

sed -i 's#quiet#quiet numa=off#g' /boot/efi/EFI/redhat/grub.confecho​

Ø 关闭SELINUX、IPTABLE、NTP​

# 关闭 selinux​

echo "SELINUX=disabled" > /etc/selinux/config​

echo "#SELINUXTYPE=targeted " >> /etc/selinux/config​

setenforce 0​

# 关闭防火墙​

service iptables stop​

chkconfig --level 35 iptables off​

# 禁止NTP服务​

service ntpd stop​

chkconfig ntpd off​

Ø 配置系统环境资源、内核参数配置​

cat >> /etc/security/limits.conf <<-EOF​

##############ORACLE SET#####################​

grid soft nproc 16384​

grid hard nproc 16384​

grid soft nofile 65536​

grid hard nofile 65536​

grid soft stack 32768​

grid hard stack 32768​

oracle soft nproc 16384​

oracle hard nproc 16384​

oracle soft nofile 65536​

oracle hard nofile 65536​

oracle soft stack 32768​

oracle hard stack 32768​

##############ORACLE SET#####################​

EOF​

ulimit -a​

Ø 配置进程打开最大数量​

echo "* - nproc 16384" > /etc/security/limits.d/90-nproc.conf​

cat /etc/security/limits.d/90-nproc.conf​

Ø 用户资源分配​

echo "session required pam_limits.so" >> /etc/pam.d/login​

cat /etc/pam.d/login​

Ø 配置Linux系统内核文件​

cat >> /etc/sysctl.conf <<-EOF​

##############ORACLE SET#####################​

fs.aio-max-nr = 1048576​

fs.file-max = 6815744​

kernel.shmmax = 214748364800​

kernel.shmall = 52428800​

kernel.shmmni = 4096​

kernel.sem = 250 32000 100 128​

net.ipv4.ip_local_port_range = 9000 65500​

net.core.rmem_default = 262144​

net.core.rmem_max = 4194304​

net.core.wmem_default = 262144​

net.core.wmem_max = 1048586​

vm.nr_hugepages = 81920​

vm.swappiness=10​

vm.min_free_kbytes=524288​

vm.vfs_cache_pressure=200​

##############ORACLE SET#####################​

EOF​

Ø 配置添加UUID参数​

echo "optinotallow=--whitelisted --replace-whitespace" > /etc/scsi_id.config​

3.4 配置环境变量​

Ø 配置oracle用户环境变量​

oracle用户需要配置相关的环境变量，在每个节点上上都进行配置​

$ echo $SHELL​

PS1="[`whoami`@`hostname`:"'$PWD]$'​

#alias sqlplus="rlwrap sqlplus"​

#alias rman="rlwrap rman"​

export PS1​

export TMP=/tmp​

export LANG=en_US​

export TMPDIR=$TMP​

export ORACLE_HOSTNAME=shyorc01​

export ORACLE_UNQNAME=shydb​

ORACLE_BASE=/oracle/app/oracle; export ORACLE_BASE​

ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1; export ORACLE_HOME​

ORACLE_SID=shydb01; export ORACLE_SID​

ORACLE_TERM=xterm; export ORACLE_TERM​

NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT​

NLS_LANG=AMERICAN_AMERICA.AL32UTF8;export NLS_LANG​

PATH=.:$PATH:$HOME/bin:$ORACLE_BASE/product/11.2.0/db_1/bin:$ORACLE_HOME/bin; export PATH​

THREADS_FLAG=native; export THREADS_FLAG​

if [ $USER = "oracle" ] || [ $USER = "grid" ]; then​

if [ $SHELL = "/bin/ksh" ]; then​

ulimit -p 16384​

ulimit -n 65536​

else​

ulimit -u 16384 -n 65536​

fi​

umask 022​

fi​

Ø 配置grid用户环境变量​

grid用户需要配置相关的环境变量，在每个节点上上都进行配置​

$ echo $SHELL​

/bin/bash​

Bourne shell (sh), Bash shell (bash), or Korn shell (ksh):​

$ vi .bash_profile​

PS1="[`whoami`@`hostname`:"'$PWD]$'​

export PS1​

umask 022​

#alias sqlplus="rlwrap sqlplus"​

export TMP=/tmp​

export LANG=en_US​

export TMPDIR=$TMP​

export ORACLE_HOSTNAME=shyorc01​

ORACLE_SID=+ASM1; export ORACLE_SID​

ORACLE_TERM=xterm; export ORACLE_TERM​

ORACLE_BASE=/oracle/app/grid; export ORACLE_BASE​

ORACLE_HOME=/oracle/app/11.2.0/grid; export ORACLE_HOME​

NLS_DATE_FORMAT="yyyy-mm-dd HH24:MI:SS"; export NLS_DATE_FORMAT​

PATH=.:$PATH:$HOME/bin:$ORACLE_HOME/bin; export PATH​

THREADS_FLAG=native; export THREADS_FLAG​

if [ $USER = "oracle" ] || [ $USER = "grid" ]; then​

if [ $SHELL = "/bin/ksh" ]; then​

ulimit -p 16384​

ulimit -n 65536​

else​

ulimit -u 16384 -n 65536​

fi​

umask 022​

fi​

3.5 配置存储​

安装oracle RAC需要对共享的存储空间进行配置，包括集群仲裁磁盘、集群信息盘、ASM数据磁盘和归档日志磁盘​

Ø 存储划分​

Ø 配置UDEV绑定共享磁盘权限​

cat > /etc/udev/rules.d/99-oracle-asmdevices.rules <<-EOF​

KERNEL=="sd?",SUBSYSTEM=="block",PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="3620658e100aeed0c​

010310c200000000", SYMLINK+="asm-ocr01",OWNER="grid", GROUP="asmadmin",MODE="0660"​

KERNEL=="sd?",SUBSYSTEM=="block",PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="3620658e100aeed0c​

01031d9e00000001", SYMLINK+="asm-ocr02",OWNER="grid", GROUP="asmadmin",MODE="0660"​

KERNEL=="sd?",SUBSYSTEM=="block",PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="3620658e100aeed0c​

010327b900000002", SYMLINK+="asm-ocr03",OWNER="grid", GROUP="asmadmin",MODE="0660"​

KERNEL=="sd?",SUBSYSTEM=="block",PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="3620658e100aeed0c​

010341cf00000003", SYMLINK+="asm-arch01",OWNER="grid", GROUP="asmadmin",MODE="0660"​

KERNEL=="sd?",SUBSYSTEM=="block",PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="3620658e100aeed0c​

0103570900000004", SYMLINK+="asm-data01",OWNER="grid", GROUP="asmadmin",MODE="0660"​

KERNEL=="sd?",SUBSYSTEM=="block",PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="3620658e100aeed0c​

010364d200000005", SYMLINK+="asm-data02",OWNER="grid", GROUP="asmadmin",MODE="0660"​

KERNEL=="sd?",SUBSYSTEM=="block",PROGRAM=="/sbin/scsi_id --whitelisted --replace-whitespace --device=/dev/$name",RESULT=="3620658e100aeed0c​

010371c900000006", SYMLINK+="asm-data03",OWNER="grid", GROUP="asmadmin",MODE="0660"​

EOF​

Ø 查看设备权限​

[root@shyorc02 software]# ll /dev/asm*​

3.6 配置网络​

Ø 配置网卡绑定（主备模式）​

shyorc01公网网卡绑定:

DEVICE="bond0"

IPADDR=192.168.25.120

NETMASK=255.255.255.0

NETWORK=192.168.25.0

GATEWAY=192.168.25.254

ONBOOT=yes

BOOTPROTO=none

USERCTL=no

TYPE=Ethernet

BONDING_OPTS="miimnotallow=200 mode=1"

shyorc02公网网卡绑定:

DEVICE=bond0

NAME=bond0

IPADDR=192.168.25.121

NETMASK=255.255.255.0

NETWORK=192.168.25.0

GATEWAY=192.168.25.254

ONBOOT=yes

DEFROUTE=yes

BOOTPROTO=none

USERCTL=no

TYPE=Ethernet

BONDING_OPTS="miimnotallow=200 mode=1"

Ø 查看网络绑定状况​

[root@shyorc01 ~]# ifconfig​

bond0 Link encap:Ethernet HWaddr EC:C0:1B:DA:65:65 ​

inet addr:192.168.25.120 Bcast:192.168.25.255 Mask:255.255.255.0​

UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1​

RX packets:320391 errors:0 dropped:130069 overruns:0 frame:0​

TX packets:376054 errors:0 dropped:0 overruns:0 carrier:0​

collisions:0 txqueuelen:0​

RX bytes:27514003 (26.2 MiB) TX bytes:381588407 (363.9 MiB)​

[root@shyorc02 ]# ifconfig​

bond0 Link encap:Ethernet HWaddr EC:C0:1B:DA:63:F9 ​

inet addr:192.168.25.121 Bcast:192.168.25.255 Mask:255.255.255.0​

UP BROADCAST RUNNING MASTER MULTICAST MTU:1500 Metric:1​

RX packets:301275 errors:0 dropped:130093 overruns:0 frame:0​

TX packets:338410 errors:0 dropped:0 overruns:0 carrier:0​

collisions:0 txqueuelen:0​

RX bytes:25567655 (24.3 MiB) TX bytes:360703608 (343.9 MiB)​

3.7 其他配置​

编辑/etc/sysconfig/network添加NOZEROCONF=yes参数​

# cat /etc/sysconfig/network​

NETWORKING=yes​

HOSTNAME=hisserver1​

NETWORKING_IPV6=no​

NOZEROCONF=yes​

安装cvuqdisk-1.0.9-1.rpm包​

#CVUQDISK_GRP=oinstall; export CVUQDISK_GRP​

#rpm -ivh $STAGE/rpm/cvuqdisk-1.0.9-1.rpm​

4 软件安装过程​

4.1 安装Grid Infrastructure 11.2.0.4​

注：在11.2版本的GSD服务状态显示为OFFLINE是正常的。​

4.2 安装Oracle DB 11.2.0.4软件​

4.3 建立RAC数据库​

5 其它修正​

5.1 后续操作​