oracle11gr2添加節(jié)點最佳實踐

./oracle11gr2添加節(jié)點最佳實踐OS信息:2DB信息:2〔一設置網(wǎng)絡3〔二添加組、目錄和用戶3〔三修改相關(guān)OS參數(shù)3〔四修改用戶環(huán)境變量3〔五設置存儲3〔六配置節(jié)點互信3〔七安裝RPM包3〔八安裝GI前檢測3〔九擴展新節(jié)點到集群中〔期間傳文件可能會花段時間14〔十按說明執(zhí)行腳本18〔十一檢查擴展結(jié)果20〔十二擴展oracle數(shù)據(jù)庫〔期間傳文件可能會花段時間27〔十三按提示執(zhí)行腳本31〔十四創(chuàng)建數(shù)據(jù)庫實例32〔十五驗證32OS信息:[grid@11grac1~]$uname-aLinux11grac12.6.32-300.10.1.el5uek#1SMPWedFeb2217:22:40EST2012i686i686i386GNU/LinuxDB信息:SQL>select*fromv$versionwhererownum<5;BANNEROracleDatabase11gEnterpriseEditionRelease.0-ProductionPL/SQLRelease.0-ProductionCORE.0ProductionTNSforLinux:Version.0-Production〔一設置網(wǎng)絡〔二添加組、目錄和用戶〔三修改相關(guān)OS參數(shù)〔四修改用戶環(huán)境變量〔五設置存儲〔六配置節(jié)點互信〔七安裝RPM包注：1-7步略<其中的修改需要參照其他節(jié)點的配置>〔八安裝GI前檢測[grid@11grac1bin]$pwd[grid@11grac1bin]$./cluvfystage-prenodeadd-n11grac2-fixup-verbosePerformingpre-checksfornodeadditionCheckingnodereachability...Check:Nodereachabilityfromnode"11grac1"DestinationNodeReachable?11grac2yesResult:Nodereachabilitycheckpassedfromnode"11grac1"Checkinguserequivalence...Check:Userequivalenceforuser"grid"NodeNameComment11grac2passedResult:Userequivalencecheckpassedforuser"grid"Checkingnodeconnectivity...Checkinghostsconfigfile...NodeNameStatusComment11grac2passed11grac1passedVerificationofthehostsconfigfilesuccessfulInterfaceinformationfornode"11grac1"NameIPAddressSubnetGatewayDef.GatewayHWAddressMTUeth06100:0C:29:31:BE:BA1500eth06300:0C:29:31:BE:BA1500eth00000:0C:29:31:BE:BA1500eth16100:0C:29:31:BE:C41500Interfaceinformationfornode"11grac2"NameIPAddressSubnetGatewayDef.GatewayHWAddressMTUeth06200:0C:29:D6:B6:091500eth16200:0C:29:D6:B6:131500Check:Nodeconnectivityforinterface"eth0"SourceDestinationConnected?11grac1:eth011grac1:eth0yes11grac1:eth011grac1:eth0yes11grac1:eth011grac2:eth0yes11grac1:eth011grac1:eth0yes11grac1:eth011grac2:eth0yes11grac1:eth011grac2:eth0yesResult:Nodeconnectivitypassedforinterface"eth0"Result:NodeconnectivitycheckpassedCheckingCRSintegrity...TheOracleclusterwareishealthyonnode"11grac1"CRSintegritycheckpassedCheckingsharedresources...CheckingCRShomelocation...Thelocation"/u01/11.2.0/grid"isnotsharedbutispresent/creatableonallnodesResult:SharedresourcescheckfornodeadditionpassedCheckingnodeconnectivity...Checkinghostsconfigfile...NodeNameStatusComment11grac2passed11grac1passedVerificationofthehostsconfigfilesuccessfulInterfaceinformationfornode"11grac1"NameIPAddressSubnetGatewayDef.GatewayHWAddressMTUeth06100:0C:29:31:BE:BA1500eth06300:0C:29:31:BE:BA1500eth00000:0C:29:31:BE:BA1500eth16100:0C:29:31:BE:C41500Interfaceinformationfornode"11grac2"NameIPAddressSubnetGatewayDef.GatewayHWAddressMTUeth06200:0C:29:D6:B6:091500eth16200:0C:29:D6:B6:131500Check:Nodeconnectivityofsubnet""SourceDestinationConnected?11grac1:eth011grac1:eth0yes11grac1:eth011grac1:eth0yes11grac1:eth011grac2:eth0yes11grac1:eth011grac1:eth0yes11grac1:eth011grac2:eth0yes11grac1:eth011grac2:eth0yesResult:Nodeconnectivitypassedforsubnet""withnode<s>11grac1,11grac2Check:TCPconnectivityofsubnet""SourceDestinationConnected?11grac1:6111grac1:63passed11grac1:6111grac1:00passed11grac1:6111grac2:62passedResult:TCPconnectivitycheckpassedforsubnet""Check:Nodeconnectivityofsubnet""SourceDestinationConnected?11grac1:eth111grac2:eth1yesResult:Nodeconnectivitypassedforsubnet""withnode<s>11grac1,11grac2Check:TCPconnectivityofsubnet""SourceDestinationConnected?11grac1:6111grac2:62passedResult:TCPconnectivitycheckpassedforsubnet""Interfacesfoundonsubnet""thatarelikelycandidatesforVIPare:Interfacesfoundonsubnet""thatarelikelycandidatesforaprivateinterconnectare:Result:NodeconnectivitycheckpassedCheck:TotalmemoryNodeNameAvailableRequiredComment11grac21.97GB<2061864.0KB>1.5GB<1572864.0KB>passedResult:TotalmemorycheckpassedCheck:AvailablememoryNodeNameAvailableRequiredComment11grac21.88GB<1970204.0KB>50MB<51200.0KB>passedResult:AvailablememorycheckpassedCheck:SwapspaceNodeNameAvailableRequiredComment11grac24GB<4192956.0KB>2.95GB<3092796.0KB>passedResult:SwapspacecheckpassedCheck:Freediskspacefor"11grac2:/tmp"PathNodeNameMountpointAvailableRequiredComment/tmp11grac2/15.54GB1GBpassedResult:Freediskspacecheckpassedfor"11grac2:/tmp"Check:Userexistencefor"grid"NodeNameStatusComment11grac2existspassedResult:Userexistencecheckpassedfor"grid"Check:RunlevelNodeNamerunlevelRequiredComment11grac233,5passedResult:RunlevelcheckpassedCheck:Hardlimitsfor"maximumopenfiledescriptors"NodeNameTypeAvailableRequiredComment11grac2hard6553665536passedResult:Hardlimitscheckpassedfor"maximumopenfiledescriptors"Check:Softlimitsfor"maximumopenfiledescriptors"NodeNameTypeAvailableRequiredComment11grac2soft10241024passedResult:Softlimitscheckpassedfor"maximumopenfiledescriptors"Check:Hardlimitsfor"maximumuserprocesses"NodeNameTypeAvailableRequiredComment11grac2hard1638416384passedResult:Hardlimitscheckpassedfor"maximumuserprocesses"Check:Softlimitsfor"maximumuserprocesses"NodeNameTypeAvailableRequiredComment11grac2soft20472047passedResult:Softlimitscheckpassedfor"maximumuserprocesses"Check:SystemarchitectureNodeNameAvailableRequiredComment11grac2i686x86passedResult:SystemarchitecturecheckpassedCheck:KernelversionNodeNameAvailableRequiredComment11grac22.6.32-300.10.1.el5uek2.6.18passedResult:KernelversioncheckpassedCheck:Kernelparameterfor"semmsl"NodeNameConfiguredRequiredComment11grac2250250passedResult:Kernelparametercheckpassedfor"semmsl"Check:Kernelparameterfor"semmns"NodeNameConfiguredRequiredComment11grac23200032000passedResult:Kernelparametercheckpassedfor"semmns"Check:Kernelparameterfor"semopm"NodeNameConfiguredRequiredComment11grac2100100passedResult:Kernelparametercheckpassedfor"semopm"Check:Kernelparameterfor"semmni"NodeNameConfiguredRequiredComment11grac2128128passedResult:Kernelparametercheckpassedfor"semmni"Check:Kernelparameterfor"shmmax"NodeNameConfiguredRequiredComment11grac21073741824536870912passedResult:Kernelparametercheckpassedfor"shmmax"Check:Kernelparameterfor"shmmni"NodeNameConfiguredRequiredComment11grac240964096passedResult:Kernelparametercheckpassedfor"shmmni"Check:Kernelparameterfor"shmall"NodeNameConfiguredRequiredComment11grac283886082097152passedResult:Kernelparametercheckpassedfor"shmall"Check:Kernelparameterfor"file-max"NodeNameConfiguredRequiredComment11grac268157446815744passedResult:Kernelparametercheckpassedfor"file-max"Check:Kernelparameterfor"ip_local_port_range"NodeNameConfiguredRequiredComment11grac2between9000&65500between9000&65500passedResult:Kernelparametercheckpassedfor"ip_local_port_range"Check:Kernelparameterfor"rmem_default"NodeNameConfiguredRequiredComment11grac2262144262144passedResult:Kernelparametercheckpassedfor"rmem_default"Check:Kernelparameterfor"rmem_max"NodeNameConfiguredRequiredComment11grac241943044194304passedResult:Kernelparametercheckpassedfor"rmem_max"Check:Kernelparameterfor"wmem_default"NodeNameConfiguredRequiredComment11grac2262144262144passedResult:Kernelparametercheckpassedfor"wmem_default"Check:Kernelparameterfor"wmem_max"NodeNameConfiguredRequiredComment11grac210485861048576passedResult:Kernelparametercheckpassedfor"wmem_max"Check:Kernelparameterfor"aio-max-nr"NodeNameConfiguredRequiredComment11grac210485761048576passedResult:Kernelparametercheckpassedfor"aio-max-nr"Check:Packageexistencefor"make-3.81"NodeNameAvailableRequiredComment11grac2make-3.81-3.el5make-3.81passedResult:Packageexistencecheckpassedfor"make-3.81"Check:Packageexistencefor"binutils-.6"NodeNameAvailableRequiredComment11grac2binutils-.6-20.el5binutils-.6passedResult:Packageexistencecheckpassedfor"binutils-.6"Check:Packageexistencefor"gcc-4.1.2"NodeNameAvailableRequiredComment11grac2gcc-4.1.2-52.el5gcc-4.1.2passedResult:Packageexistencecheckpassedfor"gcc-4.1.2"Check:Packageexistencefor"gcc-c++-4.1.2"NodeNameAvailableRequiredComment11grac2gcc-c++-4.1.2-52.el5gcc-c++-4.1.2passedResult:Packageexistencecheckpassedfor"gcc-c++-4.1.2"Check:Packageexistencefor"libgomp-4.1.2"NodeNameAvailableRequiredComment11grac2libgomp-4.4.6-3.el5.1libgomp-4.1.2passedResult:Packageexistencecheckpassedfor"libgomp-4.1.2"Check:Packageexistencefor"libaio-0.3.106"NodeNameAvailableRequiredComment11grac2libaio-0.3.106-5libaio-0.3.106passedResult:Packageexistencecheckpassedfor"libaio-0.3.106"Check:Packageexistencefor"glibc-2.5-24"NodeNameAvailableRequiredComment11grac2glibc-2.5-81glibc-2.5-24passedResult:Packageexistencecheckpassedfor"glibc-2.5-24"Check:Packageexistencefor"compat-libstdc++-33-3.2.3"NodeNameAvailableRequiredComment11grac2compat-libstdc++-33-3.2.3-61compat-libstdc++-33-3.2.3passedResult:Packageexistencecheckpassedfor"compat-libstdc++-33-3.2.3"Check:Packageexistencefor"elfutils-libelf-0.125"NodeNameAvailableRequiredComment11grac2elfutils-libelf-0.137-3.el5elfutils-libelf-0.125passedResult:Packageexistencecheckpassedfor"elfutils-libelf-0.125"Check:Packageexistencefor"elfutils-libelf-devel-0.125"NodeNameAvailableRequiredComment11grac2elfutils-libelf-devel-0.137-3.el5elfutils-libelf-devel-0.125passedResult:Packageexistencecheckpassedfor"elfutils-libelf-devel-0.125"Check:Packageexistencefor"glibc-common-2.5"NodeNameAvailableRequiredComment11grac2glibc-common-2.5-81glibc-common-2.5passedResult:Packageexistencecheckpassedfor"glibc-common-2.5"Check:Packageexistencefor"glibc-devel-2.5"NodeNameAvailableRequiredComment11grac2glibc-devel-2.5-81glibc-devel-2.5passedResult:Packageexistencecheckpassedfor"glibc-devel-2.5"Check:Packageexistencefor"glibc-headers-2.5"NodeNameAvailableRequiredComment11grac2glibc-headers-2.5-81glibc-headers-2.5passedResult:Packageexistencecheckpassedfor"glibc-headers-2.5"Check:Packageexistencefor"libaio-devel-0.3.106"NodeNameAvailableRequiredComment11grac2libaio-devel-0.3.106-5libaio-devel-0.3.106passedResult:Packageexistencecheckpassedfor"libaio-devel-0.3.106"Check:Packageexistencefor"libgcc-4.1.2"NodeNameAvailableRequiredComment11grac2libgcc-4.1.2-52.el5libgcc-4.1.2passedResult:Packageexistencecheckpassedfor"libgcc-4.1.2"Check:Packageexistencefor"libstdc++-4.1.2"NodeNameAvailableRequiredComment11grac2libstdc++-4.1.2-52.el5libstdc++-4.1.2passedResult:Packageexistencecheckpassedfor"libstdc++-4.1.2"Check:Packageexistencefor"libstdc++-devel-4.1.2"NodeNameAvailableRequiredComment11grac2libstdc++-devel-4.1.2-52.el5libstdc++-devel-4.1.2passedResult:Packageexistencecheckpassedfor"libstdc++-devel-4.1.2"Check:Packageexistencefor"sysstat-7.0.2"NodeNameAvailableRequiredComment11grac2sysstat-7.0.2-11.el5sysstat-7.0.2passedResult:Packageexistencecheckpassedfor"sysstat-7.0.2"Check:Packageexistencefor"unixODBC-2.2.11"NodeNameAvailableRequiredComment11grac2unixODBC-2.2.11-10.el5unixODBC-2.2.11passedResult:Packageexistencecheckpassedfor"unixODBC-2.2.11"Check:Packageexistencefor"unixODBC-devel-2.2.11"NodeNameAvailableRequiredComment11grac2unixODBC-devel-2.2.11-10.el5unixODBC-devel-2.2.11passedResult:Packageexistencecheckpassedfor"unixODBC-devel-2.2.11"Check:Packageexistencefor"ksh-20060214"NodeNameAvailableRequiredComment11grac2ksh-20100621-5.el5ksh-20060214passedResult:Packageexistencecheckpassedfor"ksh-20060214"CheckingformultipleuserswithUIDvalue0Result:CheckformultipleuserswithUIDvalue0passedCheckingtomakesureuser"grid"isnotin"root"groupNodeNameStatusComment11grac2doesnotexistpassedResult:User"grid"isnotpartof"root"group.CheckpassedStartingClocksynchronizationchecksusingNetworkTimeProtocol<NTP>...NTPConfigurationfilecheckstarted...NetworkTimeProtocol<NTP>configurationfilenotfoundonanyofthenodes.OracleClusterTimeSynchronizationService<CTSS>canbeusedinsteadofNTPfortimesynchronizationontheclusternodesResult:ClocksynchronizationcheckusingNetworkTimeProtocol<NTP>passedPre-checkfornodeadditionwassuccessful.[grid@11grac1bin]$〔九擴展新節(jié)點到集群中〔期間傳文件可能會花段時間[grid@11grac1bin]$pwd[grid@11grac1bin]$./addNode.sh-silent"cluster_new_nodes={11grac2}""cluster_new_virtual_hostnames={11grac2-vip}"StartingOracleUniversalInstaller...Checkingswapspace:mustbegreaterthan500MB.Actual4094MBPassedOracleUniversalInstaller,Version.0ProductionCopyright<C>1999,2009,Oracle.Allrightsreserved.Performingteststoseewhethernodes11grac2areavailable100%DoneClusterNodeAdditionSummaryGlobalSettingsNewNodesSpaceRequirementsNewNodes11grac2/:Required3.46GB:Available14.46GBInstalledProductsProductNamesOracleGridInfrastructure.0SunJDK7.0InstallerSDKComponent.0OracleOne-OffPatchInstaller.2OracleUniversalInstaller.0OracleConfigurationManagerDeconfiguration.0EnterpriseManagerCommonCoreFiles.2OracleDBCADeconfiguration.0OracleRACDeconfiguration.0OracleQualityofServiceManagement<Server>.0InstallationPluginFiles.0UniversalStorageManagerFiles.0OracleTextRequiredSupportFiles.0AutomaticStorageManagementAssistant.0OracleDatabase11gMultimediaFiles.0OracleMultimediaJavaAdvancedImaging.0OracleGlobalizationSupport.0OracleMultimediaLocatorRDBMSFiles.0OracleCoreRequiredSupportFiles.0BaliShare.0OracleDatabaseDeconfiguration.0OracleQualityofServiceManagement<Client>.0Expatlibraries.1OracleContainersforJava.0PerlModules.1SecureSocketLayer.0OracleJDBC/OCIInstantClient.0OracleMultimediaClientOption.0LDAPRequiredSupportFiles.0CharacterSetMigrationUtility.0PerlInterpreter.1PL/SQLEmbeddedGateway.0OLAPSQLScripts.0DatabaseSQLScripts.0OracleExtendedWindowingToolkit.0SSLRequiredSupportFilesforInstantClient.0SQL*PlusFilesforInstantClient.0OracleNetRequiredSupportFiles.0OracleDatabaseUserInterface.0RDBMSRequiredSupportFilesforInstantClient.0EnterpriseManagerMinimalIntegration.0XMLParserforJava.0OracleSecurityDeveloperTools.0OracleWalletManager.0EnterpriseManagerpluginCommonFiles.0PlatformRequiredSupportFiles.0OracleJFCExtendedWindowingToolkit.0RDBMSRequiredSupportFiles.0OracleIceBrowser.0OracleHelpForJava.0EnterpriseManagerCommonFiles.2DeinstallationTool.0OracleJavaClient.0ClusterVerificationUtilityFiles.0OracleNotificationService<eONS>.0OracleLDAPadministration.0ClusterVerificationUtilityCommonFiles.0OracleClusterwareRDBMSFiles.0OracleLocaleBuilder.0OracleGlobalizationSupport.0BuildtoolsCommonFiles.0OracleRACRequiredSupportFiles-HAS.0SQL*PlusRequiredSupportFiles.0XDKRequiredSupportFiles.0AgentRequiredSupportFiles.2ParserGeneratorRequiredSupportFiles.0PrecompilerRequiredSupportFiles.0InstallationCommonFiles.0RequiredSupportFiles.0OracleJDBC/THINInterfaces.0OracleMultimediaLocator.0OracleMultimedia.0HASCommonFiles.0AssistantCommonFiles.0PL/SQL.0HASFilesforDB.0OracleRecoveryManager.0OracleDatabaseUtilities.0OracleNotificationService.0SQL*Plus.0OracleNetcaClient.0OracleNet.0OracleJVM.0OracleInternetDirectoryClient.0OracleNetListener.0ClusterReadyServicesFiles.0OracleDatabase11g.0Instantiatingscriptsforaddnode<Wednesday,August29,20121:04:40PMCST>.1%Done.InstantiationofaddnodescriptscompleteCopyingtoremotenodes<Wednesday,August29,20121:04:51PMCST>96%Done.HomecopiedtonewnodesSavinginventoryonnodes<Wednesday,August29,20121:19:31PMCST>.100%Done.SaveinventorycompleteWARNING:Anewinventoryhasbeencreatedononeormorenodesinthissession.However,ithasnotyetbeenregisteredasthecentralinventoryofthissystem.Toregisterthenewinventorypleaserunthescriptat'/u01/app/oraInventory/orainstRoot.sh'withrootprivilegesonnodes'11grac2'.Ifyoudonotregistertheinventory,youmaynotbeabletoupdateorpatchtheproductsyouinstalled.Thefollowingconfigurationscriptsneedtobeexecutedasthe"root"userineachclusternode./u01/app/oraInventory/orainstRoot.sh#Onnodes11grac2/u01/11.2.0/grid/root.sh#Onnodes11grac2Toexecutetheconfigurationscripts:1.Openaterminalwindow2.Loginas"root"3.RunthescriptsineachclusternodePleasecheck'/tmp/silentInstall.log'formoredetails.〔十按說明執(zhí)行腳本[root@11grac2~]#/u01/app/oraInventory/orainstRoot.shCreatingtheOracleinventorypointerfile</etc/oraInst.loc>Changingpermissionsof/u01/app/oraInventory.Addingread,writepermissionsforgroup.Removingread,write,executepermissionsforworld.Changinggroupnameof/u01/app/oraInventorytooinstall.Theexecutionofthescriptiscomplete.RunningOracle11groot.shscript...Thefollowingenvironmentvariablesaresetas:ORACLE_OWNER=gridEnterthefullpathnameofthelocalbindirectory:[/usr/local/bin]:yCreatingydirectory...Copyingdbhometoy...Copyingoraenvtoy...Copyingcoraenvtoy...Entrieswillbeaddedtothe/etc/oratabfileasneededbyDatabaseConfigurationAssistantwhenadatabaseiscreatedFinishedrunninggenericpartofroot.shscript.Nowproduct-specificrootactionswillbeperformed.2012-08-2913:23:07:Parsingthehostname2012-08-2913:23:07:Checkingforsuperuserprivileges2012-08-2913:23:07:UserhassuperuserprivilegesCreatingtracedirectoryLOCALADDMODECreatingOCRkeysforuser'root',privgrp'root'..Operationsuccessful.AddingdaemontoinittabCRS-4123:OracleHighAvailabilityServiceshasbeenstarted.ohasdisstartingCRS-4402:TheCSSdaemonwasstartedinexclusivemodebutfoundanactiveCSSdaemononnode11grac1,number1,andisterminatingCRS-2673:Attemptingtostop'ora.cssdmonitor'on'11grac2'CRS-2677:Stopof'ora.cssdmonitor'on'11grac2'succeededCRS-2673:Attemptingtostop'ora.gpnpd'on'11grac2'CRS-2677:Stopof'ora.gpnpd'on'11grac2'succeededCRS-2673:Attemptingtostop'ora.gipcd'on'11grac2'CRS-2677:Stopof'ora.gipcd'on'11grac2'succeededCRS-2673:Attemptingtostop'ora.mdnsd'on'11grac2'CRS-2677:Stopof'ora.mdnsd'on'11grac2'succeededAnactiveclusterwasfoundduringexclusivestartup,restartingtojointheclusterCRS-2672:Attemptingtostart'ora.mdnsd'on'11grac2'CRS-2676:Startof'ora.mdnsd'on'11grac2'succeededCRS-2672:Attemptingtostart'ora.gipcd'on'11grac2'CRS-2676:Startof'ora.gipcd'on'11grac2'succeededCRS-2672:Attemptingtostart'ora.gpnpd'on'11grac2'CRS-2676:Startof'ora.gpnpd'on'11grac2'succeededCRS-2672:Attemptingtostart'ora.cssdmonitor'on'11grac2'CRS-2676:Startof'ora.cssdmonitor'on'11grac2'succeededCRS-2672:Attemptingtostart'ora.cssd'on'11grac2'CRS-2672:Attemptingtostart'ora.diskmon'on'11grac2'CRS-2676:Startof'ora.diskmon'on'11grac2'succeededCRS-2676:Startof'ora.cssd'on'11grac2'succeededCRS-2672:Attemptingtostart'ora.ctssd'on'11grac2'CRS-2676:Startof'ora.ctssd'on'11grac2'succeededCRS-2672:Attemptingtostart'ora.asm'on'11grac2'CRS-2676:Startof'ora.asm'on'11grac2'succeededCRS-2672:Attemptingtostart'ora.crsd'on'11grac2'CRS-2676:Startof'ora.crsd'on'11grac2'succeededCRS-2672:Attemptingtostart'ora.evmd'on'11grac2'CRS-2676:Startof'ora.evmd'