RAC Assurance Support Team: RAC Starter Kit and Best Practices (Windows) [ID 811271.1]

		修改时间 14-JUN-2011 类型 BULLETIN 状态 PUBLISHED

In this Document
Purpose
Scope and Application
RAC Assurance Support Team: RAC Starter Kit and Best Practices (Windows)
RAC Assurance Support Team: RAC and Oracle Clusterware Starter Kit and Best Practices (Generic)
RAC Platform Specific Starter Kits and Best Practices
RAC on Windows Step by Step Installation Instructions
RAC on Windows Highlighted Recommendations
RAC on Windows Best Practices
Tool/Utility
Networking Considerations
Storage Considerations
Memory Considerations
RAC on Windows References
Windows 2008 Specific References
References

Applies to:

Oracle Server - Enterprise Edition - Version: 10.2.0.1 to 11.2.0.1.0 - Release: 10.2 to 11.2
Microsoft Windows (32-bit)
Microsoft Windows Itanium (64-bit)
Microsoft Windows x64 (64-bit)

Purpose

The goal of the Oracle Real Application Clusters (RAC) Starter Kit is to provide you with the latest information on generic and platform specific best practices for implementing an Oracle RAC cluster. This document is compiled and provided based on Oracle Support's experience with its global RAC customer base.

This Starter Kit is not meant to replace or supplant the Oracle Documentation set, but rather, it is meant as a supplement to the same. It is imperative that the Oracle Documentation be read, understood, and referenced to provide answers to any questions that may not be clearly addressed by this Starter Kit.

All recommendations should be carefully reviewed by your own operations group and should only be implemented if the potential gain as measured against the associated risk warrants implementation. Risk assessments can only be made with a detailed knowledge of the system, application, and business environment.

As every customer environment is unique, the success of any Oracle Database implementation, including implementations of Oracle RAC, is predicated on a successful test environment. It is thus imperative that any recommendations from this Starter Kit are thoroughly tested and validated using a testing environment that is a replica of the target production environment before being implemented in the production environment to ensure that there is no negative impact associated with the recommendations that are made.

Scope and Application

This article is intended for use by all new (and existing) Oracle RAC implementers.

RAC Assurance Support Team: RAC Starter Kit and Best Practices (Windows)

RAC Assurance Support Team: RAC and Oracle Clusterware Starter Kit and Best Practices (Generic)

The following note focuses on Generic RAC Best Practices that are applicable to all platforms including a white paper on available RAC System Load Testing Tools and RAC System Test Plan outlines for 10gR2 & 11gR1 and 11gR2:

Document 810394.1 RAC Assurance Support Team: RAC and Oracle Clusterware Starter Kit and Best Practices (Generic)

RAC Platform Specific Starter Kits and Best Practices

The following notes contain detailed platform specific best practices including Step-By-Step installation cookbooks.

Document 811306.1 RAC Assurance Support Team: RAC Starter Kit and Best Practices (Linux)

Document 811280.1 RAC Assurance Support Team: RAC Starter Kit and Best Practices (Solaris)

Document 811271.1 RAC Assurance Support Team: RAC Starter Kit and Best Practices (Windows)

Document 811293.1 RAC Assurance Support Team: RAC Starter Kit and Best Practices (AIX)

Document 811303.1 RAC Assurance Support Team: RAC Starter Kit and Best Practices (HP-UX)

RAC on Windows Step by Step Installation Instructions

Click here for a Step By Step guide for installing Oracle RAC 10gR2 on Windows (2003 and 2008)

Click here for a Step By Step guide for installing Oracle RAC 11gR1 on Windows (2003 and 2008)

Click here for a Step By Step guide for installing Oracle RAC 11gR2 on Windows (2003 and 2008)

RAC on Windows Highlighted Recommendations

This is a list of the core best practices and references identified by the RAC Assurance Team relevant to Windows specific RAC implementations. The rest of this chapter provides additional best practices and references from our team.

Note that it is now possible to request one-off patches on Windows Platforms. For detailed information, please review the following article: Note:1122596.1 One off Patch process for the RDBMS bundled patch on Microsoft Windows. Please note that one-off requests (on Windows) are always subject to the discretion of our bug diagnostics and escalations support team.
Disable Anti-Virus software running on cluster nodes before and for the entire duration of the installation Oracle 11gR2 on Windows (be mindful of manual reboots during this time). Anti-Virus software may (of course) be re-enabled following the installation.
Please be aware of the following alert for RAC on Windows customers: Note 1051503.1 : RACG Problem with Windows 10204 Patch 28, 29, 30 and 31 Bundles
Do not set ORACLE_HOME as an environment variable in RAC on Windows environments. If needed (for example, when running opatch) set this variable as needed in a command prompt window.
Install the Oracle Process Manager Daemon (OPMD) to manage the proper startup of Oracle clusterware services. OPMD is available with 10.2.0.3 and 10.2.0.4 patchset installations, but you must issue the command: opmd -install on each node. For 10.2.0.4 and 11gR1, OPMD is automatically configured upon clusterware installation. Reference: Note 358156.1
Workload management services created and configured in the OCR do not register properly with the listener if the instance names were entered into the OCR in UPPER CASE. Reference: Note 731287.1
Windows should be optimized for Memory Usage of Programs not System Caching

Start -> Settings -> Control Panel -> System -> Advanced -> Performance -> Memory Usage: Adjust for best performance of -> Programs instead of System Caching

RAC on Windows Best Practices

Tool/Utility

DHCP Media sensing MUST be disabled - This is a requirement outlined in the RAC installation guide for the Windows platform in the section entitled: Network Hardware Requirements. DHCP Media Sensing is disabled by default on Windows 2003. This change must be manually implemented for Windows 2000. Instructions for implementing this change can be found on the Microsoft support website using the following link: http://support.microsoft.com/default.aspx?scid=kb;en-us;239924

Run Perfmon to monitor CPU, Memory, Network, Disk IO Rates - To aid in troubleshooting, configure Perfmon to monitor these OS statistics and to generate binary log files (.BLG). Instructions for implementing this change can be found on the Microsoft support website using the following link: http://support.microsoft.com/kb/146005

Download and implement Debugging Tools for Windows - It is a good best practice to download and install Debugging Tools for Windows (containing, among others, adplus and windbg) on each node of your RAC on Windows cluster in case those tools are needed while troubleshooting. Instructions for implementing this change can be found on the Microsoft support website using the following link: http://www.microsoft.com/whdc/devtools/debugging/default.mspx

Download and familiarize the DBA team with useful Windows utilities (such as those available through the sysinternals website: http://www.sysinternals.com/ for example: Process Explorer) that can assist in troubleshooting.

Set /USEPMTIMER in the boot.ini. Reference Note 437101.1 EXCESSIVE LMS AND LMD TRACE FILE SIZES GENERATED ON WINDOWS RAC

There is a general requirement for Oracle RAC that the time on all the nodes be the same. If the Windows Time Service is being used, it MUST be configured to prevent the time from being adjusted backwards. Perform the following steps to ensure the time is NOT adjusted backwards using Windows Time Service perform the following steps on each cluster node.

Open a command prompt and type “regedit”.
Within the registry editor locate the HKEY_LOCAL_MACHINESYSTEMCurrentControlSetServicesW32TimeConfig key.
Set the value for MaxNegPhaseCorrection? to 0 and exit the registry editor.
Open a command prompt and execute the following to put the change into effect:

cmd> W32tm /config /update

With 11gR2 time synchronization can be performed by the Clusterware using CTSSD (Cluster Time Synchronization Services Daemon) or by using the Windows Time Service.

Networking Considerations

After installing Windows Server 2003 Service Pack 2 (SP2) or Windows Server 2003 Scalable Networking Pack (SNP), turn off default SNP features. On a computer that has a TCP/IP Offload-enabled network adapter, you may experience many network-related problems like network adapters consuming lots of nonpaged pool memory or adapters requesting large blocks of contiguous memory causing the computer to stop responding when it tries to free the memory. This problem also affects Windows 2008 operating systems. See My Oracle Support Note:988008.1 and Microsoft article: http://support.microsoft.com/kb/948496/en-us for more details.

SQLNET.AUTHENTICATION_SERVICES=(NONE) will prevent CRS resources from starting. If sqlnet.authentication_services (in the sqlnet.ora file) is set to (NONE) instance startup by CRS fails as "USR_ORA_CONNECT_STR=/ as sysdba" is used when CRS starts up resources as Oracle and without NTS this does not work. Ensure SQLNET.AUTHENTICATION_SERVICES=NTS in all Oracle homes to allow sysdba authentication.

Do not use the names: PUBLIC and PRIVATE (all caps) for your public and interconnect networks (NICs) due to unpublished code defect Bug 6844099. The words public and private themselves may be used, for example: Public and Private are acceptable.

Please note that IPv6 addressing is currently not (yet) supported with RAC. For more information, reference: http://stcontent.oracle.com/content/dav/oracle/Users/Users-K/kant.patel/IPv6/OracleDatabase_IPv6_SOD.pdf

Storage Considerations

It is strongly advised to bring the entire Oracle software stack down in order to complete all disk virus scans for conventional Fat 16/32/NTFFS as well as OCFS file systems. This is because Oracle and the virus scan software use different types of locking which are not compatible. Hence a shared disk for the database configured with OCFS could have 2 nodes virus scanning at the same time and could potentially cause the cluster to crash. We strongly suggest that you virus scan only from one node and only during maintenance windows. OCFS disks that only contain Oracle database datafiles do not need to be virus scanned. OCFS disks that contain any non-database datafiles or database configuration files should be scanned periodically (with the entire Oracle stack down).

Memory Considerations

A good rule of thumb for Oracle RAC on Windows environments is to keep memory allocation under 80%. We recommend shooting for 75% allocated, that is, more than 20-25% free. This will allow for ample memory needed for Windows OS operations (including collection of physical memory dumps if required)

RAC on Windows References

Oracle Database Oracle Clusterware and Oracle Real Application Clusters Installation Guide 10g Release 2 (10.2) for Microsoft Windows: http://download.oracle.com/docs/cd/B19306_01/install.102/b14207/toc.htm
Oracle Clusterware Installation Guide
11g Release 1 (11.1) for Microsoft Windows: http://download.oracle.com/docs/cd/B28359_01/install.111/b28250/toc.htm
Document 254611.1 Shared Partition Errors in RAC Configuration on Windows 2003
Document 358156.1 Automatic Startup of Cluster Services May Hang or Fail on Windows
Document 341214.1 How To clean up after a Failed (or successful) Oracle Clusterware Installation on Windows
Document 415526.1 ERROR: "FAILED TO PATCH ALL THE FILES IN THE ORA_CRS_HOME..." running patch102.bat on Windows
Document 161549.1 Oracle Database Server and Networking Patches for Microsoft Platforms
Document 342443.1 10.2.0.x Oracle Database and Networking Patches for Microsoft Platforms
Document 244241.1 Rolling Patch - OPatch Support for RAC
Document 464683.1 Unexplained Database Slowdown Seen on Windows 2003 Service Pack 1
Document 454607.1 New Partitions in Windows 2003 RAC Environments Not Visible on Remote Nodes
Document 436067.1 WINDOWS CRS_STAT SCRIPT TO DISPLAY LONG NAMES CORRECTLY
Document 437101.1 EXCESSIVE LMS AND LMD TRACE FILE SIZES GENERATED ON WINDOWS RAC
Document 558769.1 RAC on Windows: OCSSD.EXE Is Not Copied Over When Applying mini patches Until Mini patch 19
Document 332001.1 SQLNET.AUTHENTICATION_SERVICES=(NTS) Must Be Set For OCR Status Accuracy
Document 161549.1 Oracle Database Server and Networking Patches for Microsoft Platforms
Document 373040.1 How To Identify and Locate the Latest Windows Mini-Patch For Your Patchset Version
Document 412851.1 Oracle Process Manager (Opmd) Does Not Start After Patching to 10.2.0.3
Document 731287.1 RAC on Windows: Workload Management Services Are Not Automatically Registered With The Listener
Document 729512.1 How to Start Oracle Clusterware 10gR2 Services Manually in Windows
Document 743126.1 RAC on Windows: How to Apply Oracle Windows Mini Patch Bundles in a Rolling Fashion
Document 737957.1 ALERT: Possible Memory Corruption After Applying 10.2.0.4 Patch 5 Or Higher on Windows
Document 811151.1 How to install Oracle Instantaneous Problem Detector for Clusters (IPD/OS) on Windows
Document 810915.1 How to monitor, detect and analyze OS and RAC resource related degradation and failures on Windows
Document 782450.1 RAC on Windows: How to Create Raw Logical Volumes Using
Diskpart Command Line Utility
Document 866258.1 DATABASE CREATION MAY HANG ON WINDOWS MACHINES WITH > 32 CORES
Document 1054027.1 RAC on Windows: Is NAS Shared Storage Supported for use With RAC on Windows?
Document 876374.1 RAC on Windows: How to Check That the Orafence Driver Was Correctly Updated After Applying a Patchset
Note 988008.1 RAC on Windows: Recurring Node Evictions May Be Caused by Default SNP Features Available for Windows Server 2003 SP2
Document 1122596.1 One off Patch process for the RDBMS bundled patch on Microsoft Windows
Document 1069034.1 How to Manually Remove Oracle Server Software on Microsoft Windows Platforms
Document 1271484.1 RAC on Windows: Upgrade to 11.2 Grid Infrastructure Leaves Some Clusterware Related Services Running out of the Wrong (Older) Home
Document 1273880.1 PRKN-1014 Error in DBCA and/or ASMCA After Applying 11.2.0.1 Patch 5 Bundle and Higher
Note 1252334.1 Grid Infrastructure Configuration Fails as MDNSD can not Register on Network Adapter with Invalid Mask 0.0.0.0

Windows 2008 Specific References

Document 782795.1 RAC on Windows: Oracle Clusterware Installation Steps Particular for Windows 2008 Clusters
Document 762193.1 Installing CRS on Windows 2008 Fails When Checking OCFS and
Orafence Driver's Signatures
Microsoft Article: All IP addresses are registered on the DNS servers when the IP addresses are assigned to one network adapter on a computer that is running Windows Server 2008 SP2 or Windows Vista SP2. http://support.microsoft.com/?kbid=975808

Modification History
[14-Jun-11] added reference to Note 1252334.1
[11-Aug-2009] created this Modification History section
[08-Sep-2009] added reference to Bug 6844099
[29-Jan-2010] added reference to alert Note:1051503.1
[01-Feb-2010] added reference to Note 1054027.1
[02-Feb-2010] added reference to Note 876374.1
[17-Feb-2010] added memory considerations section and statement about 80% rule of thumb
[23-Mar-2010] attached new version of 10gR2 Step by Step (including Windows 2008)
[24-Mar-2010] attached new version of 11gR1 Step by Step (including Windows 2008)
[16-Apr-2010] added reference to Note:988008.1
[21-Apr-2010] added links to generic Starter Kit as well as other platform specific starter kits
[29-Apr-2010] added Time Sync recommendation
[05-May-2010] added Rac11gR2OnWindows attachment
[07-May-2010] added reference to http://support.microsoft.com/?kbid=975808
[03-Jun-2010] updated versions to include 11.2
[24-Jun-2010] added reference to possibility for one-off patch requests on Windows
[01-Oct-2010] added Anti-Virus suggestion to 'Highlighted Recommendations'
[18-Nov-2010] added reference to 1069034.1
[24-Nov-2010] added reference to IPv6 position paper
[03-Dec-2010] added reference to upgrade document 1271484.1
[14-Dec-2010] added reference to code defect described in 1273880.1