Step-By-Step Installation of 9i RAC on Sun Cluster v3

Oracle, Oracle RAC   9i, installation, RAC  

2 comments to Step-By-Step Installation of 9i RAC on Sun Cluster v3

• admin

  April 12, 2011 at 4:52 PM · Reply

  Introduction to Sun Cluster v3
  1.0 Purpose

  Sun™ Cluster v3 was launched at the end of 2000 and is a complete re-design (i.e. not built on Sun Cluster v2 technologies) and consequently is radically different from older versions of the Sun Cluster product. This notes intends to act as an introduction to this product.
  2.0 Scope & Application

  This introduction will act as a simple overview of the product for the novice but will also delve deeper into some more technical issues relating to the product.
  3.0 Introduction to Sun Cluster v3

  Sun™ Cluster 3.0 is Sun’s next-generation clustering technology. Based on technology developed by Sun Labs, it delivers Sun’s most powerful cluster solution. Sun Cluster 3.0, according to Sun, focuses on delivering integrated availability, scalability, manageability, and ease of use with the core delivery platform — the Solaris Operating System (SPARC)™ Environment. Key to the SunPlex™ system (SunPlex™ is the “family” name for Sun Cluster – previously code-named internally as “full-moon”) is Sun Cluster 3.0. Built around Sun Cluster 3.0, as well as the Solaris 8 Operating System (SPARC) Environment and Sun server, storage, and network connectivity products, and services, a SunPlex system is designed to manage application services for tightly coupled environments.

  Sun Cluster (SC) v3 extends the Solaris Operating System (SPARC) environment into a cluster operating system. A cluster is a collection of loosely coupled computing nodes that provides a single client view of network services or applications, including databases, web services, and file services. Each cluster node is a standalone server that runs its own processes. These processes can communicate with one another to form what looks like (to a network client) a single system that co-operatively provides applications, system resources, and data to users. A cluster offers several advantages over traditional single server systems. These advantages include support for highly available and scalable applications, capacity for modular growth, and low entry price compared to traditional hardware fault-tolerant systems. This version is heavily kernalized and consequently faster in operation. Two choices are available for volume management, Veritas Volume Manager and Solstice DiskSuite.
  The goals of Sun Cluster are:-

  Reduce or eliminate system downtime because of software or hardware failure
  Ensure availability of data and applications to end users, regardless of the kind of failure that would normally take down a single server system
  Increase application throughput by enabling services to scale to additional processors by adding nodes to the cluster
  Provide enhanced availability of the system by enabling you to perform maintenance without shutting down the entire cluster

  4.0 Sun Cluster v3 New Features

  The Sun Cluster 3.0 software release has the following new features:-

  Up to eight nodes
  Sun Cluster 3.0 supports up to eight cluster nodes. Sun plans to extend this to 64 nodes eventually. Sun Cluster v2.2 had a previous limit of four nodes. Real Application Clusters (RAC) is supported with Solaris Operating System (SPARC) 8 (also known as v2.8 or v5.8) and Sun Cluster 3.0, though only two nodes are supported and limited four node support at this current stage. The limitation on SC 3.0/RAC is due to the lack of an implementation of SCSI 3 Persistent Group Reservation (PGR) in the storage subsystem. SC 3.0 uses PGR for quorum management for more than 2 nodes. SCSI 2 Reserve is used by SC 2.2 and 3.0 for 2 node clusters. SC 2.2 uses a terminal concentrator for greater than 2 nodes. Sun will support PGR in T3 storage in an update but Sun will not support PGR in earlier storage offerings (e.g. photons – A5200). EMC, however, does supports PGR and these third-party vendors are now responsible for certification against Sun Cluster. Up to four node OPS/RAC is supported with Sun Cluster 3.0 Update 2 (or higher versions) in several configurations. The key component in each four node configuration being the shared storage device. 4 node support is now available for Sun T3 Single Brick (requires VxVM 3.2) and SE 9910/9960 (SUN Enterprise Series, OEM of Hitachi HDS). Please check with Sun for more details and latest updates to node support. Certification to 8 nodes will happen at a later date.

  Cluster File System
  The Cluster File System allows mounting of cluster-wide user file systems (UFS) or High Sierra file systems (HSFS), allowing concurrent, continuous access to the file systems from any node in the cluster.

  Global device access
  Sun Cluster can access disk devices, tape devices, and CD-ROM drives from any node in the cluster.

  Cluster networking (shared address)
  While each node retains its own publicly accessible Internet Protocol (IP) address, a global IP address can be configured for the applications on the cluster, where the data services requests received through the global address are distributed to nodes in the cluster based on a selected load balancing policy.

  Scalable application support
  Sun Cluster 3.0 supports scalable data services in which client requests are distributed to any number of cluster nodes. This is used in conjunction with the shared address feature.

  Sun Management Center-based monitoring
  Sun Cluster 3.0 nodes can be monitored using the Sun Management Center system management tool.

  Solaris Operating System (SPARC) 10/00 Operating Environment is supported
  The Solaris Operating System (SPARC) 8 10/00 (October 2000) or above platform release is recommended because the number of required patches has been significantly reduced. Early versions of Solaris Operating System (SPARC) e.g. v7, v2.6 are not supported. SC v3 has now been kernalized for performance reasons and is, hence, strongly OS version dependant.

  Two new installation methods
  You can now configure the first cluster node and automatically use it to assist in configuring all additional nodes. If you have an existing
  JumpStart™ server (networked installation image server), you can also automatically add the Sun Cluster 3.0 installation to the JumpStart configuration.
  5.0 Cluster Hardware Components

  The minimum hardware components that are necessary for a cluster configuration include:-

  One administration workstation
  The administration workstation can be any Sun workstation, providing it has adequate resources to support graphics and compute intensive applications. A simple Sunblade 100 500MHz workstation is usually recommended by Sun for this purpose. You can use cluster administration tools to monitor many clusters from one administration workstation. This is not mandatory but clearly helpful.

  One Terminal Concentrator
  The Sun terminal concentrator (TC) provides direct translation from the network to serial port interfaces. Each of the serial port outputs connect to a separate node in the cluster through serial port A. The Sun nodes commonly do not have frame buffers so to display the boot prompt on a screen the Terminal Concentrator is the only access path when the operating system is down. While this is also not a mandatory prerequisite it is certainly useful for administration purposes.

  Two hosts (up to eight)
  A wide range of Sun hardware platforms are supported for use in the clustered environment. Mixed platform clusters are not supported.

  One or more public network interfaces per system
  Currently Gigabit Ethernet, ethernet and SCI are supported for the cluster interconnect.

  A private cluster transport interface
  All nodes in a cluster are linked by a private cluster transport. The transport is redundant and can be used for the following purposes:-

  Cluster-wide monitoring and recovery
  Parallel database lock and query information
  Global data access
  Dual hosted, mirrored disk storage

  The Sun Cluster environment can use several Sun and third-party storage devices. They must all accept at least dual-host connections.
  6.0 Memory Requirements

  Sun Cluster 3.0 requires extra memory above what is configured for a node under a normal workload. The extra memory equals 128 Mbytes + 10%. For example, if a standalone node normally requires 1 Gbyte of memory, you need an extra 256 Mbytes to meet memory requirements.
  7.0 Sun Cluster Data Service Support

  Each of the Sun Cluster data services (Sun Cluster 3.0 Agents), provides a control and monitoring framework that enables a standard application to be highly-available or scalable. Some of the data services can be configured for either failover or scalable operation. The list of initially supported agents is below:-
  Data Service

  Sun Cluster Resource Type Name

  Application Version
  SC 3.0 HA for DNS

  dns

  DNS on Solaris 8
  SC HA for NFS

  nfs

  NFS v2,3 on Solaris 8
  SC HA for iPlanet Web Server

  iws

  iPlanet Web Server 4.1
  SC 3.0 HA for Netscape Dir Server

  nsldap

  Netscape Dir Server 4.11
  SC 3.0 HA for Oracle

  oracle

  Oracle 8.1.6 (8i)
  SC HA for Oracle Parallel Server

  N/A

  Oracle 8.1.6 (8i)
  SC 3.0 HA for Apache

  apache

  Apache HTTP 1.39

  8.0 High Availability Strategies

  To help provide the level of system availability required by many customers, the Sun Cluster system uses the following strategies:-

  Redundant servers
  Redundant data
  Redundant public network access
  Redundant private communications (transport)
  Multi-host storage access

  Redundant Servers
  The Sun Cluster system consists of two to eight interconnected systems that are referred to as cluster host systems or nodes. The systems can be any of a range of Sun platforms and use off-the-shelf non-proprietary hardware. Note – You cannot mix systems that use peripheral component interconnect (PCI) bus technology, such as the Sun Enterprise 450 server, with SBus technology systems, such as the Sun Enterprise 3500.

  Redundant Data
  A Sun Cluster system can use either of two virtual volume management packages to provide data redundancy. The use of data mirroring provides a backup in the event of a disk drive or storage array failure.

  Redundant Public Network Interfaces
  The Sun Cluster system provides a proprietary feature, public network management (PNM), that can transfer user I/O from a failed network interface to a predefined backup interface. The switch to the backup interface is transparent to cluster applications and users.

  Redundant Transport Interface
  The cluster transport interface consists of dual high-speed private node-to-node communication interfaces. The cluster software uses only one of the interfaces at a time. If the primary interface fails, the cluster software automatically switches to the backup. This is transparent to cluster applications.
  9.0 Sun Cluster High-Availability Failover
  The Sun Cluster high-availability failover features include the following:-

  Failover Applications
  Node fault monitoring
  Network fault monitoring
  Data service fault monitoring

  10.0 Cluster File Systems

  Cluster file systems are dependent on global devices (disks, tapes, CD-ROMs) with physical connections to one or more nodes. To make a global file system, you create a standard file system on a virtual volume (Veritas or Solstice DiskSuite) and mount the volume on a mount point in the /global directory using special global mount options. A typical mount command is as follows:-

  # mount -g dev/vx/dsk/nfs-dg/vol-Ol /global/nfs

  The equivalent mount entry in the /etc/vfstab file is:

  /dev/vx/dsk/nfs-dg/vol-Ol /dev/vx/rdsk/nfs-dg/vol-Ol /global/nfs ufs 2 yes global,logging

  After the file system is mounted, it is available on all nodes in the cluster.

  Note – Veritas Volume Manager disk groups must be registered with the cluster framework software before any disk group structures can be made globally available. These are not supported currently for RAC datafiles.
  11.0 High Availability in Sun Cluster

  Sun Cluster is designed as a highly available (HA) system, that is, a system that provides near continuous access to data and applications.

  By contrast, fault-tolerant hardware systems provide constant access to data and applications, but at a higher cost because of specialised hardware. Additionally, fault-tolerant systems usually do not account for software failures.

  Sun Cluster achieves high availability through a combination of hardware and software. Redundant cluster interconnects, storage, and public networks protect against single points of failure. The cluster software continuously monitors the health of member nodes and prevents failing nodes from participating in the cluster to protect against data corruption. Also, the cluster monitors applications and their dependent system resources, and fails over or restarts applications in case of failures.
  12.0 Failover and Scalability in Sun Cluster

  Sun Cluster enables you to implement applications on either a failover or scalable basis. Failover and scalable applications can also run on the same cluster concurrently. In general, a failover application provides high availability (redundancy), whereas a scalable application provides high availability along with increased performance. A single cluster can support both failover and scalable applications.

  Failover
  Failover is the process by which the cluster automatically relocates an application from a failed primary node to a designated secondary node. With failover, Sun Cluster provides high availability. When a failover occurs, clients might see a brief interruption in service and might need to reconnect after the failover has finished. However, clients are not aware of the physical server from which they are provided the application and data.

  Scalability
  While failover is concerned with redundancy, scalability provides constant response time or throughput without regard to load. A scalable application leverages the multiple nodes in a cluster to concurrently run an application, thus providing increased performance. In a scalable configuration, each node in the cluster can provide data and process client requests.
  13.0 Sun Cluster Upgrades

  It is possible to upgrade from earlier versions of Sun’s clustering products to Sun Cluster 3.0 (SC3.0). Only 2-node(1) upgrades from Sun Cluster 2.2 (SC2.2) will be supported. Changing hardware configurations during the upgrade process will not be supported (2). Also, at the time of upgrade, the framework and all data services must be upgraded. The upgrade cannot occur piecemeal. e.g., data services cannot be upgraded separately at a later time. Since SC3.0 is a major release for Sun’s clustering product, the entire cluster must be stopped before upgrading can occur. Services will be unavailable while the cluster is being upgraded to SC3.0. At this time it is not known how long the upgrade process will take.

  Notes:-
  1. Most cluster configurations existing today, approximately 97%, are 2-node clusters.
  2. Due to hardware incompatibility between Solstice HA 1.3 (HA1.3) and SC3.0, upgrades from HA1.3 are no longer supported.
  14.0 Supported Software and Memory Requirements for Sun Cluster v3.0

  Operating environment and patches
  - Supported Solaris Operating System (SPARC) versions and patch information are available through the following web sites:-

  http://www.sun.com/software/cluster and
  http://www.sun.com/software/solaris/cluster/specs.xml

  Volume managers
  - Solstice DiskSuite™ 4.2.1+
  - VERITAS Volume Manager 3.0.4+ (see Note:178644.1 – Veritas Volume Manager on Solaris & Real Application Clusters)

• admin

  April 12, 2011 at 4:54 PM · Reply

  Oracle Real Application Clusters on Sun Cluster v3
  1.0 Purpose

  With the launch of two new clustering technologies, namely Sun Cluster (SC) v3 and Oracle’s Real Application Clusters (RAC), within a year of each other, this document intends to describe their use together.
  2.0 Scope & Application

  This document intends to act as an introduction to the specifics of running RAC on Sun Cluster v3.
  3.0 Oracle OPS and RAC on Sun Cluster v3

  Sun Cluster v3 has been certified for use with Oracle Parallel Server under v8.1.7 (32-bit & 64-bit) and also with Real Application Clusters – both under Solaris Operating System (SPARC) 8 (also known as v2.8 or v5.8). It should be noted that Sun does not plan to support RAC on Sun Cluster 2.2. This configuration has has been tested but without Sun’s support, this cannot be considered as supported or certified. For RAC on Unix, we certify against the Unix and Clusterware version. The vendor is responsible for the hardware certification.

  The Oracle Parallel Server and Real-Application Cluster configurations are characterized by two nodes that access a single database image. OPS/RAC configurations are often used for throughput applications. When a node fails, an application does not move to a backup system. OPS/RAC uses a distributed lock management (DLM or IDLM) scheme to prevent simultaneous data modification by two hosts. The lock ownership information is transferred between cluster hosts across the cluster transport system. When a failure occurs, most of the recovery work is performed by the OPS/RAC software, which resolves incomplete database transactions. The Sun Cluster software performs a relatively minor role. It initiates a portion of the database recovery process. RAC makes greater use of interconnect traffic for inter-node communications and, hence, supports SCI, regular Ethernet and Gigabit Ethernet for its interconnect technology.
  4.0 Node Support

  Sun Cluster 3.0 supports up to eight cluster nodes. Sun plan to extend this to 64 nodes eventually. Sun Cluster v2.2 had a previous limit of four nodes. Real Application Clusters (RAC) is supported with Solaris Operating System (SPARC) 8 and Sun Cluster 3.0, though only two nodes are supported and limited four node support at this current stage. The limitation on SC 3.0/RAC is due to the implementation of SCSI 3 Persistent Group Reservation (PGR in the storage subsystem. SC 3.0 uses PGR for quorum management for more than 2 nodes. SCSI 2 Reserve is used by SC 2.2 and 3.0 for 2 node clusters. SC 2.2 uses a terminal concentrator for greater than 2 nodes. Sun will support PGR in T3 storage in an update but Sun will not support PGR in earlier storage offerings (e.g., photons – A5200). EMC, however, does supports PGR and these third-party vendors are now responsible for certification against Sun Cluster. Up to four node OPS/RAC is supported with Sun Cluster 3.0 Update 2 (or higher versions) in several configurations. The key component in each four node configuration being the shared storage device. 4 node support is now available for Sun T3 Single Brick (requires VxVM 3.2) and SE 9910/9960 (SUN Enterprise Series, OEM of Hitachi HDS). Please check with Sun for more details and latest updates to node support. Certification to 8 nodes will happen at a later date.
  5.0 Volume Manager Support

  Two volume managers are supported with Sun Cluster v3:-

  Sun’s Solstice DiskSuite™ 4.2.1+
  Veritas Volume Manager™ 3.0.4+ (32-bit Oracle), 3.1.1+ (64-bit Oracle)
  (see Note:178644.1 – Veritas Volume Manager on Solaris & Real Application Clusters)

  Although Sun’s Solstice DiskSuite (SDS), integrated into Solaris Operating System (SPARC) 9 onwards as Solaris Volume Manager (SVM), is supported by Sun Cluster v3, neither SDS or SVM supports cluster wide volume management prior to Sun Cluster 3.1 Update 3, it is only for per node basis.

  Sun’s SVM and QFS are supported for Oracle 9i with the following software stack from Sun and the required Oracle patches.

  For SVM
  – Solaris 9 9/04 (Solaris 9 update 7)
  – SVM Patch 116669-03, this is required SUN patch
  – Sun Cluster 3.1 Update 3
  – Oracle 9.2.0.5 + Oracle patch 3366258

  For SharedQFS
  – Solaris 9 04/03 and above or Solaris 8 02/02 and above
  – QFS 4.2
  – Sun Cluster 3.1 Update 2 and above
  – Oracle 9.2.0.5 + Oracle patch 3566420

  In the case of the latter product, care should be taken when installing SC v3.0 with VxVM v3.0.4+ due to the requirement to match the vxio drive major numbers. It is important to ensure that the vxio driver major numbers are the same on all cluster nodes and that the number is unique. This problem is overcome in SC v3 update 1 where a utility scvxinstall has been incorporated (previously this was only available on the “Sun Cluster v3 Cool Stuff CD” downloadable from http://www.sun.com/clusters).

  When using the Veritas Volume Manager an additional license is required for the Cluster Volume Manager (CVM). Failure to install the VxVM cluster license correctly might result in a panic if OPS/RAC support is installed without VxVM functioning properly. Prior to installing the OPS/RAC packages, run the vxlicense -p check command to ensure that a valid cluster license is installed.
  6.0 The Proxy File System

  The cluster file system is based on the proxy file system (PXFS), which has the following features:-

  PXFS makes file access locations transparent. A process can open a file located anywhere in the global file system structure, and processes on all nodes can use the same path name to locate a file.
  PXFS uses coherency protocols to preserve the UNIX file access semantics even if the file is accessed concurrently from multiple nodes.
  PXFS provides continuous access to data, even when failures occur. Applications do not detect failures as long as a path to disks is still operational. This guarantee is maintained for raw disk access and all file system operations.

  Note:- PXFS is not a distinct file system type. That is, clients see the underlying file system type (for example, ufs).

  Due to the overhead of this global file system on SC 3, the GFS is supported for Oracle binaries and archive logs only and not for database files. .
  7.0 Sun Cluster Data Services – Sun Cluster HA for Oracle Parallel Server

  You can configure OPS/RAC to use the shared disk architecture of Sun Cluster. In this configuration, a single database is shared among multiple instances of OPS/RAC that access the database concurrently. Conflicting access to the same data is controlled by means of the Oracle UNIX Distributed Lock Manager (UDLM). If a process or a node crashes, the UDLM is reconfigured to recover from the failure. In the event of a node failure in an OPS/RAC environment, you can configure Oracle clients to reconnect to the surviving server without the use of the IP failover used by Sun Cluster failover data services.

  The Sun Cluster install CD’s contain the required SC udlm package:-

  Package SUNWudlm Sun Cluster Support for Oracle Parallel Server UDLM, (opt) on SunCluster v3

  To install use the pkgadd command:-

  # pkgadd -d . SUNWudlm

  Once installed, Oracle’s interface with this, the Oracle UDLM, can be installed.

  To install Sun Cluster Support for OPS/RAC with VxVM, the following Sun Cluster 3 Agents data services packages need to be installed as superuser (see Sun’s Sun Cluster 3 Data Services Installation and Configuration Guide):-

  # pkgadd -d . SUNWscucm SUNWudlmr SUNWcvmr SUNWcvm (SUNWudlm will also need to be included unless already installed from the step above).

  Before rebooting the nodes, you must ensure that you have correctly installed and configured the Oracle UDLM software.
  8.0 Oracle’s UDLM Package

  For the UDLM software to run correctly, sufficient shared memory must be available on all cluster nodes. See the OPS/RAC CD for all installation instructions. To prepare the Sun Cluster nodes, you must ensure that:

  The Oracle user account and the dba group are set up correctly.
  The system is configured to support the shared memory requirements of the UDLM.

  The Oracle UDLM package is found on the Oracle install CD under the opspatch directory for Oracle v9.0.1 and the racpatch directory for Oracle v9.2.0 1 onwards. A compressed tar file plus readme files are located in this directory. Again, to install, use the pkgadd command:-

  # pkgadd -d . ORCLudlm

  The udlm configuration files in SC3.0 are as follows:-

  - SC3.0: /etc/opt/SUNWcluster/conf/udlm.conf

  UDLM version 3.3.4.4 has been shipped with Oracle v9.0.1 and supports Sun Cluster v3 (see < >). UDLM version 3.3.4.5 was shipped with Oracle v9.2.0.1.
  9.0 Related Documents

  < > – Step-By-Step Installation of RAC on Sun Cluster v3
  < > – Solaris UDLM For Sun Solaris 2.8/Sun Cluster Software 3.0
  < > – Veritas Volume Manager on Solaris & Real Application Clusters
  < > – Introduction to Sun Cluster™ v3
  Increase Scalability through Real Application Clusters (RAC) on Solaris
  RAC Technologies Compatibility Matrix (RTCM) for Solaris Clusters
  Designing Enterprise Solutions with Sun Cluster 3.0 (Ch 6 – Database Cluster), Richard Elling & Tim Read, Sun Microsystems Press, Prentice Hall, 2002
  Sun Cluster 3.0 Administration Training Course ES-333, February 2001, Revision A
  Sun Cluster 3.0 Release Notes, Part No. 806-1428, November 2000, Revision A
  Sun Cluster 3.0 Installation Guide, Part No. 806-1419-10, November 2000, Revision A
  Sun Cluster 3.0 System Administration Guide, Part No. 806-1423-10, November 2000, Revision A
  Sun Cluster 3.0 Hardware Guide, Part No. 806-1420-10, November 2000, Revision A
  Sun Cluster 3.0 Data Services Developers’ Guide, Part No. 806-1422-10, November 2000, Revision A
  Sun Cluster 3.0 Concepts, Part No. 806-1424-1, November 2000, Revision A
  Sun Cluster 3.0 Error Messages Manual, Part No. 806-1426-10, November 2000, Revision A
  Sun Cluster 3.0 Data Services Installation and Configuration Guide, Part No. 806-1421-10, November 2000, Revision A

Copyright © 2012 Oracle Clinic – Maclean Liu的个人技术博客 - All Rights Reserved
Powered by WordPress & Atahualpa