Oracle Clusterware (formerly CRS) Rolling Upgrades

作者: Maclean Liu , post on October 6th, 2009 , English Version
【本站文章除注明转载外,均为本站原创编译】
转载请注明:文章转载自: Oracle Clinic – Maclean Liu的个人技术博客 [http://www.oracledatabase12g.com/]
本文标题: Oracle Clusterware (formerly CRS) Rolling Upgrades
本文永久地址: http://www.oracledatabase12g.com/archives/oracle-clusterware-formerly-crs-rolling-upgrades.html

© 2009, www.oracledatabase12g.com. 版权所有.文章允许转载,但必须以链接方式注明源地址,否则追究法律责任.

相关文章 | Related posts:

  1. Status of Certification of Oracle Clusterware with HACMP
  2. Complete Upgrade Checklist for Manual Upgrades from 8.X / 9.0.1 to Oracle9iR2 (9.2.0)
  3. Rolling Cursor Invalidations with DBMS_STATS in Oracle10g
  4. Know about RAC Clusterware Process OPROCD
  5. Diagnosability for CRS / EVM / RACG
  6. Diagnosing Unsuccessful CRS root.sh Issues
  7. Oracle rac进阶管理专家指导系列文档
  8. 书籍推荐:《Secrets of the Oracle Database》

3 comments to Oracle Clusterware (formerly CRS) Rolling Upgrades

  • admin

    Rolling Patch – OPatch Support for RAC

    PURPOSE
    ——-

    Describe the current support of OPatch for Real Application Clusters.
    Before reading OPatch RAC Support, you should be familiar with
    single-instance OPatch processing.

    For more information about Opatch, please refer to:
    Note.189489.1 Oracle9i Data Server Interim Patch Installation (OPatch)

    SCOPE & APPLICATION
    ——————-

    This document is intended for DBAs and System Administrators that
    are going to apply Oracle Interim Patches on RAC environment.

    Rolling Patch – OPatch Support for RAC
    ————————————–

    1 – RAC Patching methods

    OPatch supports 3 different patch methods on a RAC environment:

    . Patching RAC as a single instance (All-Node Patch)

    In this mode, OPatch applies the patch to the local node first, then
    propagates the patch to all other nodes, and finally updates the inventory.
    All instances will be down during the whole patching process.

    · Patching RAC using a minimum down-time strategy (Min. Downtime Patch)

    In this mode, OPatch patches the local node, asks users for a sub-set of
    nodes, which will be the first nodes to be patched. After the initial
    subset of nodes are patched, Opatch propagates the patch to the other nodes
    and finally updates the inventory. The downtime would happen between the
    shutdown of the the second subset of nodes and the startp of the initial
    subset of nodes patched.

    · Patching RAC using a rolling strategy – No down time (Rolling Patch)

    With this method, there is no downtime. Each node would be patched and
    brought up while all the other nodes are up and running, resulting in
    no disruption of the system.

    2 – Flow diagrams

    . All-Node Patch

    . Shutdown all Oracle instances on all nodes
    . Apply the patch to all nodes
    . Bring all nodes up

    . Minimum downtime

    . Shutdown the Oracle instance on node 1
    . Apply the patch to the Oracle instance on node 1
    . Shutdown the Oracle instance on node 2
    . Apply the patch to the Oracle instance on node 2
    . Shutdown the Oracle instance on node 3
    . At this point, instances on nodes 1 and 2 can be brought up
    . Apply the patch to the Oracle instance on node 3
    . Startup the Oracle instance on node 3

    . Rolling patch (no downtime)

    . Shutdown the Oracle instance on node 1
    . Apply the patch to the Oracle instance on node 1
    . Start the Oracle instance on node 1
    . Shutdown the Oracle instance on node 2
    . Apply the patch to the Oracle instance on node 2
    . Start the Oracle instance on node 2
    . Shutdown the Oracle instance on node 3
    . Apply the patch to the Oracle instance on node 3
    . Start the Oracle instance on node 3

    3 – How does Opatch select which method to use?

    To be eligible to be applied on as a rolling patch, the patch need to meet
    certain criteria, which are determined by Oracle developers. To be applied
    on a “rolling fashion”, the patch must be designated as a “rolling
    updatable patch” or simply “rolling patch”.

    The algorithm used to decide which method is going to be used is the
    following:

    If (users specify minimize_downtime)
    patching mechanism = Min. Downtime
    else if (patch is a rolling patch)
    patching mechanism = Rolling
    else
    patching mechanism = All-Node

    4 – Availability of rolling patches

    As patches are released they will qualified and identified as a “rolling” or
    “not rolling patch”. While most patches can be applied in a rolling
    fashion some patches can not be applied in this fashion.
    Patches that could potentially be installed on rolling fashion include:

    . Patches that do not affect the contents of the database.
    . Patches not related to the RAC internode communication infrastructure.
    . Patches that change procedural logic and do not modify common header
    definitions of kernel modules. This includes client side patches that
    only affect utilities like export, import, sql*plus, sql*loader, etc.

    Only individual patches — not patch sets — will be “rollable”. It should
    also be noted that a merge patch of a “rolling patch” and an ordinary patch
    will not be a “rolling patch”.

    From 9.2.0.4 on, all patches released will be marked as a “rolling” or
    “not rolling patch”, based on defined set of rules. Patches previously
    released are packaged as “not rolling”.

    Because the set of rules currently defined are very conservative, patches
    released as “not rolling patches”, either before and after 9.2.0.4, may be
    eligible to be re-released as “rolling patches”, after analysis from
    Oracle Development.

    If you plan to apply a patch that is marked as “not rolling” and want to
    check if is possible to take advantage of the rolling patch strategy,
    please contact Oracle Support.

    5 – How to determine if a patch is a “rolling patch” or not?

    – 9i or 10gR1: (run)
    $ opatch query -is_rolling

    Opatch will ask the patch location and then will inform if the patch is or
    not a “rolling patch”

    – 10gR2: (run)
    $ opatch query -all | grep rolling

    For Windows, the following command can be used as grep is not suitable:
    opatch query -all C:\stage\10.2.0.3_Mini_Patches\5731537 | findstr rolling
    Patch is a rolling patch: false

    6 – Current Limitations

    . Patching with Shared File System

    Currently OPatch treats Shared File System, like CFS, as a single-instance
    patch. It means that OPatch will blindly patch files under a given
    ORACLE_HOME knowing that other nodes will pick up the changes via the
    Shared File System. Unfortunately, this means that OPatch cannot take
    advantage of a rolling patch on a Shared File System environment;
    all nodes must be down throughout the patching process.

    . Patching one node at time

    The Opatch strategies discussed above (All-Node, Min. Down-Time, and Rolling)
    presumes that all nodes will be patched at the same time. Additionally,
    each node can be patched individually, at different times, using the “-local”
    key word, which will patch only the local node.

  • admin

    Applying one-off Oracle Clusterware patches in a mixed version home environment

    Applies to:

    Oracle Server Enterprise Edition – Version: 10.1.0.2 to 11.1.0.7
    Generic Windows
    Generic UNIX
    Generic Linux

    Purpose

    The Clusterware patches for 10g and 11g contain binaries that can be installed in the Clusterware home (CRS_HOME), and the database home (RDBMS_HOME). Due to normal upgrading of the Oracle software in a RAC environment, customers may want to maintain multiple Oracle homes of different versions. This document will explain how to apply Clusterware one-off patches into these different Oracle Homes. For example, if Oracle Clusterware version is 10.2.0.2 and one RDBMS version on that cluster is 10.2.0.1, while another is 10.1.0.5, how should you apply a patch that you have downloaded from Metalink? See solution below.

    Scope and Application

    BACKGROUND:

    • An Oracle patch is applied using OPatch utility, always get the latest version of the OPatch utility from Metalink (10.2 placeholder Bug 4898608; 10.1 placeholder Bug 2617419)
    • A patch is named p<bugnum>_<release>_<port>.zip, self explanatory
    • Each Clusterware one-off patch consists of TWO patch portions; the Clusterware portion on the top level of the zip file, and the RDBMS portion of the patch under a subdirectory called:       custom/server/<bugnum>, both are built for the same exact Oracle version, for example 10.2.0.2. The RDBMs portion will contain RDBMs management components and Clusterware client-side tools and libraries. Both portions will only install on homes running that version only.
    • The Oracle Clusterware version must always be the latest in the cluster compared to the RDBMS/ASM home version. So in the example above, if the most current RDBMS_HOME version is at 10.2.0.1, then the Clusterware version must be at either 10.2.0.1, or newer.
    • Non-shared Oracle Clusterware home software can always be upgraded in a rolling upgrade fashion, see Note 338706.1
    • For this discussion in this document only, an ASM home is exactly the same as an RDBMS home and whenever the term RDBMS home is used it also means ASM home. Follow the exact same patching procedures for an ASM home as for an RDBMS home.

    Applying one-off Oracle Clusterware patches in a mixed version home environment

    EXAMPLE:

    We’ll use an example with specific version numbers to make it easier to understand, but this discussion holds true to all supported Clusterware versions.

    Assume the following Oracle homes on a 4-node cluster:

    • Clusterware version of 10.2.0.2
    • RDBMS home A, running version 10.2.0.2 (RAC)
    • RDBMS home B, running version 10.1.0.5 (RAC)
    • ASM home of 10.2.0.2 (most instances using the ASM storage)

    If a one-off patch needs to be applied to the 10.2.0.2 Clusterware software, in most likelihood it will also contain an RDBMS portion bundled in that patch zip file (as described above, in a subdirectory called custom/server/<bugnum>). The Clusterware portion will install on the 10.2.0.2 Clusterware home and the
    RDBMS portion of the one-off patch will install on the 10.2.0.2 RDBMS and ASM homes. [In most cases, the Clusterware application can be done in a rolling upgrade fashion. In some cases the RDBMS portion can also be applied in a rolling upgrade fashion, see the one-off patch README for details]

    However, if you attempt to install the RDBMS portion of that one-off patch to the 10.1.0.5 RDBMS home, OPatch will correctly block that attempt with messages:

    This Oracle Home does not have components/versions required by the patch. ERROR: OPatch failed during pre-reqs check.

    Explaining that no product can be patched. This is the correct OPatch behavior, as it’s absolutely not possible and strictly not allowed to install or somehow drop 10.2.0.2 libraries or binaries into Oracle home running a version that is different from the patched files, in this case 10.1.0.5.

    So, how should you proceed? How do we patch the 10.1.0.5 RDBMS home? Or, patch older RDBMS versions than the Clusterware?

    SOLUTION:

    First, it is always safe to ignore applying the RDBMS portion of the fix, if that functionality is not required. For example, if the Clusterware fix is not affecting the 10.1.0.5 Oracle RDBMS home, simply skip applying the RDBMS portion to the 10.1.0.5. The RDBMS portion will contain primarily fixes to management components specific to the database, and Clusterware client-side components. While it is not always easy to determine if a bug fix requires installation to the RDBMS Home, whenever possible the README for the patch will include this information.

    So for our example above one could take a Clusterware patch for version 10.2.0.2 and apply it to the CRS Home, RDBMS Home A and the ASM home. Due to any of the above reasons the application of this one-off patch to the RDBMS Home B could be skipped.

    If a fix for the RDBMS Home is required, a backport should be requested to the 10.1.0.5 version. That patch will again arrive with TWO portions (Clusterware in top/root directory, and RDBMS portion in the ‘server’ directory), in our example, only the ‘server’ portion will be applied to the 10.1.0.5 RDBMS home. The top-level patch applies to a 10.1.0.5 Clusterware, which is not installed at all in our environment (we are using a newer 10.2.0.2 Clusterware). In this case one would have to skip the Clusterware apply
    steps in the README and only execute the RDBMS steps, in most cases it’s the single operation similar to:

    Unix/Linux:
    $ opatch apply custom/server/<bugnum> -local -oh <RDBMS_HOME>

    Windows:
    C:\> opatch apply custom\server\<bugnum> -local -oh <RDBMS_HOME>

    For the example above, download the one-off patch for 10.1.0.5, and apply only the RDBMS portion to RDBMS Home B.

    Always remember, a Clusterware patch contains TWO portions (Clusterware & RDBMS), never attempt to override OPatch and somehow install incorrect binaries/shared libraries or object files into not exactly same version, for example, a fix built for 10.2.0.2 should not be installed into a 10.2.0.1 Oracle home!
    If you follow the steps above you will achive this goal.

    If the one-off patch contains fixes to more than one bug, the README will detail which files affect the Clusterware home, and which affect the RDBMS home, to help decide if the RDBMS portion can safely be skipped. The majority of the fixes will be to the Clusterware home, since it’s a Clusterware one-off patch, and so, in most cases it will be safe to ignore applying the RDBMS portion, either because that bug doesn’t affect the older RDBMS version, or the customer chooses to ‘live’ with the software defect as it has minimal or no bearing on their business.

    This discussion naturally also holds true for Oracle Clusterware (CRS) Bundle patches (as they contain an RDBMS portion). For example, it’s not possible to install the RDBMS portion of a 10.2.0.2 CRS bundle into a 10.1.0.5 rdbms home. Same steps should be followed to determine the relevancy of the RDBMS fixes to the rdbms homes and one-off fixes to the older rdbms home should be requested if
    needed, many cases they will not be needed or relevant.

  • admin

    Trying apply patch 11.1.0.7 to RDBMS, it says Clusterware is not upgraded but it is

    Applies to:

    Oracle Server – Standard Edition – Version: 11.1.0.7
    Information in this document applies to any platform.
    After patching crs to 11.1.0.7 with patch 6890831, Tried to patch db/asm home – it says version of CRS is not 11.1.0.7 (11.1.0.7 needed)

    Symptoms

    On 11.1.0.7 in Production, trying to apply patch 11.1.0.7 RDBMS ORACLE_HOME, it says clusterware is not upgraded but it is

    Check of the CRS version:

    crsctl query crs activeversion
    Oracle Clusterware active version on the cluster is [11.1.0.6.0] 

    crsctl query crs softwareversion
    Oracle Clusterware version on node [vegas01] is [11.1.0.6.0]

    Cause

    The mentioned error and the output of  CRS version check means that root111.sh has not been run on each node to activate the patch 11.1.0.7.

    One can also check $ORA_CRS_HOME/install directory. Before root111.sh run, there is a directory called patch111. Once root111.sh is completed, patch111 directory will be deleted and prepatch11107 directory will be created.

    Solution

    Run root111.sh on each node (see note:338706.1), to activate the patch:

    1. Log in as the root user.
    2. As the root user, perform the following tasks:

       
    a. Shutdown the CRS daemons by issuing the following command:
           
    <$ORA_CRS_HOME>/bin/crsctl stop crs
       
    b. Run the shell script located at:
           
    <$ORA_CRS_HOME>/install/root111.sh
       
    This script will automatically start the CRS daemons on the patched node upon completion.

    3. After completing this procedure, proceed to the next node and repeat

     

Leave a Reply

  

  

  

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>