Checklist for an RMAN Restore

Checklist for an RMAN Restore

GOAL

Prior to performing an RMAN restore, there are certain actions you can take to ensure its success. Rather than running a restore for hours only to find that it failed due to something avoidable in the first place, like a missing backuppiece, it would be better to get the environment fully prepared upfront.

A lot of time and confusion can be saved if we have a check list to work with first.

SOLUTION

1) Check the Operating System and RDBMS Version

Ideally you should be restoring to the same operating system version and RDBMS Version. If restoring to a different operating system you may need to covert between the Endians.

If restoring to a higher version of the RDBMS you need to perform an upgrade before the database can be open for use:

     Restoring a database to a higher patchset (Doc ID 558408.1)

The combinations of operating systems and RDBMS can be quite complex. See the following note for further details:

     Frequently Asked Questions about Restoring Or Duplicating Between Different Versions And Platforms (Doc ID 369644.1)

2) Confirm Space Requirements

As your database grow and shrink, the space usage may vary. Confirm the directories/diskgroup, and its space requirements using the REPORT SCHEMA command.

RMAN> run {
set until time "to_date('2013 JUN 11 15:31','YYYY MON DD HH24:MI')";
report schema;
}2> 3> 4>

executing command: SET until clause

using target database control file instead of recovery catalog
Report of database schema for database with db_unique_name ORA112

List of Permanent Datafiles
===========================
File Size(MB) Tablespace           RB segs Datafile Name
---- -------- -------------------- ------- ------------------------
1    830      SYSTEM               ***     /opt/app/oracle/oradata/ORA112/system01.dbf
2    980      SYSAUX               ***     /opt/app/oracle/oradata/ORA112/sysaux01.dbf
3    205      UNDOTBS1             ***     +DG1/ora112/datafile/undotbs1.256.807783447
4    100      USERS                ***     +DG1/ora112/datafile/users.264.817484131
5    345      EXAMPLE              ***     /opt/app/oracle/oradata/ORA112/example01.dbf
6    100      MYTS                 ***     +DG1/ora112/datafile/myts.267.816708795

From the above we can see the tablespace, the datafiles and space requirements. The datafile's directories and diskgroups must pre-exist and have sufficient space to hold the datafiles. Otherwise you need to use the SET NEWNAME to restore to an alternate location that has the permission and space availability.

Note: you may also need space in the archive log destination if the required archivelogs will be also be restored from their backuppieces

3) Backup Scripts and Logs

In preparation for the restore, have the backup scripts and logs handy. The output from an RMAN command will be displayed to the standard output by default. If running interactively it will be displayed on your screen. You can always spool the output using the command:

RMAN> SPOOL TRACE TO X.OUT
SET ECHO ON
# place the rest of your rman commands here
SPOOL TRACE OFF;

An RMAN backup log will help us confirm the backup's success/failure, what files were actually backed up, and timings such as the start and end of backup, and elapsed times.

If restoring to the same host, consider temporarily turning off your scheduled backup jobs first. Since the database is being restored we do not want backups jobs to run, nor archivelogs needed for recovery to be removed.

4) Be Mindful of the FRA

If using a restored controlfile, RMAN will perform an implicit crosscheck and cataloging of the FRA if the FRA is in use. This may add to your restore time, and can sometimes cause incarnation issues if the database has under gone a few resetlogs.

If you don't need the files in the FRA, you can either remove the files from the FRA or just disable it during the restore, and re-enable it afterwards.

RMAN restore hangs with "Starting implicit crosscheck" & "cataloging files" (Doc ID 1489027.1)

5) Obtain a Summary of the RMAN Backup

To give us a quick overview of the backups taken:

RMAN> spool trace to bk.out;
set echo on;
list incarnation of database;
list backup summary;
list backup of datafile 1;
list copy of datafile 1;
spool trace off;

6) Previewing the Restore and Recovery

You can apply RESTORE ... PREVIEW to any RESTORE operation to create a detailed list of every backup to be used in the requested RESTORE operation, and the necessary target SCN for recovery after the RESTORE operation is complete. This command accesses the RMAN repository to query the backup metadata, but does not actually read the backup files to ensure that they can be restored.

Blockrecovery can also be previewed:

RMAN> blockrecover corruption list preview;

The entire restore can be previewed in one command:

RMAN> run {
set until time "to_date('2013 JUN 11 15:31','YYYY MON DD HH24:MI')";
restore controlfile database preview;
}

A restore preview will list all backups required for the entire restore and recovery operation. From this list you can then go ahead and confirm that the backuppieces are physically avaliable on disk or tape before performing the actual restore.

Useful items from the restore preview:

RMAN> run {
2> set until time "to_date('2014 MAY 02 11:22','YYYY MON DD HH24:MI')";
3> restore database preview;
4> }
...
BS Key  Type LV         Size            Device Type    Elapsed Time   Completion Time   
------- ---- --         ----------      -----------    ------------   --------------------
266     Incr 0 [1]      1.49G[2]        DISK[3]        00:01:31[4]       02 MAY 2014 11:21:57 [5]
        BP Key: 268   Status: AVAILABLE[6]  Compressed: NO[7] Tag: TAG20140502T112027
        Piece Name: /opt/app/oracle/fra/ORA112/backupset/2014_05_02/o1_mf_nnnd0_TAG20140502T112027_9p5wpwp4_.bkp [8]

  List of Datafiles in backup set 266 [9]
  File LV Type Ckp SCN    Ckp Time             Name
  ---- -- ---- ---------- -------------------- ----
  1    0  Incr 53204035   02 MAY 2014 11:20:27 /opt/app/oracle/oradata/ora112b/data_D-ORA112_TS-SYSTEM_FNO-1
  4    0  Incr 53204035   02 MAY 2014 11:20:27 /opt/app/oracle/oradata/ora112b/data_D-ORA112_TS-USERS_FNO-4
  7    0  Incr 53204035   02 MAY 2014 11:20:27 /opt/app/oracle/oradata/ora112b/data_D-ORA112_TS-RMAN_TS_FNO-7
  9    0  Incr 53204035   02 MAY 2014 11:20:27 /opt/app/oracle/fra/ORA112/datafile/o1_mf_fda_ts_9nw1gqt5_.dbf
...
  List of Archived Logs in backup set 268
  Thrd Seq     Low SCN    Low Time             Next SCN   Next Time
  ---- ------- ---------- -------------------- ---------- ---------
  1    317     53204027   02 MAY 2014 11:20:25 53204084   02 MAY 2014 11:22:03
Media recovery start SCN is 53204027
Recovery must be done beyond SCN 53204036 to clear datafile fuzziness[10]
Finished restore at 02 MAY 2014 11:24:07

[1] This is an incremental level 0 backup
[2] The backuppiece is 1.49G in size
[3] This is a backup to DISK rather than tape
[4] The backuppiece took 1 minute and 31 seconds to backup
[5] The time when this backuppiece completed
[6] The backuppiece is available for use according to the RMAN repository
[7] This is NOT a compressed backup
[8] This is the backuppiece. It must exist on disk for use:

$ ls -l /opt/app/oracle/fra/ORA112/backupset/2014_05_02/o1_mf_nnnd0_TAG20140502T112027_9p5wpwp4_.bkp
-rw-r-----. 1 oracle oinstall 1601011712 May  2 11:21 /opt/app/oracle/fra/ORA112/backupset/2014_05_02/o1_mf_nnnd0_TAG20140502T112027_9p5wpwp4_.bkp

[9] the datafiles contained in this backuppiece
[10] this is just informational message. It simply telling you that the SET UNTIL must reflect a time/SCN that is beyond this value before the database can be opened

a) Preview the Controlfile Restore

If perform a database restore to a point in time in the past, you should also restore the controlfile as it wil reflect the database structure at the time. This is especially important if there are tablespaces or datafiles being added or dropped over the database's lifetime.

 An example of a controlfile restore preview:

Recovery Manager: Release 11.2.0.3.0 - Production on Tue Jun 18 22:27:47 2013

Copyright (c) 1982, 2011, Oracle and/or its affiliates.  All rights reserved.

connected to target database: HRPRD91 (DBID=756818178, not open)

RMAN> run {
2> allocate channel 'dev_0' type 'sbt_tape'
3> parms 'SBT_LIBRARY=/opt/omni/lib/libob2oracle8_64bit.so,ENV=(OB2BARTYPE=Oracle8,OB2APPNAME=HRPRD91,OB2BARLIST=VTL_hp909npc_HRPRD91_logs1)';
set until time "to_date('Jun 18 2013 08:00:00','Mon DD YYYY HH24:MI:SS')";
4> restore controlfile preview;
5> 6> }

using target database control file instead of recovery catalog
allocated channel: dev_0
channel dev_0: SID=6 device type=SBT_TAPE
channel dev_0: Data Protector A.06.20/406

executing command: SET until clause

Starting restore at 06/18/2013 22:28:04


List of Backup Sets
===================


BS Key  Type LV Size       Device Type Elapsed Time Completion Time
------- ---- -- ---------- ----------- ------------ -------------------
52857   Full    19.00M     SBT_TAPE    00:00:09     06/18/2013 04:02:04
       BP Key: 52857   Status: AVAILABLE  Compressed: NO  Tag: TAG20130618T040153
       Handle: VTL_hp909npc_HRPRD91_logs1.dbf [1]  Media: 0a226c50:513b5915:6b9f:12cc [2]
 Control File Included: Ckp SCN: 238614135722   Ckp time: 06/18/2013 04:01:55
 
List of Backup Sets
===================


BS Key  Size       Device Type Elapsed Time Completion Time
------- ---------- ----------- ------------ -------------------
52858   430.50M    SBT_TAPE    00:00:36     06/18/2013 09:01:10
       BP Key: 52858   Status: AVAILABLE  Compressed: NO  Tag: TAG20130618T090033
       Handle: VTL_hp909npc_HRPRD91_logs1.dbf [1]   Media: 0a226c50:5091d0b9:120c:4f13 [2]

 List of Archived Logs in backup set 52858
 Thrd Seq     Low SCN    Low Time            Next SCN   Next Time
 ---- ------- ---------- ------------------- ---------- ---------
 1    14822   238614135104 06/18/2013 04:00:28 238614215594 06/18/2013 08:40:24 [3]
validation succeeded for backup piece
Media recovery start SCN is 238614135722
Recovery must be done beyond SCN 238614135722 to clear datafile fuzziness
validation succeeded for backup piece
Finished restore at 06/18/2013 22:28:13
released channel: dev_0

[1] "Handle" is the backuppiece name
[2] Media is the lable given to the tape
[3] This is the achivelog needed to recover this controlfile

That is, you need to ensure that the backuppiece [1] on tape [2] is loaded and ready for use

b) Preview the Database Restore

An example of a point in time restore preview command:

RMAN> spool trace to res.prev
run {
set until time "to_date('2013 JUN 11 15:31','YYYY MON DD HH24:MI')";
restore database preview;
}
spool trace off

From the preview output, look for errors such as "datafile x not processed"; "data file x will be created", "no backup of archived log"; "file x is excluded from whole database backup"; "could not locate pieces of backup set key x". etc.

RMAN-06026: some targets not found - aborting restore
RMAN-06023: no backup or copy of datafile 4 found to restore

RMAN-06100: no channel to restore a backup or copy of datafile 18
no channel to restore a backup or copy of log thread 1 seq 186 lowscn 609321 

Readonly datafiles

In 10g READ ONLY datafiles will only be restored if using CHECK READONLY. Whereas in 11g the read only datafiles are restored by default, except if you use SKIP READONLY.

datafile 29 not processed because file is read-only

If restoring to a new host, do you want the read only datafile to be restored or not?

Datafile will be created rather than restored

During the preview, if you get the error message below:

data file 25 will be created automatically during restore operation

then you should confirm why there seems to be no backup for this datafile. If you have all archivelogs from the time of this datafile creation to complete the recovery then it should be ok. Otherwise you should find out RMAN is not using the backup. Is there even a backup to use?

RMAN> list backup of datafile 25;
RMAN> list copy of datafile 25;

If you get the message that file 1 will be created then there is a problem. File 1, which is the system datafile, cannot be recreated automatically from the restore process. It must be restored from a backup.

Missing archivelog

Recovery will need this archivelog so you need to find the backuppiece containing the archivelog, or the archivelog itself

no backup of log thread 1 seq 55004 lowscn 11304778445 found to restore

 RMAN-06100 No Channel to Restore a Backup or Copy

Often this is a sign that the appropriate channels have not been allocated to perform the restore. For example, if the backups are on tape, ensure that an SBT channel is allocated.

RMAN-06100 during restore/duplicate (Doc ID 1615996.1)

 

c) Recovery Preview

Recovery Preview is only available from 11g onwards.

Once the database is restored, mounted, then the recovery can also be previewed:

RMAN> run {
set until time "to_date('2013 JUN 11 15:31','YYYY MON DD HH24:MI')";
recover database preview;
}

7) How Long will the Restore Take?

If restoring to the same host, and all loads being equal, then a restore operation should take a similar time as the backup. Look at Elapsed Time from the restore preview:

BS Key  Type   LV   Size        Device Type  Elapsed Time  Completion Time   
------- ----   --  ----------   -----------  ------------  --------------------
258     Full       1.49G        DISK         00:01:33      02 MAY 2014 10:14:55

8) Validating the Restore

As an alternative to RESTORE ... PREVIEW, you can use the RESTORE ... VALIDATE HEADER command. In addition to listing the files needed for restore and recovery, the RESTORE ... VALIDATE HEADER command validates the backup file headers to determine whether the files on disk or in the media management catalog correspond to the metadata in the RMAN repository.

When planning your restore and recovery operation, use RESTORE ... PREVIEW or RESTORE ... VALIDATE HEADER to ensure that all required backups are available or to identify situations in which you may want to direct RMAN to use or avoid specific backups.

 RMAN> restore database validate;

Starting restore at 26 JUN 2013 14:41:23
using channel ORA_DISK_1
using channel ORA_DISK_2

skipping datafile 6; already restored to file +DG1/ora112/datafile/myts.267.816708795
datafile 7 will be created automatically during restore operation
channel ORA_DISK_1: starting validation of datafile backup set
channel ORA_DISK_2: starting validation of datafile backup set
channel ORA_DISK_1: reading from backup piece /opt/app/oracle/fra/ORA112/backupset/2013_06_11/o1_mf_nnndf_TAG20130611T152919_8vfffl84_.bkp
channel ORA_DISK_2: reading from backup piece /opt/app/oracle/fra/ORA112/backupset/2013_06_11/o1_mf_nnndf_TAG20130611T152919_8vffflhp_.bkp
channel ORA_DISK_2: ORA-19870: error while restoring backup piece /opt/app/oracle/fra/ORA112/backupset/2013_06_11/o1_mf_nnndf_TAG20130611T152919_8vffflhp_.bkp
ORA-19505: failed to identify file "/opt/app/oracle/fra/ORA112/backupset/2013_06_11/o1_mf_nnndf_TAG20130611T152919_8vffflhp_.bkp"
ORA-27037: unable to obtain file status
Linux-x86_64 Error: 2: No such file or directory
Additional information: 3

channel ORA_DISK_1: piece handle=/opt/app/oracle/fra/ORA112/backupset/2013_06_11/o1_mf_nnndf_TAG20130611T152919_8vfffl84_.bkp tag=TAG20130611T152919
channel ORA_DISK_1: restored backup piece 1
channel ORA_DISK_1: validation complete, elapsed time: 00:00:03
failover to previous backup

datafile 7 will be created automatically during restore operation
channel ORA_DISK_1: scanning datafile copy /opt/app/oracle/fra/ORA112/datafile/o1_mf_system_8v2s25b8_.dbf
channel ORA_DISK_2: scanning datafile copy /opt/app/oracle/fra/ORA112/datafile/o1_mf_example_8v2s48b4_.dbf
failover to previous backup

datafile 7 will be created automatically during restore operation
Finished restore at 26 JUN 2013 14:41:53

RMAN> list backuppiece '/opt/app/oracle/fra/ORA112/backupset/2013_06_11/o1_mf_nnndf_TAG20130611T152919_8vffflhp_.bkp';


List of Backup Pieces
BP Key  BS Key  Pc# Cp# Status      Device Type Piece Name
------- ------- --- --- ----------- ----------- ----------
43      43      1   1   AVAILABLE   DISK        /opt/app/oracle/fra/ORA112/backupset/2013_06_11/o1_mf_nnndf_TAG20130611T152919_8vffflhp_.bkp

From the above we can see that the backuppiece /opt/app/oracle/fra/ORA112/backupset/2013_06_11/o1_mf_nnndf_TAG20130611T152919_8vffflhp_.bkp is missing from disk even though it is listed as AVAILABLE for use. So you need to locate this backuppiece before performing the actual restore.

Additional Info:
(Doc ID 1481151.1) RMAN "restore... Validate [header]" Lists Full Or L0 Backups Only

9) Gotchas

NOLOGGING

Be mindful of NOLOGGING operations - was the original database in FORCE LOGGING mode? Otherwise, do you have nologging operations that may result in NOLOGGING corruption once the database is recovered?

Check For Logging / Nologging On DB Object(s) (Doc ID 269274.1)

ORA-1578 / ORA-26040 Corrupt blocks by NOLOGGING - Error explanation and solution (Doc ID 794505.1)

====================================================================