rman备份出现ORA-19502/ORA-27063错误

两个通道在写入rman备份到磁盘中的时候,在05/15/2012 04:39:58发现磁盘空间不足,两个通道分别准备写入30480897/30753665号块的时

RMAN> 2> 3> 4> 5> 6> 7> 8> 9> 10> 11> 12> 13> 14> 15> 16> 17> 18> 19> 20>

allocated channel: t11

channel t11: sid=824 instance=ncdb1 devtype=DISK

 

allocated channel: t12

channel t12: sid=838 instance=ncdb1 devtype=DISK

 

allocated channel: t13

channel t13: sid=809 instance=ncdb1 devtype=DISK

 

allocated channel: t14

channel t14: sid=886 instance=ncdb1 devtype=DISK

 

allocated channel: t15

channel t15: sid=620 instance=ncdb1 devtype=DISK

 

allocated channel: t16

channel t16: sid=599 instance=ncdb1 devtype=DISK

 

allocated channel: t17

channel t17: sid=482 instance=ncdb1 devtype=DISK

 

allocated channel: t18

channel t18: sid=506 instance=ncdb1 devtype=DISK

一共开通8个通道

 

channel t12: starting full datafile backupset

channel t12: specifying datafile(s) in backupset

input datafile fno=00008 name=/dev/rnc32g_39

input datafile fno=00016 name=/dev/rnc32g_47

input datafile fno=00024 name=/dev/rnc32g_57

input datafile fno=00032 name=/dev/rnc32g_25

input datafile fno=00040 name=/dev/rnc32g_33

input datafile fno=00048 name=/dev/rnc32g_3

input datafile fno=00056 name=/dev/rnc32g_11

input datafile fno=00064 name=/dev/rnc32g_19

input datafile fno=00072 name=/dev/rnc32g_67

input datafile fno=00080 name=/dev/rnc32g_106

input datafile fno=00088 name=/dev/rnc32g_114

input datafile fno=00096 name=/dev/rnc32g_87

input datafile fno=00104 name=/dev/rnc32g_95

input datafile fno=00112 name=/dev/rnc32g_103

input datafile fno=00120 name=/dev/rnc32g_75

input datafile fno=00003 name=/dev/rnc50_sysaux

input datafile fno=00130 name=/dev/rnc32g_119

channel t12: starting piece 1 at 14-MAY-12

--通道12备份数据文件

 

channel t17: starting full datafile backupset

channel t17: specifying datafile(s) in backupset

input datafile fno=00002 name=/dev/rnc32g_22

input datafile fno=00013 name=/dev/rnc32g_44

input datafile fno=00021 name=/dev/rnc32g_54

input datafile fno=00029 name=/dev/rnc32g_62

input datafile fno=00037 name=/dev/rnc32g_30

input datafile fno=00045 name=/dev/rnc32g_38

input datafile fno=00053 name=/dev/rnc32g_8

input datafile fno=00061 name=/dev/rnc32g_16

input datafile fno=00069 name=/dev/rnc32g_64

input datafile fno=00077 name=/dev/rncundo_33g_4

input datafile fno=00085 name=/dev/rnc32g_111

input datafile fno=00093 name=/dev/rnc32g_84

input datafile fno=00101 name=/dev/rnc32g_92

input datafile fno=00109 name=/dev/rnc32g_100

input datafile fno=00117 name=/dev/rnc32g_72

input datafile fno=00006 name=/dev/rnc50_4g_1

channel t17: starting piece 1 at 14-MAY-12

--通道17备份数据文件

 

channel t15: finished piece 1 at 15-MAY-12

piece handle=/rman/db_mpnb04jl_1_1 tag=TAG20120514T204954 comment=NONE

channel t15: backup set complete, elapsed time: 06:07:59

channel t11: finished piece 1 at 15-MAY-12

piece handle=/rman/db_mlnb04jk_1_1 tag=TAG20120514T204954 comment=NONE

channel t11: backup set complete, elapsed time: 06:17:25

channel t16: finished piece 1 at 15-MAY-12

piece handle=/rman/db_mqnb04jm_1_1 tag=TAG20120514T204954 comment=NONE

channel t16: backup set complete, elapsed time: 06:34:49

channel t14: finished piece 1 at 15-MAY-12

piece handle=/rman/db_monb04jl_1_1 tag=TAG20120514T204954 comment=NONE

channel t14: backup set complete, elapsed time: 06:40:05

channel t18: finished piece 1 at 15-MAY-12

piece handle=/rman/db_msnb04jn_1_1 tag=TAG20120514T204954 comment=NONE

channel t18: backup set complete, elapsed time: 06:43:38

channel t13: finished piece 1 at 15-MAY-12

piece handle=/rman/db_mnnb04jl_1_1 tag=TAG20120514T204954 comment=NONE

channel t13: backup set complete, elapsed time: 07:40:56

--这里可以看出rman的备份完成了通道11/13/14/15/16/18，也就是说目前为止通道12/17未完成.

 

RMAN-03009: failure of backup command on t12 channel at 05/15/2012 04:39:58

ORA-19502: write error on file "/rman/db_mmnb04jl_1_1", blockno 30481025 (blocksize=8192)

ORA-27063: number of bytes read/written is incorrect

IBM AIX RISC System/6000 Error: 28: No space left on device

Additional information: -1

Additional information: 1048576

ORA-19502: write error on file "/rman/db_mmnb04jl_1_1", blockno 30480897 (blocksize=8192)

ORA-27063: number of bytes read/written is incorrect

IBM AIX RISC System/6000 Error: 28: No space left on device

channel t12 disabled, job failed on it will be run on another channel

--通道12报错(硬盘空间不足)

 

channel t11: starting full datafile backupset

channel t11: specifying datafile(s) in backupset

input datafile fno=00008 name=/dev/rnc32g_39

input datafile fno=00016 name=/dev/rnc32g_47

input datafile fno=00024 name=/dev/rnc32g_57

input datafile fno=00032 name=/dev/rnc32g_25

input datafile fno=00040 name=/dev/rnc32g_33

input datafile fno=00048 name=/dev/rnc32g_3

input datafile fno=00056 name=/dev/rnc32g_11

input datafile fno=00064 name=/dev/rnc32g_19

input datafile fno=00072 name=/dev/rnc32g_67

input datafile fno=00080 name=/dev/rnc32g_106

input datafile fno=00088 name=/dev/rnc32g_114

input datafile fno=00096 name=/dev/rnc32g_87

input datafile fno=00104 name=/dev/rnc32g_95

input datafile fno=00112 name=/dev/rnc32g_103

input datafile fno=00120 name=/dev/rnc32g_75

input datafile fno=00003 name=/dev/rnc50_sysaux

input datafile fno=00130 name=/dev/rnc32g_119

channel t11: starting piece 1 at 15-MAY-12

--在通道12报错后,通道11已经完成了上次备份,所以启动备份通道12出错的数据文件

 

RMAN-03009: failure of backup command on t17 channel at 05/15/2012 04:39:58

ORA-19502: write error on file "/rman/db_mrnb04jm_1_1", blockno 30753793 (blocksize=8192)

ORA-27063: number of bytes read/written is incorrect

IBM AIX RISC System/6000 Error: 28: No space left on device

Additional information: -1

Additional information: 1048576

ORA-19502: write error on file "/rman/db_mrnb04jm_1_1", blockno 30753665 (blocksize=8192)

ORA-27063: number of bytes read/written is incorrect

IBM AIX RISC System/6000 Error: 28: No space left on device

channel t17 disabled, job failed on it will be run on another channel

--通道17也因为磁盘空间报错

 

channel t13: starting full datafile backupset

channel t13: specifying datafile(s) in backupset

input datafile fno=00002 name=/dev/rnc32g_22

input datafile fno=00013 name=/dev/rnc32g_44

input datafile fno=00021 name=/dev/rnc32g_54

input datafile fno=00029 name=/dev/rnc32g_62

input datafile fno=00037 name=/dev/rnc32g_30

input datafile fno=00045 name=/dev/rnc32g_38

input datafile fno=00053 name=/dev/rnc32g_8

input datafile fno=00061 name=/dev/rnc32g_16

input datafile fno=00069 name=/dev/rnc32g_64

input datafile fno=00077 name=/dev/rncundo_33g_4

input datafile fno=00085 name=/dev/rnc32g_111

input datafile fno=00093 name=/dev/rnc32g_84

input datafile fno=00101 name=/dev/rnc32g_92

input datafile fno=00109 name=/dev/rnc32g_100

input datafile fno=00117 name=/dev/rnc32g_72

input datafile fno=00006 name=/dev/rnc50_4g_1

channel t13: starting piece 1 at 15-MAY-12

--通道13也尝试备份通道17失败的数据文件

 

RMAN-03009: failure of backup command on t11 channel at 05/15/2012 04:39:59

ORA-19504: failed to create file "/rman/db_mtnb104u_1_1"

ORA-27044: unable to write the header block of file

IBM AIX RISC System/6000 Error: 28: No space left on device

Additional information: 3

Addition

--因为当前没有空闲空间,通道11终止,

--这个时候rman异常终止,导致后续的通道13终止记录未打印到日志

阅读完rman日志,很好理解因为存放rman备份的磁盘空间不足导致了一系列错误
 
检查磁盘剩余空间
 

 

 

Filesystem    GB blocks      Free %Used    Iused %Iused Mounted on

/dev/hd4          10.00      9.75    3%    6548    1% /

/dev/hd2          10.00      4.55  55%    84383    8% /usr

/dev/hd9var        5.00      4.04  20%    6290    1% /var

/dev/hd3          5.00      3.87  23%    1551    1% /tmp

/dev/hd1          10.00      9.91    1%      382    1% /home

/proc                -        -    -        -    -  /proc

/dev/hd10opt      5.00      4.89    3%    3502    1% /opt

/dev/archalv      99.00    82.98  17%      96    1% /archa

/dev/fslv01      40.00    19.49  52%    72324    2% /ora10

/dev/fslv00    1800.00    467.25  75%      10    1% /rman

这下让人迷糊了,磁盘空间还剩余467.25G,怎么会报错呢？
 
分析原因
 

 

 

RMAN-03009: failure of backup command on t12 channel at 05/15/2012 04:39:58

ORA-19502: write error on file "/rman/db_mmnb04jl_1_1", blockno 30481025 (blocksize=8192)

ORA-27063: number of bytes read/written is incorrect

IBM AIX RISC System/6000 Error: 28: No space left on device

Additional information: -1

Additional information: 1048576

ORA-19502: write error on file "/rman/db_mmnb04jl_1_1", blockno 30480897 (blocksize=8192)

ORA-27063: number of bytes read/written is incorrect

IBM AIX RISC System/6000 Error: 28: No space left on device

channel t12 disabled, job failed on it will be run on another channel

 

RMAN-03009: failure of backup command on t17 channel at 05/15/2012 04:39:58

ORA-19502: write error on file "/rman/db_mrnb04jm_1_1", blockno 30753793 (blocksize=8192)

ORA-27063: number of bytes read/written is incorrect

IBM AIX RISC System/6000 Error: 28: No space left on device

Additional information: -1

Additional information: 1048576

ORA-19502: write error on file "/rman/db_mrnb04jm_1_1", blockno 30753665 (blocksize=8192)

ORA-27063: number of bytes read/written is incorrect

IBM AIX RISC System/6000 Error: 28: No space left on device

channel t17 disabled, job failed on it will be run on another channel