Tuesday, July 12, 2016

How to run xfs_repair to resolve Input/output error

Problem:
/data2 is reporting I/O error.


root@shivinlocal data2]# ls
 ls: reading directory .: Input/output error

Checked the df -h and identify the /data2 partition is on /dev/sda1

Login to the server as root
umount /data2

If the unmount is showing error like this 
[root@searchlb01 /]# umount /data2
umount: /data2: device is busy.
        (In some cases useful info about processes that use
         the device is found by lsof(8) or fuser(1)) 


[root@searchlb01 /]# lsof /dev/sda1
lsof: WARNING: can't stat() xfs file system /data2
      Output information may be incomplete.
[root@searchlb01 /]# umount -l /data2


[root@shivinlocal ~]# xfs_repair /dev/sda1
xfs_repair: /dev/sda1 contains a mounted filesystem
fatal error -- couldn't initialize XFS library

How can we solve this error?

Comment out the fstab entry for /data2
Reboot the server

[root@shivinlocal ~]# xfs_repair /dev/sda1
Phase 1 - find and verify superblock...
Phase 2 - using internal log
 - zero log...
ERROR: The filesystem has valuable metadata changes in a log which needs to
be replayed. Mount the filesystem to replay the log, and unmount it before
re-running xfs_repair. If you are unable to mount the filesystem, then use
the -L option to destroy the log and attempt a repair.
Note that destroying the log may cause corruption -- please attempt a mount
of the filesystem before doing this.

For fixing the above error

[root@sdw6 ~]# xfs_repair -L /dev/sda1
Phase 1 - find and verify superblock...
Phase 2 - using internal log
 - zero log...
ALERT: The filesystem has valuable metadata changes in a log which is being
destroyed because the -L option was used.
 - scan filesystem freespace and inode maps...
 - found root inode chunk
Phase 3 - for each AG...
 - scan and clear agi unlinked lists...
 - process known inodes and perform inode discovery...
 - agno = 0
4461d940: Badness in key lookup (length)
bp=(bno 179821952, len 16384 bytes) key=(bno 179821952, len 8192 bytes)
 - agno = 1
 - agno = 2
46e21940: Badness in key lookup (length)
bp=(bno 539478608, len 16384 bytes) key=(bno 539478608, len 8192 bytes)
 - agno = 3
 - agno = 4
 - agno = 5
 - agno = 6
4be29940: Badness in key lookup (length)
...
disconnected inode 10737418540, moving to lost+found
Phase 7 - verify and correct link counts...
done

Uncomment the fastab entry
[root@shivinlocal ~]# mount -a /data2
 [root@shivinlocal data2]# ls
folder1 lost+found search99 secondfolder
 
Make sure all the services are UP and running. 

No comments:

Post a Comment