File system error recovery guide

Prev Next

Available in Classic

This document describes how to recover file system errors for each OS.

How to recover Linux server file system errors

If a file system error is suspected on the Linux server, restore the file system through the server connection console.

Use the following methods to check for I/O errors.

  • Directly access the server
  • If the server is inaccessible
    • In the VPC environment of the NAVER Cloud Platform console, navigate to i_menu > Services > Compute > Server > Server and click on [Managing server and changing server settings] > Server console screenshot.
      server-ts-repair-filesystem-vpc-01_ko

If a file system error occurs on VM, the following symptoms may arise:

  • I/O error
    server-ts-repair-filesystem-02
  • initramfs prompt recovery mode
    server-ts-repair-filesystem-03

Commands to recover Linux server file system

To recover file system errors on a Linux server, run the following commands:

  • fsck: ext4 file system inspection and recovery tool
  • xfs_repair: XFS file system inspection and recovery tool
Note

  • When you run the fsck command, the supported option information is as follows:

    Option Description
    -y Automatically recover the file system by processing the answer to all questions as yes
    -n Proceed with only checking the file system by processing the answer to all questions as no
    -a Automatically run the recovery without asking for confirmation to run the command
    • Not recommended
    -f Forcibly check the file system regardless of whether the file system is abnormal

  • When you run the xfs_repair command, the supported option information is as follows:

    Option Description
    -n Only check the file system without conducting the actual recovery process
    -f Forcibly check for unmounted file system or files
    -L Forcibly purge the log to run the file system recovery as a stronger option than f
    • May result in data loss

1. Preparation to recover file system errors

The following shows how to prepare for file system error recovery.

  1. Check the partition and file system type and any errors that occurred.

    • Suspected errors types according to the situations are as follows:

      Situation Error type
      Booting failed File system errors of default storage
      Booting succeeded but a file system-related error occurred File system errors of additional storage

    • File system-related errors are logged in /var/log/messages, and you can check the disc with the problems in the I/O error details in the console as well.

    • Partition check: run the cat /proc/partitions command.

      [root@servera ~]# cat /proc/partitions 
major minor  #blocks  name
202        0   52428800 xvda
202        1       1024 xvda1
202        2   52425728 xvda2

    • File system type check: run the blkid command.

      [root@servera ~]# blkid
/dev/xvda1: UUID="f95bed0a-11af-4b2c-bfcc-4afb91a68fc1" TYPE="xfs" 
/dev/xvda2: UUID="0692fdb8-bb3c-4094-83f0-fe95a339b8c1" TYPE="xfs"

  2. See Create snapshot to create a snapshot of the server to be recovered.

    Caution

    The server may be damaged by an incorrect recovery attempt. To prevent the server from being damaged, create a snapshot before performing the recovery.

2. Recover file system errors

Check how to recover file system errors in the following cases:

When a file system error occurs in the default storage

When an error occurs in default storage, the root file system check and mount will fail and will fall into initramfs prompt recovery mode.
The following shows how to recover the file system errors occurring in the default storage.

  1. Conduct file system check on the default storage in initramfs.
    <example> When the device with the problem is /dev/xvda1

    File system Commands
    ext4 fsck.ext4 /dev/xvda1
    xfs
    • xfs_repair /dev/xvda1
      • If recovery is not possible because of unused options, add the -L option to perform
      • Using the -L option may cause data loss
    • xfs_repair -L /dev/xvda1

  2. Conduct file system check once more and reboot the server when clean is displayed.

When a file system error occurs in the additional storage

The following shows how to recover the file system errors occurring in the additional storage.

  1. Run the following command to unmount the failed device.

    umount /dir (/dir: additional storage mount directory)
    Note

    If the unmount doesn't proceed properly, you must run the force unmount command or fuser command in sequence to check and terminate processes that are used in a specific file, file system, or port.

    • Force unmount command
      • umount -l /mnt/a
      • umount -f /mnt/a
    • Order to use fuser command
      1. Use fuser -cu /mnt/a to check the user or the process using the directory
      2. Use fuser -ck /mnt/a to force-end the process
      3. Unmount with umount /mnt/a

  2. Conduct file system check on the device with the error.
    <example> When the device with the problem is /dev/xvdb1

    File system Commands
    ext4 fsck.ext4 /dev/xvdb1
    xfs
    • xfs_repair /dev/xvdb1
      • If recovery is not possible because of unused options, add the -L option to perform
      • Using the -L option may cause data loss
    • xfs_repair -L /dev/xvdb1

  3. Remount additional storage with the following:

    • If /etc/fstab has mount settings for additional storage area
      mount -a
    • If running a direct mount
      mount dir (/dir: additional storage mount directory)

  4. Conduct file system check once more and reboot the server when clean is displayed.

When the file system error is not recovered

If the file system error does not recover after attempting fsck or xfs_repair, you must force a reboot of the server.
To force reboot the server, in the Classic environment of the NAVER Cloud Platform console, navigate to i_menu > Services > Compute > Server > Server and select the server to force reboot, then click [Force stop] > [Start], in that order.

Caution

If the server does not normalize after the forced reboot, data recovery through snapshot is required. See Create storage with snapshot.

How to recover Windows server file system errors

To recover file system errors on a Windows server, check how to recover file system errors according to the situation with the following:

Commands to recover Windows server file system

Use the chkdsk command to recover file system errors on a Windows server. chkdsk is a Windows tool that supports file system check and recovery. If you run it without any options, it only checks the default storage.

Caution

If you cancel or stop the chkdsk running midway, although it will not be damaged more than the previous state, you should wait for the chkdsk run to complete.

Note

When you run the chkdsk command, the supported option information is as follows: For more information, see Microsoft article.

Option Description
/f Recover detected errors
  • The drive must be locked when applying this option. If it is not locked, a message is displayed indicating whether to schedule the drive check on the next reboot
/r Recover readable information by finding a bad sector
  • The drive must be locked when applying this option
/x Force unmount the drive if necessary, or invalidate all open handles on the drive

When OS hang occurs

If an OS hang occurs, you must force reboot the server.
To force reboot the server, in the Classic environment of the NAVER Cloud Platform console, navigate to i_menu > Services > Compute > Server > Server and select the server to force reboot, then click [Force stop] > [Start], in that order.

Caution

If the server does not normalize after the forced reboot, data recovery through snapshot is required. See Create storage with snapshot.

When a file system error occurs in the default storage

The following shows how to recover the file system errors occurring in the default storage.

  1. Enter the following command to run chkdsk.
    chkdsk C: /f /r /x
  2. Enter Y in the message to schedule a drive check on the next reboot.
  3. Proceed with the reboot.

When a file system error occurs in the additional storage

The following shows how to recover the file system errors occurring in the additional storage.

  1. Run the run - diskmgmt.msc command to check the drive name of the additional storage to be recovered.
  2. Enter the following command to run chkdsk.
    chkdsk <drive name> /f /r /x