kernel: EXT fs error

[jappish84]

Hey guys,

I just discovered my system log is filling up with this error:

Jul 22 14:07:31 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:33 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:34 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:36 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:38 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:39 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:41 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:42 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:44 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:45 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:47 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:49 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:50 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:52 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:54 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:55 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:57 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:07:59 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:00 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:02 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:03 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:05 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:06 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:08 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:10 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:11 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:13 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:14 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:16 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:18 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:19 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:21 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:22 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:24 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:25 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:27 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429
Jul 22 14:08:29 kernel: EXT4-fs error (device sda5): ext4_lookup:1051: inode #206864385: comm nfsd: deleted inode referenced: 206864429

Everything seems to be working fine though.

Not sure if this is the place to post this thread, but that's what I'm doing since I'm on Asuswrt-merlin 380.60_beta2 on a RT-AC3200

 

[Nullity]

You are using NFS?

 

[jappish84]

You are using NFS?

Yes, and it's working fine as far as I can tell

 

[Nullity]

Are you familiar with the command-line? If so, try unmounting sda5 and running fsck on it.

 

[jappish84]

I've SSH:d into the router but non of my very basic linux commands are working, I've tried fsck but i'm getting 'not found' so I guess it's not installed? Tried installing with 'apt-get' but that's also 'not found'. I'm I using it wrong?

 

[jappish84]

ok, so I've successfully unmounted and understand I need to use fsck.ext4 command, just need to figure out the flag and device to use now =)

 

[jappish84]

ok, I'm not really sure what I'm doing here, any help would be appreciated.

admin@3TB:/tmp/home/root# fsck.ext4 /dev/sda5
e2fsck 1.42.13 (17-May-2015)
Backing up journal inode block information.

3TB contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Error allocating icount link information: Memory allocation failed

3TB: ***** FILE SYSTEM WAS MODIFIED *****
e2fsck: aborted

3TB: ***** FILE SYSTEM WAS MODIFIED *****
admin@3TB:/tmp/home/root# fsck
-sh: fsck: not found
admin@3TB:/tmp/home/root# fsck.ext4
Usage: fsck.ext4 [-panyrcdfvtDFV] [-b superblock] [-B blocksize]
                [-I inode_buffer_blocks] [-P process_inode_size]
                [-l|-L bad_blocks_file] [-C fd] [-j external_journal]
                [-E extended-options] device

Emergency help:
-p                   Automatic repair (no questions)
-n                   Make no changes to the filesystem
-y                   Assume "yes" to all questions
-c                   Check for bad blocks and add them to the badblock list
-f                   Force checking even if filesystem is marked clean
-v                   Be verbose
-b superblock        Use alternative superblock
-B blocksize         Force blocksize when looking for superblock
-j external_journal  Set location of the external journal
-l bad_blocks_file   Add to badblocks list
-L bad_blocks_file   Set badblocks list
admin@3TB:/tmp/home/root# fsck.ext4 -p /dev/sda5
3TB: Backing up journal inode block information.

3TB contains a file system with errors, check forced.
3TB: Error allocating icount link information: Memory allocation failed

3TB: ***** FILE SYSTEM WAS MODIFIED *****
e2fsck: aborted

3TB: ***** FILE SYSTEM WAS MODIFIED *****
admin@3TB:/tmp/home/root#

 

[ColinTaylor]

Unfortunately your router does not have enough RAM to perform a fsck on such a large drive. Try connecting the drive to a PC running Linux (use a Live-CD if necessary) and perform the fsck from there.

 

[jappish84]

Thanks man, would've never figured that out on my own. Will try connecting it to my pc later tonight. I'll report back here when I'm done.

Should I use the auto repair flag, or any other option with fsck?

 

[ColinTaylor]

Personally I would just use it without any extra options.

 

[Nullity]

I think that your problem stems from the fact that AsusWRT does not safely unmount when restarting or powering-off.

Before restarting the device, you may need to SSH in and manually unmount... or just ignore the warnings that are logged.

Thanks man, would've never figured that out on my own. Will try connecting it to my pc later tonight. I'll report back here when I'm done.

Should I use the auto repair flag, or any other option with fsck?

No flags initially, but if errors are found then you may need to add a flag to fix it.

 

[RMerlin]

I think that your problem stems from the fact that AsusWRT does not safely unmount when restarting or powering-off.

It should, and years ago I even added a trio of sync() calls in the shutdown code to ensure that all buffers are properly flushed. But there are cases where the router actually crashes during a reboot, which could lead to potential filesystem issues.

 

[jappish84]

Thanks for the help guys! Just finished running fsck with my pc and it found and fixed some errors. Router log looks ok now