[WORKAROUND] Major disk errors (but only when running QEMU)

[ajgringo619]

Got another strange situation that I'm trying to resolve. On (3) different VMs, all located in my $HOME partition (an SSD on /dev/sdb2), I've had my system partially lockup. The journal shows these errors (this is from my host system; the VM logs are clean):

Code: Select all

Nov 28 11:23:29 dss-mint kernel: ata5.00: exception Emask 0x0 SAct 0x7e21 SErr 0x0 action 0x6 frozen
Nov 28 11:23:29 dss-mint kernel: ata5.00: failed command: READ FPDMA QUEUED
Nov 28 11:23:29 dss-mint kernel: ata5.00: cmd 60/08:00:b0:d1:75/00:00:01:00:00/40 tag 0 ncq dma 4096 in
                                          res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 28 11:23:29 dss-mint kernel: ata5.00: status: { DRDY }
Nov 28 11:23:29 dss-mint kernel: ata5.00: failed command: WRITE FPDMA QUEUED
Nov 28 11:23:29 dss-mint kernel: ata5.00: cmd 61/00:28:00:b8:5b/0a:00:18:00:00/40 tag 5 ncq dma 1310720 ou
                                          res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 28 11:23:29 dss-mint kernel: ata5.00: status: { DRDY }
Nov 28 11:23:29 dss-mint kernel: ata5.00: failed command: READ FPDMA QUEUED
Nov 28 11:23:29 dss-mint kernel: ata5.00: cmd 60/28:48:c8:e9:8d/00:00:02:00:00/40 tag 9 ncq dma 20480 in
                                          res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 28 11:23:29 dss-mint kernel: ata5.00: status: { DRDY }
Nov 28 11:23:29 dss-mint kernel: ata5.00: failed command: READ FPDMA QUEUED
Nov 28 11:23:29 dss-mint kernel: ata5.00: cmd 60/10:50:00:ea:8d/00:00:02:00:00/40 tag 10 ncq dma 8192 in
                                          res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 28 11:23:29 dss-mint kernel: ata5.00: status: { DRDY }
Nov 28 11:23:29 dss-mint kernel: ata5.00: failed command: READ FPDMA QUEUED
Nov 28 11:23:29 dss-mint kernel: ata5.00: cmd 60/08:58:20:ea:8d/00:00:02:00:00/40 tag 11 ncq dma 4096 in
                                          res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 28 11:23:29 dss-mint kernel: ata5.00: status: { DRDY }
Nov 28 11:23:29 dss-mint kernel: ata5.00: failed command: READ FPDMA QUEUED
Nov 28 11:23:29 dss-mint kernel: ata5.00: cmd 60/40:60:30:ea:8d/00:00:02:00:00/40 tag 12 ncq dma 32768 in
                                          res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 28 11:23:29 dss-mint kernel: ata5.00: status: { DRDY }
Nov 28 11:23:29 dss-mint kernel: ata5.00: failed command: READ FPDMA QUEUED
Nov 28 11:23:29 dss-mint kernel: ata5.00: cmd 60/20:68:78:ea:8d/00:00:02:00:00/40 tag 13 ncq dma 16384 in
                                          res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 28 11:23:29 dss-mint kernel: ata5.00: status: { DRDY }
Nov 28 11:23:29 dss-mint kernel: ata5.00: failed command: READ FPDMA QUEUED
Nov 28 11:23:29 dss-mint kernel: ata5.00: cmd 60/08:70:a8:ad:b3/00:00:00:00:00/40 tag 14 ncq dma 4096 in
                                          res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
Nov 28 11:23:29 dss-mint kernel: ata5.00: status: { DRDY }
Nov 28 11:23:29 dss-mint kernel: ata5: hard resetting link
Nov 28 11:23:29 dss-mint kernel: ata5: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Nov 28 11:23:29 dss-mint kernel: ata5.00: configured for UDMA/133
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#9 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#9 Sense Key : Illegal Request [current] 
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#9 Add. Sense: Unaligned write command
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#9 CDB: Read(10) 28 00 02 8d e9 c8 00 00 28 00
Nov 28 11:23:29 dss-mint kernel: blk_update_request: I/O error, dev sdb, sector 42854856 op 0x0:(READ) flags 0x80700 phys_seg 5 prio class 0
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#10 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#10 Sense Key : Illegal Request [current] 
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#10 Add. Sense: Unaligned write command
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#10 CDB: Read(10) 28 00 02 8d ea 00 00 00 10 00
Nov 28 11:23:29 dss-mint kernel: blk_update_request: I/O error, dev sdb, sector 42854912 op 0x0:(READ) flags 0x80700 phys_seg 2 prio class 0
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#11 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#11 Sense Key : Illegal Request [current] 
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#11 Add. Sense: Unaligned write command
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#11 CDB: Read(10) 28 00 02 8d ea 20 00 00 08 00
Nov 28 11:23:29 dss-mint kernel: blk_update_request: I/O error, dev sdb, sector 42854944 op 0x0:(READ) flags 0x80700 phys_seg 1 prio class 0
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#12 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#12 Sense Key : Illegal Request [current] 
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#12 Add. Sense: Unaligned write command
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#12 CDB: Read(10) 28 00 02 8d ea 30 00 00 40 00
Nov 28 11:23:29 dss-mint kernel: blk_update_request: I/O error, dev sdb, sector 42854960 op 0x0:(READ) flags 0x80700 phys_seg 8 prio class 0
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#13 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#13 Sense Key : Illegal Request [current] 
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#13 Add. Sense: Unaligned write command
Nov 28 11:23:29 dss-mint kernel: sd 4:0:0:0: [sdb] tag#13 CDB: Read(10) 28 00 02 8d ea 78 00 00 20 00
Nov 28 11:23:29 dss-mint kernel: blk_update_request: I/O error, dev sdb, sector 42855032 op 0x0:(READ) flags 0x80700 phys_seg 4 prio class 0
Nov 28 11:23:29 dss-mint kernel: ata5: EH complete

The strange thing is that I can't replicate these errors doing any other tasks, all SMART tests come back negative. I've seen a few references to changing the cache settings on the QCOW2 images, but so far they're not helping. I really don't think that my SSD is failing, as I should have hit at least (1) error running my daily/weekly/monthly backups (which also reside on /dev/sdb). I also use this disk to store my Clonezilla images; again, there's never been a disk failure.

[ajgringo619]

A couple of other notes on this subject:

• The filesystem that my VMs are hosted on is EXT4, so why I'm getting these disk errors when running VMs w/BTRFS is baffling. My initial test of Fedora 33 w/EXT4 seems to be working OK.

• This could be related to my other freezing problem (most certainly related to the v455 branch of Nvidia drivers).

[ajgringo619]

Just got the error/freeze again, this time on a Ubuntu 20 VM (w/EXT4 partitions) trying to build a Linux-from-Scratch system. Could this possibly be an SATA cable issue? I'm grasping at straws here; again, this never happens when I'm only running my host system, no matter how hard I'm pushing it.

[deepakdeshp]

Check your disk for errors.
https://help.ubuntu.com/community/Smartmontools

[ajgringo619]

I've found that switching to VMware has eliminated this problem, although it's probably related to a newer kernel as well.