Virtual-box and VDI file

[gm10]

I ran a test with a Linux VM (using nbd) and it crashed hard when I tried to unmount; this was using the 4.18.0-14 kernel. Unfortunately, the only examples I can find on the 'net also complain about instability, but they are 8+ years old.

"with a Linux VM" means you tried to nbd mount the vdi from within a VM? I never tried that but that sounds like inception.

[ajgringo619]

No, from the host (testing another distro). After the unmount crashed, I had to delete the .vdi file (couldn't even reinstall). I think the problem was using a dynamic disk.

[gm10]

I'm using dynamic disks, I never crashed. Did you have the vdi mounted in a VM at the same time maybe? I know that causes corruption as I believe I already mentioned, but never led to a crash on my end, either. But could be any number of factors including what was actually in the vdi and the file systems in use.

But couldn't you simply reboot and then use the vdi normally again?

[ajgringo619]

The VM was not running. I copied over a bunch of source files (no issues - over 100MB), then tried to unmount. My HDD light when crazy, but the command never finished. I rebooted, tried to launched the VM - got stuck after the GRUB screen. Tried to reinstall the system, installer crashed when installing GRUB.

Was this more related to the 4.18 kernel I was testing or the dynamic disk? No idea. I'm going to try again later. Again, the only references I found to this were talking about Ubuntu 10 and earlier.

[ajgringo619]

Just tested again under kernel 4.15.0-45. Took almost 3 minutes for the umount command to finish, but at least it finished this time. I checked the system logs and found this:

Code: Select all

Feb 05 10:14:04 kernel: INFO: task umount:27254 blocked for more than 120 seconds.
Feb 05 10:14:04 kernel:       Tainted: G           OE    4.15.0-45-generic #48-Ubuntu
Feb 05 10:14:04 kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Feb 05 10:14:04 kernel: umount          D    0 27254  15665 0x00000000
Feb 05 10:14:04 kernel: Call Trace:
Feb 05 10:14:04 kernel:  __schedule+0x291/0x8a0
Feb 05 10:14:04 kernel:  schedule+0x2c/0x80
Feb 05 10:14:04 kernel:  wb_wait_for_completion+0x64/0x90
Feb 05 10:14:04 kernel:  ? wait_woken+0x80/0x80
Feb 05 10:14:04 kernel:  __writeback_inodes_sb_nr+0xa1/0xd0
Feb 05 10:14:04 kernel:  writeback_inodes_sb+0x27/0x30
Feb 05 10:14:04 kernel:  __sync_filesystem+0x51/0x60
Feb 05 10:14:04 kernel:  sync_filesystem+0x28/0x40
Feb 05 10:14:04 kernel:  generic_shutdown_super+0x27/0x120
Feb 05 10:14:04 kernel:  kill_block_super+0x2c/0x80
Feb 05 10:14:04 kernel:  deactivate_locked_super+0x48/0x80
Feb 05 10:14:04 kernel:  deactivate_super+0x40/0x60
Feb 05 10:14:04 kernel:  cleanup_mnt+0x3f/0x80
Feb 05 10:14:04 kernel:  __cleanup_mnt+0x12/0x20
Feb 05 10:14:04 kernel:  task_work_run+0x9d/0xc0
Feb 05 10:14:04 kernel:  exit_to_usermode_loop+0xc0/0xd0
Feb 05 10:14:04 kernel:  do_syscall_64+0x115/0x130
Feb 05 10:14:04 kernel:  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
Feb 05 10:14:04 kernel: RIP: 0033:0x7f5f54f018c7
Feb 05 10:14:04 kernel: RSP: 002b:00007ffe5b943cf8 EFLAGS: 00000246 ORIG_RAX: 00000000000000a6
Feb 05 10:14:04 kernel: RAX: 0000000000000000 RBX: 0000563cf7242a40 RCX: 00007f5f54f018c7
Feb 05 10:14:04 kernel: RDX: 0000000000000001 RSI: 0000000000000000 RDI: 0000563cf7242c20
Feb 05 10:14:04 kernel: RBP: 0000000000000000 R08: 0000563cf7243a50 R09: 0000000000000004
Feb 05 10:14:04 kernel: R10: 000000000000000b R11: 0000000000000246 R12: 0000563cf7242c20
Feb 05 10:14:04 kernel: R13: 00007f5f5542a8a4 R14: 0000000000000000 R15: 0000000000000000

I checked the VM and it's fine - no errors, seems normal. Any ideas?

[gm10]

Dunno, seems you may have had active handles to the mounted file system. Close anything that was accessing it and run

Code: Select all

sudo sync

before unmounting, maybe it helps.

[ajgringo619]

Definitely helps a lot; thank you. I did notice more errors after running umount, but I didn't see any issues with the VM:

Code: Select all

Feb 05 10:46:28 kernel: block nbd0: NBD_DISCONNECT
Feb 05 10:46:28 kernel: block nbd0: shutting down sockets
Feb 05 10:46:28 kernel: nbd0: detected capacity change from 0 to 107374182400
Feb 05 10:46:28 kernel: print_req_error: 2 callbacks suppressed
Feb 05 10:46:28 kernel: print_req_error: I/O error, dev nbd0, sector 0
Feb 05 10:46:28 kernel: buffer_io_error: 2 callbacks suppressed
Feb 05 10:46:28 kernel: Buffer I/O error on dev nbd0, logical block 0, async page read
Feb 05 10:46:28 kernel: print_req_error: I/O error, dev nbd0, sector 0
Feb 05 10:46:28 kernel: Buffer I/O error on dev nbd0, logical block 0, async page read
Feb 05 10:46:28 kernel: print_req_error: I/O error, dev nbd0, sector 0
Feb 05 10:46:28 kernel: Buffer I/O error on dev nbd0, logical block 0, async page read
Feb 05 10:46:28 kernel: ldm_validate_partition_table(): Disk read failed.
Feb 05 10:46:28 kernel: print_req_error: I/O error, dev nbd0, sector 0
Feb 05 10:46:28 kernel: Buffer I/O error on dev nbd0, logical block 0, async page read
Feb 05 10:46:28 kernel: print_req_error: I/O error, dev nbd0, sector 0
Feb 05 10:46:28 kernel: Buffer I/O error on dev nbd0, logical block 0, async page read
Feb 05 10:46:28 kernel: print_req_error: I/O error, dev nbd0, sector 0
Feb 05 10:46:28 kernel: Buffer I/O error on dev nbd0, logical block 0, async page read
Feb 05 10:46:28 kernel: print_req_error: I/O error, dev nbd0, sector 0
Feb 05 10:46:28 kernel: Buffer I/O error on dev nbd0, logical block 0, async page read
Feb 05 10:46:28 kernel: Dev nbd0: unable to read RDB block 0
Feb 05 10:46:28 kernel: print_req_error: I/O error, dev nbd0, sector 0
Feb 05 10:46:28 kernel: Buffer I/O error on dev nbd0, logical block 0, async page read
Feb 05 10:46:28 kernel: print_req_error: I/O error, dev nbd0, sector 0
Feb 05 10:46:28 kernel: Buffer I/O error on dev nbd0, logical block 0, async page read
Feb 05 10:46:28 kernel: print_req_error: I/O error, dev nbd0, sector 24
Feb 05 10:46:28 kernel: Buffer I/O error on dev nbd0, logical block 3, async page read
Feb 05 10:46:28 kernel:  nbd0: unable to read partition table

[thx-1138]

...seems to be a bug. Supposedly fixed under 4.18, but since Ubuntu also comes with it's own set of patches...no idea.

Maybe further try mapping (and/or unloading) the VM's partitions explicitly with kpartx,
& see if then it still misbehaves under recent kernel revisions...
From what few i can understand from the last comment in the bug report above though,
this shouldn't really matter / affect it (making me think it's more likely to be an Ubuntu-related issue)...

[ajgringo619]

Thanks for the update. One thing that I've seen in subsequent tests is that my HDD starts spinning almost immediately after the final file is copied. It seems like the drive has a ton of data cached and didn't start actually committing the writes until the command was done. I think the first crash would have been avoided if I had just been a little more patient.

The only other thing that I changed from previous VB installs is I've enabled the cache on the virtual SATA controller. This was done to speed up the HDD performance (when I was comparing systems to QEMU/KVM). But I can't believe this would be an issue when the VM isn't running.

[thx-1138]

...this is a relatively generic thing that comes to mind (ie. not directly relevant to nbd above),
but have you maybe changed any vm.dirty* settings?
tlp for example (if you have such installed), changes them automatically depending on whether on bat or ac -
but you can obviously manually adjust them to fit your needs if the current settings are causing you issues.

You could also go more...'aggressive', and fiddle with hdparm -W0 /dev/sdx and then back to hdparm -W /dev/sdx,
to turn off disk caching temporarily before loading the vdi & copying files to it, then re-enable it back
(personally, i never fiddle around that much...i just let disk caching on)...

[ajgringo619]

Other than the "Use Host I/O Cache" setting (in the VM), I haven't changed anything. It's working - no disk/file corruption, VMs are loading without error - so I'm going to leave things alone for now.

[Skalman]

Thank You gm10.
First today could I try to install:
skalman@Nissan:~$ apt install qemu-utils
Reading package lists... Done
Building dependency tree
Reading state information... Done

After that I got err messages:

Err:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librados2 amd64 12.2.8-0ubuntu0.18.04.1
404 Not Found [IP: 91.189.88.149 80]
Err:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 librbd1 amd64 12.2.8-0ubuntu0.18.04.1
404 Not Found [IP: 91.189.88.149 80]

and

E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/m ... _amd64.deb 404 Not Found [IP: 91.189.88.149 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/pool/m ... _amd64.deb 404 Not Found [IP: 91.189.88.149 80]
E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

so naturally I got the error:

skalman@Nissan:~$ sudo modprobe nbd
skalman@Nissan:~$ sudo qemu-nbd --connect /dev/nbd0 /home/VirtualBox/XPcomputer/XPcomputer.vdi
sudo: qemu-nbd: command not found
And the /dev catalog has no nbd0 sub-catalog
what to do now/next?
Please give me some hint
thank You.

Someone wrote if i write direct to the vdi file there could be demage in the file order or somthing similar.
Is that the case?
I would likte to be able to write and delete files from the vdi and the VM-sandbox not running att the same time.
If that is not possible then I skip the whole idea.

Thank You.