[SOLVED] Slow Data Transfer between two USB3 HDD's (similar to USB2 speeds)

[Laugh2]

BACKGROUND
I'm transferring 2TB of data from an external HDD to a second external HDD. Both are USB3 drives connected by cables with USB A and the flat USB3 Micro B connectors. The USB ports in use are both blue USB3 ports at the back of the desktop. The 4 CPU cores are running at around 30% on average. The system is LM20 (fully updated).

The command I'm using for the transfer is

Code: Select all

rsync -azvuhP '/media/<USER>/<SOURCE DIRECTORY NAME>' '/media/<USER>/<DESTINATION DIRECTORY NAME>'

These rsync parameters are: 1) -a = archive (of all sub-directories) 2) -z = compress (for slow connections) 3) -v = for verbose output 4) -u = skip files that are newer on the receiver 5) -h = for "human readable" output 6) -P = keep partially transferred files and show progress during transfer.

TRANSFER RATES
After 3hours 20minutes I've transferred 245.6GB of a total of 2TB i.e. about 12%, giving a actual transfer rate of 20.5MB/s (or about 74GB/h) resulting in about 29hours for the full 2TB. Rsync reports transfer rates between 27MB/s and 31MB/s for large files and less for small files. However, this transfer rate matches USB2 and not USB3 rates.

Benchmarking by the DISKS utility in LM20 gives the following transfer rates:
- Source HDD (format NTFS): Ave.Read Rate=132.6MB/s, Ave.Write Rate=78.0MB/s, Ave.Access Time=15.38ms
- Destination HDD (format Ext4): Ave.Read Rate=133.4MB/s, Ave.Write Rate=62.6.0MB/s, Ave.Access Time=19.23ms

QUESTION
So the question is WHY is the transfer running only at USB2 speeds?

I've looked at some other postings but I haven't been able to resolve the problem:
viewtopic.php?t=271364
https://www.linux.org/threads/slow-and- ... 00g.20731/
https://askubuntu.com/questions/1289433 ... ead-speeds

SYSTEM / INFO
The outputs for the following commands are given below:
1. inxi -Fxz
2. lsusb
3. lsusb -t
4. dmesg | grep usb

System = Linux Mint 20 Ulyana 64-bit (MATE 1.24.0)
Kernel = Linux 5.4.0-96-generic x86_64
Memory = 15.5 GiB
CPU = Intel Core i5-7400 CPU @ 3.00GHz × 4

Code: Select all

inxi -Fxz
System:
  Kernel: 5.4.0-96-generic x86_64 bits: 64 compiler: gcc v: 9.3.0 
  Desktop: MATE 1.24.0 Distro: Linux Mint 20 Ulyana base: Ubuntu 20.04 focal 
Machine:
  Type: Desktop System: Gigabyte product: H110M-H v: N/A serial: <filter> 
  Mobo: Gigabyte model: H110M-H-CF v: x.x serial: <filter> 
  UEFI [Legacy]: American Megatrends v: F24 date: 04/11/2018 
...
CPU:
  Topology: Quad Core model: Intel Core i5-7400 bits: 64 type: MCP 
  arch: Kaby Lake rev: 9 L2 cache: 6144 KiB 
  flags: avx avx2 lm nx pae sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx 
  bogomips: 24000 
  Speed: 800 MHz min/max: 800/3500 MHz Core speeds (MHz): 1: 800 2: 800 
  3: 800 4: 800 
...
Drives:
  Local Storage: total: 5.68 TiB used: 3.29 TiB (57.9%) 
  ID-1: /dev/sda vendor: SanDisk model: SDSSDA240G size: 223.57 GiB 
  ID-2: /dev/sdb type: USB vendor: Seagate model: Expansion Desk size: 2.73 TiB 
  ID-3: /dev/sdc vendor: Seagate model: ST1000DX001-1NS162 size: 931.51 GiB 
  ID-4: /dev/sdd type: USB vendor: Seagate model: Expansion Desk size: 1.82 TiB 
Partition:
  ID-1: / size: 218.57 GiB used: 194.10 GiB (88.8%) fs: ext4 dev: /dev/sda5 
...

Code: Select all

lsusb
Bus 002 Device 005: ID 0bc2:331a Seagate RSS LLC Expansion Desk
Bus 002 Device 006: ID 0bc2:3320 Seagate RSS LLC SRD00F2 [Expansion Desktop Drive]
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 003: ID 046d:c52b Logitech, Inc. Unifying Receiver
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

Code: Select all

lsusb -t
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/4p, 5000M
    |__ Port 3: Dev 6, If 0, Class=Mass Storage, Driver=uas, 5000M
    |__ Port 4: Dev 5, If 0, Class=Mass Storage, Driver=uas, 5000M
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci_hcd/10p, 480M
    |__ Port 6: Dev 3, If 0, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 6: Dev 3, If 1, Class=Human Interface Device, Driver=usbhid, 12M
    |__ Port 6: Dev 3, If 2, Class=Human Interface Device, Driver=usbhid, 12M

Code: Select all

dmesg | grep usb
[    0.231876] usbcore: registered new interface driver usbfs
[    0.231879] usbcore: registered new interface driver hub
[    0.231883] usbcore: registered new device driver usb
[    0.298119] pci 0000:00:14.0: quirk_usb_early_handoff+0x0/0x670 took 30555 usecs
[    0.642427] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 5.04
[    0.642428] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.642428] usb usb1: Product: xHCI Host Controller
[    0.642429] usb usb1: Manufacturer: Linux 5.4.0-96-generic xhci-hcd
[    0.642430] usb usb1: SerialNumber: 0000:00:14.0
[    0.643539] usb usb2: New USB device found, idVendor=1d6b, idProduct=0003, bcdDevice= 5.04
[    0.643539] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[    0.643540] usb usb2: Product: xHCI Host Controller
[    0.643541] usb usb2: Manufacturer: Linux 5.4.0-96-generic xhci-hcd
[    0.643541] usb usb2: SerialNumber: 0000:00:14.0
[    0.899856] usb 1-2: new high-speed USB device number 2 using xhci_hcd
[    1.048520] usb 1-2: New USB device found, idVendor=152d, idProduct=2351, bcdDevice= 0.00
[    1.048522] usb 1-2: New USB device strings: Mfr=10, Product=11, SerialNumber=5
[    1.048523] usb 1-2: Product: USB/eSATA To SATA Birdge
[    1.048523] usb 1-2: Manufacturer: JMicron
[    1.048524] usb 1-2: SerialNumber: DCC2E949588F
[    1.051801] usb-storage 1-2:1.0: USB Mass Storage device detected
[    1.051854] scsi host4: usb-storage 1-2:1.0
[    1.051901] usbcore: registered new interface driver usb-storage
[    1.052665] usbcore: registered new interface driver uas
[    1.180042] usb 2-1: new SuperSpeed Gen 1 USB device number 2 using xhci_hcd
[    1.188616] usb 2-1: New USB device found, idVendor=0bc2, idProduct=3320, bcdDevice= 1.00
[    1.188617] usb 2-1: New USB device strings: Mfr=2, Product=3, SerialNumber=1
[    1.188618] usb 2-1: Product: Expansion Desk
[    1.188619] usb 2-1: Manufacturer: Seagate
[    1.188619] usb 2-1: SerialNumber: NA4MMZVS
[    1.315967] usb 1-6: new full-speed USB device number 3 using xhci_hcd
[    1.466745] usb 1-6: New USB device found, idVendor=046d, idProduct=c52b, bcdDevice=12.03
[    1.466746] usb 1-6: New USB device strings: Mfr=1, Product=2, SerialNumber=0
[    1.466747] usb 1-6: Product: USB Receiver
[    1.466748] usb 1-6: Manufacturer: Logitech
[    1.476406] usbcore: registered new interface driver usbhid
[    1.476406] usbhid: USB HID core driver
[    1.478056] input: Logitech USB Receiver as /devices/pci0000:00/0000:00:14.0/usb1/1-6/1-6:1.0/0003:046D:C52B.0001/input/input4
[    1.536191] hid-generic 0003:046D:C52B.0001: input,hidraw0: USB HID v1.11 Keyboard [Logitech USB Receiver] on usb-0000:00:14.0-6/input0
[    1.536370] input: Logitech USB Receiver Mouse as /devices/pci0000:00/0000:00:14.0/usb1/1-6/1-6:1.1/0003:046D:C52B.0002/input/input5
[    1.536651] input: Logitech USB Receiver Consumer Control as /devices/pci0000:00/0000:00:14.0/usb1/1-6/1-6:1.1/0003:046D:C52B.0002/input/input6
[    1.592024] usb 2-3: new SuperSpeed Gen 1 USB device number 3 using xhci_hcd
[    1.596113] input: Logitech USB Receiver System Control as /devices/pci0000:00/0000:00:14.0/usb1/1-6/1-6:1.1/0003:046D:C52B.0002/input/input7
[    1.596414] hid-generic 0003:046D:C52B.0002: input,hiddev0,hidraw1: USB HID v1.11 Mouse [Logitech USB Receiver] on usb-0000:00:14.0-6/input1
[    1.596695] hid-generic 0003:046D:C52B.0003: hiddev1,hidraw2: USB HID v1.11 Device [Logitech USB Receiver] on usb-0000:00:14.0-6/input2
[    1.612785] usb 2-3: New USB device found, idVendor=2537, idProduct=1066, bcdDevice= 1.00
[    1.612786] usb 2-3: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    1.612787] usb 2-3: Product: NS1066
[    1.612788] usb 2-3: Manufacturer: Norelsys
[    1.612788] usb 2-3: SerialNumber: 0123456789ABCDE
[    1.614042] usb-storage 2-3:1.0: USB Mass Storage device detected
[    1.614157] scsi host6: usb-storage 2-3:1.0
[    1.740065] usb 2-4: new SuperSpeed Gen 1 USB device number 4 using xhci_hcd
[    1.760672] logitech-djreceiver 0003:046D:C52B.0003: hiddev0,hidraw0: USB HID v1.11 Device [Logitech USB Receiver] on usb-0000:00:14.0-6/input2
[    1.760964] usb 2-4: New USB device found, idVendor=0bc2, idProduct=331a, bcdDevice= 9.11
[    1.760965] usb 2-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[    1.760966] usb 2-4: Product: Expansion Desk
[    1.760966] usb 2-4: Manufacturer: Seagate
[    1.760967] usb 2-4: SerialNumber: NA8X66Q4
[    1.881592] input: Logitech Unifying Device. Wireless PID:2010 Keyboard as /devices/pci0000:00/0000:00:14.0/usb1/1-6/1-6:1.2/0003:046D:C52B.0003/0003:046D:2010.0004/input/input9
[    1.881640] input: Logitech Unifying Device. Wireless PID:2010 Consumer Control as /devices/pci0000:00/0000:00:14.0/usb1/1-6/1-6:1.2/0003:046D:C52B.0003/0003:046D:2010.0004/input/input10
[    1.881668] input: Logitech Unifying Device. Wireless PID:2010 System Control as /devices/pci0000:00/0000:00:14.0/usb1/1-6/1-6:1.2/0003:046D:C52B.0003/0003:046D:2010.0004/input/input11
[    1.881700] hid-generic 0003:046D:2010.0004: input,hidraw1: USB HID v1.11 Keyboard [Logitech Unifying Device. Wireless PID:2010] on usb-0000:00:14.0-6/input2:1
[    1.883536] input: Logitech Unifying Device. Wireless PID:4031 Mouse as /devices/pci0000:00/0000:00:14.0/usb1/1-6/1-6:1.2/0003:046D:C52B.0003/0003:046D:4031.0005/input/input15
[    1.883590] hid-generic 0003:046D:4031.0005: input,hidraw2: USB HID v1.11 Mouse [Logitech Unifying Device. Wireless PID:4031] on usb-0000:00:14.0-6/input2:2
[    2.017580] input: Logitech K800 as /devices/pci0000:00/0000:00:14.0/usb1/1-6/1-6:1.2/0003:046D:C52B.0003/0003:046D:2010.0004/input/input19
[    2.017778] logitech-hidpp-device 0003:046D:2010.0004: input,hidraw1: USB HID v1.11 Keyboard [Logitech K800] on usb-0000:00:14.0-6/input2:1
[    2.077517] input: Logitech M310/M310t as /devices/pci0000:00/0000:00:14.0/usb1/1-6/1-6:1.2/0003:046D:C52B.0003/0003:046D:4031.0005/input/input20
[    2.077720] logitech-hidpp-device 0003:046D:4031.0005: input,hidraw2: USB HID v1.11 Mouse [Logitech M310/M310t] on usb-0000:00:14.0-6/input2:2
[ 2274.940374] usb 1-2: USB disconnect, device number 2
[ 3422.288212] usb 2-1: USB disconnect, device number 2
[ 3493.771491] usb 2-3: USB disconnect, device number 3
[ 3554.425974] usb 2-4: USB disconnect, device number 4
[ 3556.430098] usb 2-4: new SuperSpeed Gen 1 USB device number 5 using xhci_hcd
[ 3556.451326] usb 2-4: New USB device found, idVendor=0bc2, idProduct=331a, bcdDevice= 9.11
[ 3556.451331] usb 2-4: New USB device strings: Mfr=1, Product=2, SerialNumber=3
[ 3556.451335] usb 2-4: Product: Expansion Desk
[ 3556.451338] usb 2-4: Manufacturer: Seagate
[ 3556.451341] usb 2-4: SerialNumber: NA8X66Q4
[ 3576.769395] usb 2-3: new SuperSpeed Gen 1 USB device number 6 using xhci_hcd
[ 3576.790280] usb 2-3: New USB device found, idVendor=0bc2, idProduct=3320, bcdDevice= 1.00
[ 3576.790282] usb 2-3: New USB device strings: Mfr=2, Product=3, SerialNumber=1
[ 3576.790283] usb 2-3: Product: Expansion Desk
[ 3576.790283] usb 2-3: Manufacturer: Seagate
[ 3576.790284] usb 2-3: SerialNumber: NA4MMZVS

[vimes666]

2) -z = compress (for slow connections)

The compress is a bottleneck here. You do not need that on a usb-usb connection.

[Laugh2]

2) -z = compress (for slow connections)

The compress is a bottleneck here. You do not need that on a usb-usb connection.

Thank you, wimes666. That did make a very useful improvement.

NEW TRANSFER RATE
I removed the -z option i.e. compression, and transferred data for 90min. 266.8GB were transferred at a rate of 49MB/s i.e. 178GB/h, which is a 2.4 times acceleration. High transfer rates for large files were around 110MB/s, which were 3.5 to 4 times faster. Very useful!

EXPECTED/DESIRED TRANSFER RATE
USB2 has a theoretical transfer rate of 480Mb/s or 60MB/s, so the transfer is now approaching the maximum theoretical transfer rate for USB2. In reality real transfer rates are less than theoretical i.e. less than 60MB/s. However, I'm using USB3.0, which has a maximum theoretical transfer rate of 5Gb/s or 625MB/s, which is 10 times higher. So are higher transfer rates to my external HDD's realistic?
Refer e.g. https://www.partitionwizard.com/partiti ... peeds.html

I tried to find some numbers for actual transfer rates to external HDD's in the internet.
1) Timan states "For example, the sustained (real-life) transfer speeds for external hard drives are around 85MBps for USB 3.0 and 22MBps for USB 2.0, which indicates about a 4 times improvement." Refer: https://www.auslogics.com/en/articles/h ... fer-speed/
2) Dr.Gough tests a Maxtor M3 Portable 4Tb USB 3.0 and achieves lower transfer rates with write caching off. Although my internal HDD has write caching enabled, I was unable to enable write caching for the two external HDDs using

Code: Select all

sudo hdparm -W /dev/sdd

Refer: https://goughlui.com/2016/09/28/review- ... m401tcbgm/
3) George B. posted real transfer rates of 40MB/s to an external HDD with USB3.0 also using RSYNC. He wrote "... Pretty Good. No hangup, crashes, or errors ...". Refer: https://www.linux.org/threads/slow-and- ... 00g.20731/

So it seems I may have reached the maximum achievable transfer rates for these HDD's at 49MB/s i.e. 178GB/h, and should accept long transfer times when handling TB's of data. For this reason I've added [SOLVED] to the title. Thank you, vimes666.

[EDIT]
The rate of transfer and additionally compression of the backup/archive can be achieved using TAR and PIGZ e.g. refer:
viewtopic.php?f=47&t=366980 . With my i5 CPU I achieved an acceleration of about 4x i.e. more than 80MB/s compared to 20.5MB/s, which I had initially. (Since the 4 CPU cores were running at 100% then the only way to further accelerate may be software compressing faster than PIGZ.)