A way to display file copy progress when copying to a USB stick

Forum rules
Before you post please read how to get help
ColdBootII
Level 3
Level 3
Posts: 169
Joined: Wed Aug 02, 2017 8:19 am

A way to display file copy progress when copying to a USB stick

Postby ColdBootII » Sun Oct 15, 2017 12:27 pm

I don't know whether Scripts&Bash or Tutorial section is more appropriate for this post.

Anyway, most of us, who copy large files to a USB stick, somewhat more often, know the situation... not having a clue how much more time it will take or rather, how much data was written to the stick. The progress bar and the amount of data seem to freeze at some high value, and we have to sit back and give it some (undefined) time to complete...

Here is my unrefined script-workaround to amend the situation a bit. Feel free to improve it in all ways you find necessary (it's completely Open Source :mrgreen: ):

Code: Select all

#!  /bin/bash

while [ $(awk "{ print \$9 }" /sys/block/sdb/stat) -gt 0 ]
    do
        let DataWrittenUSB=$(awk "{ print \$7 }" /sys/block/sdb/stat)*512/1000000
        notify-send -t 5 $DataWrittenUSB'MB written to USB...'
        sleep 5
    done

notify-send "Copy to USB has finished/not started."


As you can see, it displays how much data was written to the stick and if you intend to try it, you should first check what is your USB's device name in /sys/log as it may not be sdb as is on my machine. One other thing, you need to know the sector size of your USB stick and if it's not 512 bytes, change that accordingly here:

Code: Select all

/sys/block/sdb/stat)*<type sector size here>/1024/1024


Didn't know how to display continually changing value in Zenity (if it's possible) so I've got the built-in notify-send to take it on (and it doesn't steel the focus). Creates a bundle of notifications, in the process, but what's perfect these days?.. :lol:

Cheers! :D
Last edited by ColdBootII on Mon Oct 16, 2017 10:06 am, edited 1 time in total.

ColdBootII
Level 3
Level 3
Posts: 169
Joined: Wed Aug 02, 2017 8:19 am

Re: A way to display file copy progress when copying to a USB stick

Postby ColdBootII » Sun Oct 15, 2017 9:10 pm

Heh, :mrgreen:

I just recalled there is CommandRunner applet for Cinnamon which is perfect. Like Genmon Xfce plugin, it can display stdout coming from a script so, you won't need to clear hundreds of notifications after all. :mrgreen:

A modified version to fit with this applet:

Code: Select all

#!  /bin/bash

if [ $(awk "{ print \$9 }" /sys/block/sdb/stat) -gt 0 ]
    then
        let DataWrittenUSB=$(awk "{ print \$7 }" /sys/block/sdb/stat)*512/1000000
        echo $DataWrittenUSB'MB written...'
 else
        echo "USB idle"
 fi
 

Image

For the life of me, I can't remember what could come in handy for this purpose in KDE's panel but there must be something you'll find out...

Cheers! :D
Last edited by ColdBootII on Mon Oct 16, 2017 10:06 am, edited 1 time in total.

User avatar
catweazel
Level 10
Level 10
Posts: 3323
Joined: Fri Oct 12, 2012 9:44 pm
Location: Australian Antarctic Territory

Re: A way to display file copy progress when copying to a USB stick

Postby catweazel » Mon Oct 16, 2017 1:44 am

ColdBootII wrote:For the life of me, I can't remember what could come in handy for this purpose in KDE's panel but there must be something you'll find out...

Some twit b0rked KDE's ability to show progress dialogs. The 'fixes' posted online don't work any more. GRRRR!
If your problem is fixed, please mark your thread as [SOLVED] by editing the title of the first message in the thread.

ColdBootII
Level 3
Level 3
Posts: 169
Joined: Wed Aug 02, 2017 8:19 am

Re: A way to display file copy progress when copying to a USB stick

Postby ColdBootII » Mon Oct 16, 2017 10:18 am

I've made a typical mistake (for me) dividing with 1024 when 1000 bytes is a kilobyte - corrected... :mrgreen:

With xclip installed, it should be possible to find and display the total size of files to be copied next to copy progress. Just a wee bit more work to deal with long file names containing awkward characters and spaces.

catweazel wrote:
Some twit b0rked KDE's ability to show progress dialogs. The 'fixes' posted online don't work any more. GRRRR!


They'll gonna fix it, don't fret it. :mrgreen:

User avatar
BG405
Level 5
Level 5
Posts: 827
Joined: Fri Mar 11, 2016 3:09 pm
Location: England

Re: A way to display file copy progress when copying to a USB stick

Postby BG405 » Mon Oct 16, 2017 11:54 am

catweazel wrote:Some twit b0rked KDE's ability to show progress dialogs.

Like this, you mean?
Screenshot_20171016_160800_Cropped.jpeg

Not very clear I know, I can't find the transparency settings :oops: I'll have to resize that thin progress bar as well.
Dell Inspiron 1525 - LM17.3 CE 64-------------------Acer D255E 2GB - LM17.3 KDE 32
Toshiba NB305 - LM17.3 Xfce 32--------------------K7S5A Athlon 1.2GHz - LM17.3 Xfce 32 & WinXP-Pro
Acer Aspire E11 ES1-111M - LM18.2 KDE 64 -----Dell PII 350 64MB - Puppy 4.3 & Win98-SE

User avatar
Misko_2083
Level 1
Level 1
Posts: 15
Joined: Mon Jan 12, 2015 10:17 am

Re: A way to display file copy progress when copying to a USB stick

Postby Misko_2083 » Mon Oct 16, 2017 1:52 pm

ColdBootII wrote:I just recalled there is CommandRunner applet for Cinnamon which is perfect. Like Genmon Xfce plugin, it can display stdout coming from a script so, you won't need to clear hundreds of notifications after all. :mrgreen:

A modified version to fit with this applet:

Code: Select all

#!  /bin/bash

if [ $(awk "{ print \$9 }" /sys/block/sdb/stat) -gt 0 ]
    then
        let DataWrittenUSB=$(awk "{ print \$7 }" /sys/block/sdb/stat)*512/1000000
        echo $DataWrittenUSB'MB written...'
 else
        echo "USB idle"
 fi
 


To tell you the truth, I have no idea why would you monitor when something is copied to usb when there is a progress bar in most file managers.
Perhaps it can be used for conky or semething else.
Anyway, I decided to give it a try with Xfce's "Generic monitor" plugin and upgraded your script to watch for all usb drives. I've set the update period to 2 seconds.

Code: Select all

#!/bin/bash

if ls -l /dev/disk/by-path/*usb* &>/dev/null; then

   USB_DET=($(ls -l /dev/disk/by-path/*usb* | grep -v "part" | awk '{print $NF}' | awk -F "/" '{print $NF}' | sort))

   for usb in "${USB_DET[@]}"
   do
      if [ $(awk "{ print \$9 }" /sys/block/$usb/stat) -gt 0 ]
      then
         sector_size=$(cat /sys/block/$usb/queue/hw_sector_size)

         DATA_WRITTEN=$(awk '{load = $7*sector/1000/1000
                  printf "%.1f", load}'  sector="$sector_size" /sys/block/$usb/stat)
              printf "%s: %s%s " "$usb" "$DATA_WRITTEN" "MiB"
      else
         printf "%s: %s " "$usb" "idle"
      fi
   done
else
   echo "None"
fi

If no usb drive is plugged in it wil write out "None"
Image
You can insert multiple usb drives.
Image
When you copy to usb drive.
Image
It writes out with the floating point which is defined in the awk command printf "%.1f"
Not sure how it will bahave in CommandRunner.

This was just for fun. When coping files, I usually use the pv command to displays a progress bar.

ColdBootII
Level 3
Level 3
Posts: 169
Joined: Wed Aug 02, 2017 8:19 am

Re: A way to display file copy progress when copying to a USB stick

Postby ColdBootII » Mon Oct 16, 2017 2:36 pm

Misko_2083 wrote:To tell you the truth, I have no idea why would you monitor when something is copied to usb when there is a progress bar in most file managers.
Perhaps it can be used for conky or semething else.
Anyway, I decided to give it a try with Xfce's "Generic monitor" plugin and upgraded your script to watch for all usb drives. I've set the update period to 2 seconds.

Code: Select all

#!/bin/bash

if ls -l /dev/disk/by-path/*usb* &>/dev/null; then

   USB_DET=($(ls -l /dev/disk/by-path/*usb* | grep -v "part" | awk '{print $NF}' | awk -F "/" '{print $NF}' | sort))

   for usb in "${USB_DET[@]}"
   do
      if [ $(awk "{ print \$9 }" /sys/block/$usb/stat) -gt 0 ]
      then
         sector_size=$(cat /sys/block/$usb/queue/hw_sector_size)

         DATA_WRITTEN=$(awk '{load = $7*sector/1000/1000
                  printf "%.1f", load}'  sector="$sector_size" /sys/block/$usb/stat)
              printf "%s: %s%s " "$usb" "$DATA_WRITTEN" "MiB"
      else
         printf "%s: %s " "$usb" "idle"
      fi
   done
else
   echo "None"
fi

If no usb drive is plugged in it wil write out "None"
Image
You can insert multiple usb drives.
Image
When you copy to usb drive.
Image
It writes out with the floating point which is defined in the awk command printf "%.1f"
Not sure how it will bahave in CommandRunner.

This was just for fun. When coping files, I usually use the pv command to displays a progress bar.


Thanks Mishko_2083, nicely done! :D

I use Nemo for most any file-managerial purposes and I wanted this because the file-copy progress to USB, as displayed in the dialog, is inaccurate. It seems to be showing only the progress of copying file(s) to memory cache(which is very fast) and then stalls, for longer or shorter periods depending on sizes of file(s) copied, because it doesn't display actual syncing progress at all. pv doesn't cut it for me either nor any other file manager I've tried using in Cinnamon. I have no idea why is that...

Anyway, I have given up. This is as far as this script can go from my POV since I don't know how to pass the info about selected files to the script, when they are dragged-and-dropped to USB in Nemo. Apparently, it is not a copy operation that involves the use of clipboard and xclip shows nothing of use.

Cheers! :D

User avatar
BG405
Level 5
Level 5
Posts: 827
Joined: Fri Mar 11, 2016 3:09 pm
Location: England

Re: A way to display file copy progress when copying to a USB stick

Postby BG405 » Mon Oct 16, 2017 6:03 pm

ColdBootII wrote:doesn't display actual syncing progress

I see what you mean. I have noticed the same thing when writing floppy images to actual floppy disks (the big 5¼-inch ones which really are floppy lol) - sometimes dd reports it's copied and then the disk drive comes to life a second or so later .. these USB sticks probably do have a small buffer as well.

Not sure of a practical way round this without interrupting data writes at timed intervals for a check, which would be a lot slower.
Dell Inspiron 1525 - LM17.3 CE 64-------------------Acer D255E 2GB - LM17.3 KDE 32
Toshiba NB305 - LM17.3 Xfce 32--------------------K7S5A Athlon 1.2GHz - LM17.3 Xfce 32 & WinXP-Pro
Acer Aspire E11 ES1-111M - LM18.2 KDE 64 -----Dell PII 350 64MB - Puppy 4.3 & Win98-SE

ColdBootII
Level 3
Level 3
Posts: 169
Joined: Wed Aug 02, 2017 8:19 am

Re: A way to display file copy progress when copying to a USB stick

Postby ColdBootII » Mon Oct 16, 2017 7:50 pm

(I have inadvertently destroyed this post, sorry :oops: :mrgreen: )
Last edited by ColdBootII on Wed Oct 18, 2017 5:46 pm, edited 3 times in total.

User avatar
catweazel
Level 10
Level 10
Posts: 3323
Joined: Fri Oct 12, 2012 9:44 pm
Location: Australian Antarctic Territory

Re: A way to display file copy progress when copying to a USB stick

Postby catweazel » Tue Oct 17, 2017 1:59 am

BG405 wrote:
catweazel wrote:Some twit b0rked KDE's ability to show progress dialogs.

Like this, you mean?Screenshot_20171016_160800_Cropped.jpeg
Not very clear I know

Yes, that. The alleged fix doesn't work.
If your problem is fixed, please mark your thread as [SOLVED] by editing the title of the first message in the thread.

ColdBootII
Level 3
Level 3
Posts: 169
Joined: Wed Aug 02, 2017 8:19 am

Re: A way to display file copy progress when copying to a USB stick

Postby ColdBootII » Wed Oct 18, 2017 5:38 pm

However, if it happens to be writing to a NTFS, it closes much too soon and I can only probe, now and then, if transfers are over by attempting to safely remove it. Haven't had the time to see if the script will fare any better.


As an illustration of the above, quoted from my previous post in this thread, I just did a copy to a NTFS formated USB 2.0 flash stick. I attempted to copy 4 files of 2.8GB total size and the progress dialog went away before 50% was really written to the stick...:

Image

On the other hand, the script went on and showed that USB is idle exactly indicating when it can be removed. For those interested, reference for the write sectors field in stat is here:

https://www.kernel.org/doc/Documentation/block/stat.txt

Cheers! :D


Return to “Scripts & Bash”

Who is online

Users browsing this forum: No registered users and 2 guests