A way to display file copy progress when copying to a USB stick

About writing shell scripts and making the most of your shell
Forum rules
Topics in this forum are automatically closed 6 months after creation.
Locked
ColdBootII

A way to display file copy progress when copying to a USB stick

Post by ColdBootII »

I don't know whether Scripts&Bash or Tutorial section is more appropriate for this post.

Anyway, most of us, who copy large files to a USB stick, somewhat more often, know the situation... not having a clue how much more time it will take or rather, how much data was written to the stick. The progress bar and the amount of data seem to freeze at some high value, and we have to sit back and give it some (undefined) time to complete...

Here is my unrefined script-workaround to amend the situation a bit. Feel free to improve it in all ways you find necessary (it's completely Open Source :mrgreen: ):

Code: Select all

#!  /bin/bash

while [ $(awk "{ print \$9 }" /sys/block/sdb/stat) -gt 0 ]
    do
        let DataWrittenUSB=$(awk "{ print \$7 }" /sys/block/sdb/stat)*512/1000000
        notify-send -t 5 $DataWrittenUSB'MB written to USB...'
        sleep 5
    done

notify-send "Copy to USB has finished/not started."
As you can see, it displays how much data was written to the stick and if you intend to try it, you should first check what is your USB's device name in /sys/log as it may not be sdb as is on my machine. One other thing, you need to know the sector size of your USB stick and if it's not 512 bytes, change that accordingly here:

Code: Select all

/sys/block/sdb/stat)*<type sector size here>/1024/1024
Didn't know how to display continually changing value in Zenity (if it's possible) so I've got the built-in notify-send to take it on (and it doesn't steel the focus). Creates a bundle of notifications, in the process, but what's perfect these days?.. :lol:

Cheers! :D
Last edited by LockBot on Wed Dec 28, 2022 7:16 am, edited 2 times in total.
Reason: Topic automatically closed 6 months after creation. New replies are no longer allowed.
ColdBootII

Re: A way to display file copy progress when copying to a USB stick

Post by ColdBootII »

Heh, :mrgreen:

I just recalled there is CommandRunner applet for Cinnamon which is perfect. Like Genmon Xfce plugin, it can display stdout coming from a script so, you won't need to clear hundreds of notifications after all. :mrgreen:

A modified version to fit with this applet:

Code: Select all

#!  /bin/bash

if [ $(awk "{ print \$9 }" /sys/block/sdb/stat) -gt 0 ]
    then
        let DataWrittenUSB=$(awk "{ print \$7 }" /sys/block/sdb/stat)*512/1000000
        echo $DataWrittenUSB'MB written...'
 else
        echo "USB idle"
 fi
 
Image

For the life of me, I can't remember what could come in handy for this purpose in KDE's panel but there must be something you'll find out...

Cheers! :D
Last edited by ColdBootII on Mon Oct 16, 2017 10:06 am, edited 1 time in total.
User avatar
catweazel
Level 19
Level 19
Posts: 9763
Joined: Fri Oct 12, 2012 9:44 pm
Location: Australian Antarctic Territory

Re: A way to display file copy progress when copying to a USB stick

Post by catweazel »

ColdBootII wrote:For the life of me, I can't remember what could come in handy for this purpose in KDE's panel but there must be something you'll find out...
Some twit b0rked KDE's ability to show progress dialogs. The 'fixes' posted online don't work any more. GRRRR!
"There is, ultimately, only one truth -- cogito, ergo sum -- everything else is an assumption." - Me, my swansong.
ColdBootII

Re: A way to display file copy progress when copying to a USB stick

Post by ColdBootII »

I've made a typical mistake (for me) dividing with 1024 when 1000 bytes is a kilobyte - corrected... :mrgreen:

With xclip installed, it should be possible to find and display the total size of files to be copied next to copy progress. Just a wee bit more work to deal with long file names containing awkward characters and spaces.
catweazel wrote:
Some twit b0rked KDE's ability to show progress dialogs. The 'fixes' posted online don't work any more. GRRRR!
They'll gonna fix it, don't fret it. :mrgreen:
User avatar
BG405
Level 9
Level 9
Posts: 2501
Joined: Fri Mar 11, 2016 3:09 pm
Location: England

Re: A way to display file copy progress when copying to a USB stick

Post by BG405 »

catweazel wrote:Some twit b0rked KDE's ability to show progress dialogs.
Like this, you mean?
Screenshot_20171016_160800_Cropped.jpeg
Not very clear I know, I can't find the transparency settings :oops: I'll have to resize that thin progress bar as well.
Dell Inspiron 1525 - LM17.3 CE 64-------------------Lenovo T440 - Manjaro KDE with Mint VMs
Toshiba NB250 - Manjaro KDE------------------------Acer Aspire One D255E - LM21.3 Xfce
Acer Aspire E11 ES1-111M - LM18.2 KDE 64 ----Two ROMS don't make a WRITE
Misko_2083

Re: A way to display file copy progress when copying to a USB stick

Post by Misko_2083 »

ColdBootII wrote: I just recalled there is CommandRunner applet for Cinnamon which is perfect. Like Genmon Xfce plugin, it can display stdout coming from a script so, you won't need to clear hundreds of notifications after all. :mrgreen:

A modified version to fit with this applet:

Code: Select all

#!  /bin/bash

if [ $(awk "{ print \$9 }" /sys/block/sdb/stat) -gt 0 ]
    then
        let DataWrittenUSB=$(awk "{ print \$7 }" /sys/block/sdb/stat)*512/1000000
        echo $DataWrittenUSB'MB written...'
 else
        echo "USB idle"
 fi
 
To tell you the truth, I have no idea why would you monitor when something is copied to usb when there is a progress bar in most file managers.
Perhaps it can be used for conky or semething else.
Anyway, I decided to give it a try with Xfce's "Generic monitor" plugin and upgraded your script to watch for all usb drives. I've set the update period to 2 seconds.

Code: Select all

#!/bin/bash

if ls -l /dev/disk/by-path/*usb* &>/dev/null; then

	USB_DET=($(ls -l /dev/disk/by-path/*usb* | grep -v "part" | awk '{print $NF}' | awk -F "/" '{print $NF}' | sort))

	for usb in "${USB_DET[@]}"
	do
		if [ $(awk "{ print \$9 }" /sys/block/$usb/stat) -gt 0 ]
		then
			sector_size=$(cat /sys/block/$usb/queue/hw_sector_size)

			DATA_WRITTEN=$(awk '{load = $7*sector/1000/1000
						printf "%.1f", load}'  sector="$sector_size" /sys/block/$usb/stat)
        		printf "%s: %s%s " "$usb" "$DATA_WRITTEN" "MiB"
		else
			printf "%s: %s " "$usb" "idle"
		fi
	done
else
	echo "None"
fi
If no usb drive is plugged in it wil write out "None"
Image
You can insert multiple usb drives.
Image
When you copy to usb drive.
Image
It writes out with the floating point which is defined in the awk command printf "%.1f"
Not sure how it will bahave in CommandRunner.

This was just for fun. When coping files, I usually use the pv command to displays a progress bar.
ColdBootII

Re: A way to display file copy progress when copying to a USB stick

Post by ColdBootII »

Misko_2083 wrote: To tell you the truth, I have no idea why would you monitor when something is copied to usb when there is a progress bar in most file managers.
Perhaps it can be used for conky or semething else.
Anyway, I decided to give it a try with Xfce's "Generic monitor" plugin and upgraded your script to watch for all usb drives. I've set the update period to 2 seconds.

Code: Select all

#!/bin/bash

if ls -l /dev/disk/by-path/*usb* &>/dev/null; then

	USB_DET=($(ls -l /dev/disk/by-path/*usb* | grep -v "part" | awk '{print $NF}' | awk -F "/" '{print $NF}' | sort))

	for usb in "${USB_DET[@]}"
	do
		if [ $(awk "{ print \$9 }" /sys/block/$usb/stat) -gt 0 ]
		then
			sector_size=$(cat /sys/block/$usb/queue/hw_sector_size)

			DATA_WRITTEN=$(awk '{load = $7*sector/1000/1000
						printf "%.1f", load}'  sector="$sector_size" /sys/block/$usb/stat)
        		printf "%s: %s%s " "$usb" "$DATA_WRITTEN" "MiB"
		else
			printf "%s: %s " "$usb" "idle"
		fi
	done
else
	echo "None"
fi
If no usb drive is plugged in it wil write out "None"
Image
You can insert multiple usb drives.
Image
When you copy to usb drive.
Image
It writes out with the floating point which is defined in the awk command printf "%.1f"
Not sure how it will bahave in CommandRunner.

This was just for fun. When coping files, I usually use the pv command to displays a progress bar.
Thanks Mishko_2083, nicely done! :D

I use Nemo for most any file-managerial purposes and I wanted this because the file-copy progress to USB, as displayed in the dialog, is inaccurate. It seems to be showing only the progress of copying file(s) to memory cache(which is very fast) and then stalls, for longer or shorter periods depending on sizes of file(s) copied, because it doesn't display actual syncing progress at all. pv doesn't cut it for me either nor any other file manager I've tried using in Cinnamon. I have no idea why is that...

Anyway, I have given up. This is as far as this script can go from my POV since I don't know how to pass the info about selected files to the script, when they are dragged-and-dropped to USB in Nemo. Apparently, it is not a copy operation that involves the use of clipboard and xclip shows nothing of use.

Cheers! :D
User avatar
BG405
Level 9
Level 9
Posts: 2501
Joined: Fri Mar 11, 2016 3:09 pm
Location: England

Re: A way to display file copy progress when copying to a USB stick

Post by BG405 »

ColdBootII wrote:doesn't display actual syncing progress
I see what you mean. I have noticed the same thing when writing floppy images to actual floppy disks (the big 5¼-inch ones which really are floppy lol) - sometimes dd reports it's copied and then the disk drive comes to life a second or so later .. these USB sticks probably do have a small buffer as well.

Not sure of a practical way round this without interrupting data writes at timed intervals for a check, which would be a lot slower.
Dell Inspiron 1525 - LM17.3 CE 64-------------------Lenovo T440 - Manjaro KDE with Mint VMs
Toshiba NB250 - Manjaro KDE------------------------Acer Aspire One D255E - LM21.3 Xfce
Acer Aspire E11 ES1-111M - LM18.2 KDE 64 ----Two ROMS don't make a WRITE
ColdBootII

Re: A way to display file copy progress when copying to a USB stick

Post by ColdBootII »

(I have inadvertently destroyed this post, sorry :oops: :mrgreen: )
Last edited by ColdBootII on Wed Oct 18, 2017 5:46 pm, edited 3 times in total.
User avatar
catweazel
Level 19
Level 19
Posts: 9763
Joined: Fri Oct 12, 2012 9:44 pm
Location: Australian Antarctic Territory

Re: A way to display file copy progress when copying to a USB stick

Post by catweazel »

BG405 wrote:
catweazel wrote:Some twit b0rked KDE's ability to show progress dialogs.
Like this, you mean?Screenshot_20171016_160800_Cropped.jpeg
Not very clear I know
Yes, that. The alleged fix doesn't work.
"There is, ultimately, only one truth -- cogito, ergo sum -- everything else is an assumption." - Me, my swansong.
ColdBootII

Re: A way to display file copy progress when copying to a USB stick

Post by ColdBootII »

However, if it happens to be writing to a NTFS, it closes much too soon and I can only probe, now and then, if transfers are over by attempting to safely remove it. Haven't had the time to see if the script will fare any better.
As an illustration of the above, quoted from my previous post in this thread, I just did a copy to a NTFS formated USB 2.0 flash stick. I attempted to copy 4 files of 2.8GB total size and the progress dialog went away before 50% was really written to the stick...:

Image

On the other hand, the script went on and showed that USB is idle exactly indicating when it can be removed. For those interested, reference for the write sectors field in stat is here:

https://www.kernel.org/doc/Documentation/block/stat.txt

Cheers! :D
Locked

Return to “Scripts & Bash”