Page 1 of 1

System Hung while copying large file

Posted: Wed Feb 28, 2018 2:42 am
by tszkitc
Hi all,

I am using linuxmint 18.3 with a swap partition of 8gb. 8gb ram as well.

When I copy and paste a large file of 9gb in size. The system hung up.

May i know what is the problem? Could it be the swap partition is too small?

Thank you.

Re: System Hung while copying large file

Posted: Wed Feb 28, 2018 3:26 am
by Mute Ant
Not swap, no. A default Mint kernel is set to use a maximum of 10% RAM as a disk-write buffer...
cat /proc/sys/vm/dirty_background_ratio -----> 10 -----> 800MB

If the write-speed of the destination drive is lower than the read-speed of the source, and it often is, the process will appear to hang when that buffer is full. No problem with small files, lumpy with large files.

You can't speed-up the destination drive, but you can smooth-out the progress report by reducing the buffer size. Something similar to the drive's internal RAM, I use 10MB with no problems...
sudo sysctl vm.dirty_bytes=10000000
...which stays in effect until the next boot.

Re: System Hung while copying large file

Posted: Wed Feb 28, 2018 12:04 pm
by Flemur
tszkitc wrote:
Wed Feb 28, 2018 2:42 am
When I copy and paste a large file of 9gb in size. The system hung up.
Are you copying with Cinnamon and nemo? To or from a USB gizmo?
If so, try "cp" in a terminal, or the simple "xfe" file-browser.

Re: System Hung while copying large file

Posted: Thu Mar 08, 2018 7:15 am
by tszkitc
Anyway to permanently solve this problem?

How come windows don't have this kind of problem?

Thanks.

Re: System Hung while copying large file

Posted: Thu Mar 08, 2018 8:18 am
by Mute Ant
"How come Windows don't have this kind of problem?" It does, exactly the same problem. Microsoft have no more control over what you connect than Linux. Their solution (last time I looked) was to de-buffer writing to incompetent storage and simply report what's happening... Image
Linux can do the same with a 'synchronous mount' if you want to try it.

Re: System Hung while copying large file

Posted: Thu Mar 08, 2018 9:30 am
by JoeFootball
tszkitc wrote:Anyway to permanently solve this problem?
Use the appropriate tools for the job at hand. :) To me, copying a 9 GB file would be an unusual situation, and one that would dictate special handling, as detailed above.

Joe

Re: System Hung while copying large file

Posted: Thu Mar 08, 2018 10:02 am
by Flemur
tszkitc wrote:
Thu Mar 08, 2018 7:15 am
Anyway to permanently solve this problem?
Why, yes, yes there is!

Are you copying with Cinnamon and nemo? To or from a USB gizmo?
If so, try "cp" in a terminal, or the simple "xfe" file-browser.

Re: System Hung while copying large file

Posted: Thu Mar 08, 2018 8:17 pm
by tszkitc
So when it hung, I should just wait or reboot?

How to use CP or Xfe by the way?

Thanks.

Re: System Hung while copying large file

Posted: Thu Mar 08, 2018 8:46 pm
by tszkitc
By the way, I was trying to copy a 9GB file from /home to the other partition.

Only one physical HDD.

Thanks.

Re: System Hung while copying large file

Posted: Thu Mar 08, 2018 8:48 pm
by Flemur
tszkitc wrote:
Thu Mar 08, 2018 8:17 pm
So when it hung, I should just wait or reboot?
How to use CP or Xfe by the way?
Thanks.
To restart it -
https://lifehacker.com/298891/gently-re ... zen-system
1. Hold down the Alt and SysRq (Print Screen) keys.
2. While holding those down, type the following in order. Nothing will appear to happen until the last letter is pressed: REISUB
3. Watch your computer reboot magically.

You use "cp" in a terminal. "xfe" is another file browser which you can install.

Re: System Hung while copying large file

Posted: Thu Mar 08, 2018 9:55 pm
by Mute Ant
"Copy a 9GB file from /home to the other partition, only one physical HDD." I get 1GB/minute doing this, using the Thunar file-manager with drag-and-drop. You almost certainly have a better HDD than the one I am using[*], so if your system stays unresponsive for more than 9 minutes, it's the fault of the software you are using, not the hardware. Strictly speaking, it's not a Linux problem either, Linux is just the kernel.

[*] Right now my OS is on a ST380011A circa 2003 80GB 7200rpm PATA. That can manage 50MB/s write-only, 50MB/s read-only and 16MB/s doing mixed read-write duplicating the 9GB file.

Re: System Hung while copying large file

Posted: Thu Mar 08, 2018 10:15 pm
by Mattyboy
tszkitc wrote:
Thu Mar 08, 2018 8:17 pm
How to use CP
https://www.computerhope.com/unix/ucp.htm
Easier than it looks. You just need to learn the source and destination paths.
Tip use pwd "print working Directory"
And the tab key to auto fill.
The -R "Copy directories recursively" is probably the most useful when dealing with a directory full of files.

That will make sense eventually ;)

Re: System Hung while copying large file

Posted: Fri Mar 09, 2018 8:39 am
by JoeFootball
tszkitc wrote:How to use CP or Xfe by the way?

Code: Select all

sudo apt install xfe
Joe