System Hung while copying large file

All Gurus once were Newbies
Forum rules
There are no such things as "stupid" questions. However if you think your question is a bit stupid, then this is the right place for you to post it. Please stick to easy to-the-point questions that you feel people can answer fast. For long and complicated questions prefer the other forums within the support section.
Before you post please read how to get help
Post Reply
tszkitc
Level 1
Level 1
Posts: 9
Joined: Sun Jan 08, 2017 9:17 pm

System Hung while copying large file

Post by tszkitc » Wed Feb 28, 2018 2:42 am

Hi all,

I am using linuxmint 18.3 with a swap partition of 8gb. 8gb ram as well.

When I copy and paste a large file of 9gb in size. The system hung up.

May i know what is the problem? Could it be the swap partition is too small?

Thank you.

Mute Ant
Level 13
Level 13
Posts: 4951
Joined: Tue Sep 03, 2013 7:45 pm
Location: Norfolk UK

Re: System Hung while copying large file

Post by Mute Ant » Wed Feb 28, 2018 3:26 am

Not swap, no. A default Mint kernel is set to use a maximum of 10% RAM as a disk-write buffer...
cat /proc/sys/vm/dirty_background_ratio -----> 10 -----> 800MB

If the write-speed of the destination drive is lower than the read-speed of the source, and it often is, the process will appear to hang when that buffer is full. No problem with small files, lumpy with large files.

You can't speed-up the destination drive, but you can smooth-out the progress report by reducing the buffer size. Something similar to the drive's internal RAM, I use 10MB with no problems...
sudo sysctl vm.dirty_bytes=10000000
...which stays in effect until the next boot.
Four Sprung Duck Technique!

User avatar
Flemur
Level 16
Level 16
Posts: 6290
Joined: Mon Aug 20, 2012 9:41 pm
Location: Potemkin Village

Re: System Hung while copying large file

Post by Flemur » Wed Feb 28, 2018 12:04 pm

tszkitc wrote:
Wed Feb 28, 2018 2:42 am
When I copy and paste a large file of 9gb in size. The system hung up.
Are you copying with Cinnamon and nemo? To or from a USB gizmo?
If so, try "cp" in a terminal, or the simple "xfe" file-browser.
Please edit your original post title to include [SOLVED] if/when it is solved!
Your data and OS are backed up....right?
Mint 18.3 Xfce/fluxbox/pulse-less
Xubuntu 17.10/fluxbox/pulse-less

tszkitc
Level 1
Level 1
Posts: 9
Joined: Sun Jan 08, 2017 9:17 pm

Re: System Hung while copying large file

Post by tszkitc » Thu Mar 08, 2018 7:15 am

Anyway to permanently solve this problem?

How come windows don't have this kind of problem?

Thanks.

Mute Ant
Level 13
Level 13
Posts: 4951
Joined: Tue Sep 03, 2013 7:45 pm
Location: Norfolk UK

Re: System Hung while copying large file

Post by Mute Ant » Thu Mar 08, 2018 8:18 am

"How come Windows don't have this kind of problem?" It does, exactly the same problem. Microsoft have no more control over what you connect than Linux. Their solution (last time I looked) was to de-buffer writing to incompetent storage and simply report what's happening... Image
Linux can do the same with a 'synchronous mount' if you want to try it.
Four Sprung Duck Technique!

User avatar
JoeFootball
Level 7
Level 7
Posts: 1507
Joined: Tue Nov 24, 2009 1:52 pm
Location: /home/usa/mn/minneapolis/joe

Re: System Hung while copying large file

Post by JoeFootball » Thu Mar 08, 2018 9:30 am

tszkitc wrote:Anyway to permanently solve this problem?
Use the appropriate tools for the job at hand. :) To me, copying a 9 GB file would be an unusual situation, and one that would dictate special handling, as detailed above.

Joe

User avatar
Flemur
Level 16
Level 16
Posts: 6290
Joined: Mon Aug 20, 2012 9:41 pm
Location: Potemkin Village

Re: System Hung while copying large file

Post by Flemur » Thu Mar 08, 2018 10:02 am

tszkitc wrote:
Thu Mar 08, 2018 7:15 am
Anyway to permanently solve this problem?
Why, yes, yes there is!

Are you copying with Cinnamon and nemo? To or from a USB gizmo?
If so, try "cp" in a terminal, or the simple "xfe" file-browser.
Please edit your original post title to include [SOLVED] if/when it is solved!
Your data and OS are backed up....right?
Mint 18.3 Xfce/fluxbox/pulse-less
Xubuntu 17.10/fluxbox/pulse-less

tszkitc
Level 1
Level 1
Posts: 9
Joined: Sun Jan 08, 2017 9:17 pm

Re: System Hung while copying large file

Post by tszkitc » Thu Mar 08, 2018 8:17 pm

So when it hung, I should just wait or reboot?

How to use CP or Xfe by the way?

Thanks.

tszkitc
Level 1
Level 1
Posts: 9
Joined: Sun Jan 08, 2017 9:17 pm

Re: System Hung while copying large file

Post by tszkitc » Thu Mar 08, 2018 8:46 pm

By the way, I was trying to copy a 9GB file from /home to the other partition.

Only one physical HDD.

Thanks.

User avatar
Flemur
Level 16
Level 16
Posts: 6290
Joined: Mon Aug 20, 2012 9:41 pm
Location: Potemkin Village

Re: System Hung while copying large file

Post by Flemur » Thu Mar 08, 2018 8:48 pm

tszkitc wrote:
Thu Mar 08, 2018 8:17 pm
So when it hung, I should just wait or reboot?
How to use CP or Xfe by the way?
Thanks.
To restart it -
https://lifehacker.com/298891/gently-re ... zen-system
1. Hold down the Alt and SysRq (Print Screen) keys.
2. While holding those down, type the following in order. Nothing will appear to happen until the last letter is pressed: REISUB
3. Watch your computer reboot magically.

You use "cp" in a terminal. "xfe" is another file browser which you can install.
Please edit your original post title to include [SOLVED] if/when it is solved!
Your data and OS are backed up....right?
Mint 18.3 Xfce/fluxbox/pulse-less
Xubuntu 17.10/fluxbox/pulse-less

Mute Ant
Level 13
Level 13
Posts: 4951
Joined: Tue Sep 03, 2013 7:45 pm
Location: Norfolk UK

Re: System Hung while copying large file

Post by Mute Ant » Thu Mar 08, 2018 9:55 pm

"Copy a 9GB file from /home to the other partition, only one physical HDD." I get 1GB/minute doing this, using the Thunar file-manager with drag-and-drop. You almost certainly have a better HDD than the one I am using[*], so if your system stays unresponsive for more than 9 minutes, it's the fault of the software you are using, not the hardware. Strictly speaking, it's not a Linux problem either, Linux is just the kernel.

[*] Right now my OS is on a ST380011A circa 2003 80GB 7200rpm PATA. That can manage 50MB/s write-only, 50MB/s read-only and 16MB/s doing mixed read-write duplicating the 9GB file.
Four Sprung Duck Technique!

Mattyboy
Level 6
Level 6
Posts: 1196
Joined: Thu Mar 26, 2015 2:17 pm

Re: System Hung while copying large file

Post by Mattyboy » Thu Mar 08, 2018 10:15 pm

tszkitc wrote:
Thu Mar 08, 2018 8:17 pm
How to use CP
https://www.computerhope.com/unix/ucp.htm
Easier than it looks. You just need to learn the source and destination paths.
Tip use pwd "print working Directory"
And the tab key to auto fill.
The -R "Copy directories recursively" is probably the most useful when dealing with a directory full of files.

That will make sense eventually ;)

User avatar
JoeFootball
Level 7
Level 7
Posts: 1507
Joined: Tue Nov 24, 2009 1:52 pm
Location: /home/usa/mn/minneapolis/joe

Re: System Hung while copying large file

Post by JoeFootball » Fri Mar 09, 2018 8:39 am

tszkitc wrote:How to use CP or Xfe by the way?

Code: Select all

sudo apt install xfe
Joe

Post Reply

Return to “Newbie Questions”