Nvidia dual GPU overclocking and monitor management

Please post suggestions for improvement of Cinnamon here
https://github.com/linuxmint/Cinnamon
Post Reply
digibill
Level 1
Level 1
Posts: 12
Joined: Tue Jan 26, 2016 6:53 am

Nvidia dual GPU overclocking and monitor management

Post by digibill » Mon Dec 04, 2017 9:29 am

I am opening this new thread, although my previous one did not conclude somewhere, because my setup now has changed, and so has changed what I want to do.

My system in short: AMD Ryzen 5 1600, 16GB DDR4 RAM, Nvidia GTX 1070 8GB on pci-e #1 bus, Nvidia GTX 1060 8GB on pci-e #2 bus, a single Dell U2717 Monitor (now running on 1070)

Complete info:

Code: Select all

System:    Host: vasilis-desktop Kernel: 4.10.0-40-generic x86_64 (64 bit gcc: 5.4.0)
           Desktop: Cinnamon 3.6.6 (Gtk 3.18.9-1ubuntu3.3) dm: mdm Distro: Linux Mint 18.3 Sylvia
Machine:   Mobo: ASUSTeK model: PRIME B350-PLUS v: Rev X.0x
           Bios: American Megatrends v: 3203 date: 11/09/2017
CPU:       Hexa core AMD Ryzen 5 1600 Six-Core (-HT-MCP-) cache: 3072 KB
           flags: (lm nx sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 svm) bmips: 38326
           clock speeds: min/max: 1550/3200 MHz 1: 1550 MHz 2: 1550 MHz 3: 1550 MHz 4: 1550 MHz
           5: 1550 MHz 6: 1550 MHz 7: 1550 MHz 8: 1550 MHz 9: 1550 MHz 10: 2800 MHz 11: 1550 MHz
           12: 1550 MHz
Graphics:  Card-1: NVIDIA Device 1c03 bus-ID: 06:00.0 chip-ID: 10de:1c03
           Card-2: NVIDIA Device 1b81 bus-ID: 07:00.0 chip-ID: 10de:1b81
           Display Server: X.Org 1.18.4 driver: nvidia Resolution: 2560x1440@59.95hz
           GLX Renderer: GeForce GTX 1070/PCIe/SSE2
           GLX Version: 4.5.0 NVIDIA 384.90 Direct Rendering: Yes
Audio:     Card-1 Advanced Micro Devices [AMD] Device 1457
           driver: snd_hda_intel bus-ID: 09:00.3 chip-ID: 1022:1457
           Card-2 NVIDIA Device 10f0 driver: snd_hda_intel bus-ID: 07:00.1 chip-ID: 10de:10f0
           Card-3 NVIDIA Device 10f1 driver: snd_hda_intel bus-ID: 06:00.1 chip-ID: 10de:10f1
           Sound: Advanced Linux Sound Architecture v: k4.10.0-40-generic
Network:   Card: Realtek RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller
           driver: r8169 v: 2.3LK-NAPI port: e000 bus-ID: 03:00.0 chip-ID: 10ec:8168
           IF: enp3s0 state: up speed: 100 Mbps duplex: full mac: <filter>
Drives:    HDD Total Size: 1120.2GB (50.1% used)
           ID-1: /dev/sda model: KINGSTON_SA400S3 size: 120.0GB serial: 50026B72730295D0
           ID-2: /dev/sdb model: WDC_WD5000AAKS size: 500.1GB serial: WD-WCASY2913267
           ID-3: /dev/sdg model: WDC_WD5000AAKS size: 500.1GB serial: WD-WCAWF0914984
Partition: ID-1: / size: 108G used: 37G (37%) fs: ext4 dev: /dev/sda5
           ID-2: /home size: 459G used: 351G (81%) fs: ext4 dev: /dev/sdg1
           ID-3: swap-1 size: 3.00GB used: 0.00GB (0%) fs: swap dev: /dev/sda1
RAID:      System: supported: N/A
           No RAID devices: /proc/mdstat, md_mod kernel module present
           Unused Devices: none
Sensors:   System Temperatures: cpu: N/A mobo: N/A gpu: 0.0:63C
           Fan Speeds (in rpm): cpu: 0
Repos:     Active apt sources in file: /etc/apt/sources.list
           deb http: //download.ebz.epson.net/dsc/op/stable/debian/ lsb3.2 main
           Active apt sources in file: /etc/apt/sources.list.d/atareao-atareao-xenial.list
           deb http: //ppa.launchpad.net/atareao/atareao/ubuntu xenial main
           deb-src http: //ppa.launchpad.net/atareao/atareao/ubuntu xenial main
           Active apt sources in file: /etc/apt/sources.list.d/atareao-nemo-extensions-xenial.list
           deb http: //ppa.launchpad.net/atareao/nemo-extensions/ubuntu xenial main
           deb-src http: //ppa.launchpad.net/atareao/nemo-extensions/ubuntu xenial main
           Active apt sources in file: /etc/apt/sources.list.d/clipgrab-team-ppa-xenial.list
           deb http: //ppa.launchpad.net/clipgrab-team/ppa/ubuntu xenial main
           deb-src http: //ppa.launchpad.net/clipgrab-team/ppa/ubuntu xenial main
           Active apt sources in file: /etc/apt/sources.list.d/cuda.list
           deb http: //developer.download.nvidia.com/compute/cuda/repos/ubuntu1604/x86_64 /
           Active apt sources in file: /etc/apt/sources.list.d/ethereum-ethereum-xenial.list
           deb http: //ppa.launchpad.net/ethereum/ethereum/ubuntu xenial main
           deb-src http: //ppa.launchpad.net/ethereum/ethereum/ubuntu xenial main
           Active apt sources in file: /etc/apt/sources.list.d/google-chrome.list
           deb [arch=amd64] http: //dl.google.com/linux/chrome/deb/ stable main
           Active apt sources in file: /etc/apt/sources.list.d/maidsafe.list
           deb http: //apt.maidsafe.net/repos/apt/debian jessie main
           deb http: //apt.maidsafe.net/repos/apt/debian wheezy main
           Active apt sources in file: /etc/apt/sources.list.d/megasync.list
           deb https: //mega.nz/linux/MEGAsync/xUbuntu_16.04/ ./
           Active apt sources in file: /etc/apt/sources.list.d/nvidiux-nvidiux-xenial.list
           deb http: //ppa.launchpad.net/nvidiux/nvidiux/ubuntu xenial main
           deb-src http: //ppa.launchpad.net/nvidiux/nvidiux/ubuntu xenial main
           Active apt sources in file: /etc/apt/sources.list.d/official-package-repositories.list
           deb http: //ftp.otenet.gr/linux/linuxmint-packages sylvia main upstream import backport
           deb http: //ftp.cc.uoc.gr/mirrors/linux/ubuntu/packages xenial main restricted universe multiverse
           deb http: //ftp.cc.uoc.gr/mirrors/linux/ubuntu/packages xenial-updates main restricted universe multiverse
           deb http: //ftp.cc.uoc.gr/mirrors/linux/ubuntu/packages xenial-backports main restricted universe multiverse
           deb http: //security.ubuntu.com/ubuntu/ xenial-security main restricted universe multiverse
           deb http: //archive.canonical.com/ubuntu/ xenial partner
           Active apt sources in file: /etc/apt/sources.list.d/openshot_developers-ppa-xenial.list
           deb http: //ppa.launchpad.net/openshot.developers/ppa/ubuntu xenial main
           deb-src http: //ppa.launchpad.net/openshot.developers/ppa/ubuntu xenial main
           Active apt sources in file: /etc/apt/sources.list.d/pmjdebruijn-darktable-release-xenial.list
           deb http: //ppa.launchpad.net/pmjdebruijn/darktable-release/ubuntu xenial main
           deb-src http: //ppa.launchpad.net/pmjdebruijn/darktable-release/ubuntu xenial main
           Active apt sources in file: /etc/apt/sources.list.d/saiarcot895-flightgear-xenial.list
           deb http: //ppa.launchpad.net/saiarcot895/flightgear/ubuntu xenial main
           deb-src http: //ppa.launchpad.net/saiarcot895/flightgear/ubuntu xenial main
           Active apt sources in file: /etc/apt/sources.list.d/spotify.list
           deb http: //repository.spotify.com stable non-free
           Active apt sources in file: /etc/apt/sources.list.d/thomas_tsai-ubuntu-tuxboot-xenial.list
           deb http: //ppa.launchpad.net/thomas.tsai/ubuntu-tuxboot/ubuntu xenial main
           deb-src http: //ppa.launchpad.net/thomas.tsai/ubuntu-tuxboot/ubuntu xenial main
Info:      Processes: 349 Uptime: 31 min Memory: 2869.6/16044.2MB
           Init: systemd v: 229 runlevel: 5 default: 2 Gcc sys: 5.4.0
           Client: Unknown python2.7 client inxi: 2.2.35
This PC is running 24/7 minning Ethereum with both GPUs and is also used for varius other tasks.

What I want to do:
  • Overclock both GPUs
  • Set my monitor to run on the 1060, which is located on the second (#2) pci-e slot of my m/b (Asus Prime B350 plus)
What I have tried:

First of all, a physical swap of the GPUs on the motherboard slots is out of the question: due to their physical dimensions, the 1070 (if set to the lower pci-e slot) interrupts the airflow to the 1060 to the point that I have a great rise in 1060's temperature (not critical, but I do not want to gumble on this). So, the hardware layout has to stay as it is now.

Now, for the overclocking: an you may know, to enable overclocking on nvidia GPUs, one has to add the "coolbits 28" option on the x server entry of xorg.conf file. When it comes to multiple GPUs and / or monitors, the solution is to add virtual X servers for each of the cards (even those not connected to a monitor) to the xorg.conf, creating a server layout section with all of your real and virtual screens. For example:

Code: Select all

Section "ServerLayout"  
    Identifier    "Layout0"     
#   Our real monitor
    Screen 0      "Screen0" 0 0     
#   Our virtual monitors
    Screen 1      "Screen1"     
    Screen 2      "Screen2"
#    ....
    Screen 3      "Screen3"     
    InputDevice   "Keyboard0" "CoreKeyboard"
    InputDevice   "Mouse0"    "CorePointer" 
EndSection
This can be done either by hand (by adding the relevant lines in the xorg.cong manually), or by this command:

Code: Select all

sudo nvidia-xconfig -a --cool-bits=28 --allow-empty-initial-configuration
.

The above command creates a xorg.conf with x server and virtual monitors, but gives me the following error:

Code: Select all

sudo nvidia-xconfig -a --cool-bits=28 --allow-empty-initial-configuration

WARNING: Unable to locate/open X configuration file.

Package xorg-server was not found in the pkg-config search path.
Perhaps you should add the directory containing `xorg-server.pc'
to the PKG_CONFIG_PATH environment variable
No package 'xorg-server' found
Option "AllowEmptyInitialConfiguration" "True" added to Screen "Screen0".
Option "AllowEmptyInitialConfiguration" "True" added to Screen "Screen1".
New X configuration file written to '/etc/X11/xorg.conf'
(There are various sources on the internet that promote the above solution, for example here)

On my occasion, none of the above worked. No matter what changes I do to xorg.conf, coolbits enables overclocking only at the 1st GPU (1070, 1st slot). This is what I get on nvidia-settings:

Image

I have also tried setting the frequencies on the second GPU (1060) via terminal:

Code: Select all

nvidia-settings -a [gpu:0]/GPUMemoryTransferRateOffset[3]=1000
but this does not work, although I do not get any errors.

So, at the moment I am able to overclock only the 1st GPU (1070, pci-e slot #1) via nvidia-settings. Also, the 1060 (#2 pci-e slot) is not able to drive my monitor (system does not recognize the monitor when plugged in this card. However, I am able to drive multiple monitors when connected on the 1st GPU (1070)).

I have checked all BIOS settings (in case there is an option in the BIOS preventing the second pci-e slot from working) but I haven't find anything relevant. Moreover, the 2nd GPU (1060) does work with the miner (so, it is not disabled by the BIOS).
I have also tried to completely delete all xorg.conf files from /etc/X11/ and the nvidia-settings still allow me to change the frequencies on the first GPU (1070), as if it gets a "coolbits 28" option from somewhere else!!

So, I have come to a dead end.
Without overclocking I only partial utilize my GPUs for mining (just for reference, the combination of 1060 + 1070 can easily get 55-58 MH/s, and now I am getting 45-49 MH/s max)
Also, I want to use my monitor on the 1060 so as when I want to do various demanding tasks (eg photo processing) to stop the miner from this card (which is the weaker) and not to lose the greater hash rates of the 1070.

All the above are so crucial to me that, in case I am unable to resolve them, I would consider switching distro after many years of exclusive use of Linux Mint Cinammon....

Thanks in advance for every help effort.

Paul Liu
Level 1
Level 1
Posts: 1
Joined: Fri Dec 22, 2017 5:59 am

Re: Nvidia dual GPU overclocking and monitor management

Post by Paul Liu » Fri Dec 22, 2017 6:37 am

If you have xorg.conf with coo-bits generated, you should be able to OC both GPU cards. I am using nvidia-smi and OC for both my GTX 1070 works for me.

I used to have one GTX 1060 and GTX 1070, and that's my OC bash file.

Code: Select all

cat ~/NvidiaOCSettings.sh
#!/bin/bash
nvidia-settings -a '[gpu:0]/GPUMemoryTransferRateOffset[3]=1090'
nvidia-settings -a '[gpu:0]/GPUGraphicsClockOffset[3]=-50'
sudo nvidia-smi -pm 1
sudo nvidia-smi -i 0 -pl 130 

nvidia-settings -a '[gpu:1]/GPUMemoryTransferRateOffset[3]=1150'
nvidia-settings -a '[gpu:1]/GPUGraphicsClockOffset[3]=-50'
sudo nvidia-smi -pm 1
sudo nvidia-smi -i 1 -pl 115 
However, my problem is the Cinnamon will crash if I tried to use the generated xorg.conf using the method you've used. Let me know if your Cinnamon works fine for you or not.

digibill
Level 1
Level 1
Posts: 12
Joined: Tue Jan 26, 2016 6:53 am

Re: Nvidia dual GPU overclocking and monitor management

Post by digibill » Fri Dec 29, 2017 6:41 pm

I've tried completely deleting the xorg.conf file, and STILL I get the same behavior: I can overclock GPU #1, but not GPU #2.
It seems that nvidia-settings does read another xorg.conf file?? But where is it?!!?

socketrocket889
Level 1
Level 1
Posts: 1
Joined: Tue Jan 16, 2018 9:43 pm

Re: Nvidia dual GPU overclocking and monitor management

Post by socketrocket889 » Tue Jan 16, 2018 10:09 pm

i was wondering if anyone knew a gu miner for linux mint and where i could find instructions to install and run it? im currently using minergate on my cpu but ive heard its bad on your cpu so i wanna switch to gu, i have an amd rs780 {radeon hd 3200} i dont mine heavliy i just do it in the backround of everything else any suggestions?

digibill
Level 1
Level 1
Posts: 12
Joined: Tue Jan 26, 2016 6:53 am

Re: Nvidia dual GPU overclocking and monitor management

Post by digibill » Sun Jan 28, 2018 7:13 am

Paul Liu wrote:However, my problem is the Cinnamon will crash if I tried to use the generated xorg.conf using the method you've used. Let me know if your Cinnamon works fine for you or not.
Same situation here.
After a clean Linux Mint install, I was able to generate a xorg.conf that enables overclocking, but Cinnamon crashed on each startup.

Finally, I was forced to switch to Ubuntu 17.10. This is not something I am happy with, as I love Linux Mint and I really prefer Cinnamon over other DEs, but overclocking all my GPU is vital for my mining....

Citizen229
Level 5
Level 5
Posts: 736
Joined: Fri Nov 04, 2016 12:09 pm
Location: NW Ohio

Re: Nvidia dual GPU overclocking and monitor management

Post by Citizen229 » Sun Jan 28, 2018 7:24 am

My experience with coolbits (GUI) has been the cards run at the boost rated speeds by default in linux. Coolbits can tweak for a bit more but in general not worth it. Then again i Fold and not Mining.

You would more than likely get the same output by going headless versus stressing hardware. ( mining hardware in a corner running without a monitor, managed remotely). This assumes your mining software has network usage in it.

Also Cinnamon is heavy on the eye candy, this takes GPU cycles for display, not the best DE for folding/mining.

Another option(not the best) change ryzen for an APU set up for screen rendering from APU and both gpu slots can run undisturbed by mining software.

/salute for having a 365 watt draw. Wish we had you!
Folding@home Project
Team Linux Mint-76140
PM for info on how you can help. Or visit viewtopic.php?f=58&t=243792
Seeking GPU's, i5/7/9's , Ryzens.

digibill
Level 1
Level 1
Posts: 12
Joined: Tue Jan 26, 2016 6:53 am

Re: Nvidia dual GPU overclocking and monitor management

Post by digibill » Sun Jan 28, 2018 2:31 pm

Citizen229 wrote:You would more than likely get the same output by going headless versus stressing hardware. ( mining hardware in a corner running without a monitor, managed remotely). This assumes your mining software has network usage in it.
I also have a second pc, setup as ethereum mining rig with 4 1070 GPUs. This runs headless and the difference in hashrates is notable with and without overclocking.
Overclocking goes together with underpowering. So, "coolbits" is essential if you want to achieve maximum hashrates with minimum power consumption: my 1070s run overclocked at +796 MHz and underpowered to 115W (instead of 150 - 180 that is theri normal). With this configuration I get ~30MH instead of 26-27 out-of-the-box.
Also Cinnamon is heavy on the eye candy, this takes GPU cycles for display, not the best DE for folding/mining.
My experience with Cinnamon is that it is notably quicker in responce in comparison to GNOME 3, even when the GPU that drives the monitor is utilized 100% from the miner. OK, I would have a far easier DE such as LMDE, but on this machine I also do other thing apart from mining...

Post Reply

Return to “Cinnamon”