NVIDIA Driver Not Usable on LDME5

Questions about hardware and drivers
Forum rules
Before you post read how to get help. Topics in this forum are automatically closed 6 months after creation.
User avatar
SMG
Level 25
Level 25
Posts: 31325
Joined: Sun Jul 26, 2020 6:15 pm
Location: USA

Re: NVIDIA Driver Not Usable on LDME5

Post by SMG »

Did you remove the Nvidia drivers from the Debian package before you installed the Nvidia drivers from the website?

Let's try checking Xorg log again. Let us know the url it returns.

Code: Select all

cat /var/log/Xorg.0.log | nc termbin.com 9999
Image
A woman typing on a laptop with LM20.3 Cinnamon.
osmanakar312
Level 2
Level 2
Posts: 61
Joined: Sun Mar 27, 2022 4:47 pm

Re: NVIDIA Driver Not Usable on LDME5

Post by osmanakar312 »

https://termbin.com/t0z74

And yes, i removed debian packages before installation.
User avatar
SMG
Level 25
Level 25
Posts: 31325
Joined: Sun Jul 26, 2020 6:15 pm
Location: USA

Re: NVIDIA Driver Not Usable on LDME5

Post by SMG »

This had the same start as the other one where only the Intel drm device was added before probing the GPUs.

Code: Select all

[     5.432] (II) xfree86: Adding drm device (/dev/dri/card0)
[     5.443] (--) PCI:*(0@0:2:0) 8086:0166:144d:c652 rev 9, Mem @ 0xf1000000/4194304, 0xe0000000/268435456, I/O @ 0x00004000/64, BIOS @ 0x????????/131072
[     5.443] (--) PCI: (1@0:0:0) 10de:1140:144d:c652 rev 161, Mem @ 0xf0000000/16777216, 0xc0000000/268435456, 0xd0000000/33554432, I/O @ 0x00003000/128
However, the Nvidia kernel driver did load first as one would expect in a dual-graphics situation. But, because the drm device for card1 did not load, modesetting (Intel display driver) was first to load and there is no sign of the nvidia display driver (probably because there was nothing for it).

The renderer module disappeared which is why the Intel renderer di not load.

Code: Select all

[     5.593] (II) Loading sub module "glamoregl"
[     5.593] (II) LoadModule: "glamoregl"
[     5.593] (WW) Warning, couldn't open module glamoregl
[     5.593] (EE) modeset: Failed to load module "glamoregl" (module does not exist, 0)
[     5.593] (EE) modeset(0): Failed to load glamor module.
After thinking about it for a bit, I recall a similar incident happening on Ubuntu-based Mint where someone installed the Nvidia-390 driver from the Nvidia website. As explained by roblm in this post:
When installing the Nvidia-390 driver that is downloaded from Nvidia’s website on an Optimus-based laptop, the Nvidia installer will remove that file [/usr/lib/xorg/modules/libglamoregl.so]. Also, libglx.so is removed in /usr/lib/xorg/modules/extensions/.
From another topic where a person did the same thing, the steps to restore use of the Intel renderer would be:
  • Boot to a live session of LMDE5 and copy those files from the install usb to your hard drive.
    • Copy /usr/lib/xorg/modules/libglamoregl.so from your install usb to /usr/lib/xorg/modules on your hard drive.
    • Copy /usr/lib/xorg/modules/extensions/libglx.so from your install usb to /usr/lib/xorg/modules/extensions on your hard drive.
  • After you copy them, you will want to reboot for them to take effect.
Those topics were Ubuntu-based Mint, but I would think the files would be the same with LMDE5. Let us know if that is not the case.

There is also this line. Not sure of the underlying cause, but maybe it's because card1 was not added as a device at the beginning.

Code: Select all

[     5.652] (EE) Failed to initialize GLX extension (Compatible NVIDIA X driver not found)
Card1 was added later (which is too late in the process).

Code: Select all

[     6.331] (II) config/udev: removing GPU device /sys/devices/pci0000:00/0000:00:01.0/0000:01:00.0/drm/card1 /dev/dri/card1
[     6.331] (II) config/udev: Adding drm device (/dev/dri/card1)
[     6.331] (II) xfree86: Adding drm device (/dev/dri/card1)
Let's try checking logs again.

Code: Select all

journalctl -b | grep -i "drm\|nvidia"
That command checks the log of the current boot cycle for anything with drm or Nvidia in it.
Image
A woman typing on a laptop with LM20.3 Cinnamon.
osmanakar312
Level 2
Level 2
Posts: 61
Joined: Sun Mar 27, 2022 4:47 pm

Re: NVIDIA Driver Not Usable on LDME5

Post by osmanakar312 »

I've copied these files and it worked. I recovered Intel GPU again, Cinnamon works fine and HW is available(as direct renderer.)

NVIDIA directly deleted libglamoregl.so and deleted old libglx.so file and put different file called libglx.so.390.147 and created a symbolic connection(i dont know actual name) and named with libglx.so

Original libglx.so in Live USB is 130 kb and NVIDIA's file is 15 MB.

Code: Select all

May 16 02:48:01 crusland kernel: [drm] Initialized i915 1.6.0 20200917 for 0000:00:02.0 on minor 0
May 16 02:48:01 crusland kernel: fbcon: i915drmfb (fb0) is primary device
May 16 02:48:01 crusland kernel: ata1.00: supports DRM functions and may not be fully accessible
May 16 02:48:01 crusland kernel: ata1.00: supports DRM functions and may not be fully accessible
May 16 02:48:01 crusland kernel: i915 0000:00:02.0: [drm] fb0: i915drmfb frame buffer device
May 16 02:48:01 crusland systemd[1]: Starting Load Kernel Module drm...
May 16 02:48:01 crusland systemd[1]: modprobe@drm.service: Succeeded.
May 16 02:48:01 crusland systemd[1]: Finished Load Kernel Module drm.
May 16 02:48:02 crusland kernel: audit: type=1400 audit(1652658481.950:2): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe" pid=372 comm="apparmor_parser"
May 16 02:48:02 crusland kernel: audit: type=1400 audit(1652658481.950:3): apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe//kmod" pid=372 comm="apparmor_parser"
May 16 02:48:01 crusland audit[372]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe" pid=372 comm="apparmor_parser"
May 16 02:48:01 crusland audit[372]: AVC apparmor="STATUS" operation="profile_load" profile="unconfined" name="nvidia_modprobe//kmod" pid=372 comm="apparmor_parser"
May 16 02:48:02 crusland systemd[1]: Starting Load Kernel Module drm...
May 16 02:48:02 crusland systemd[1]: modprobe@drm.service: Succeeded.
May 16 02:48:02 crusland systemd[1]: Finished Load Kernel Module drm.
May 16 02:48:02 crusland kernel: nvidia: loading out-of-tree module taints kernel.
May 16 02:48:02 crusland kernel: nvidia: module license 'NVIDIA' taints kernel.
May 16 02:48:02 crusland kernel: nvidia: module verification failed: signature and/or required key missing - tainting kernel
May 16 02:48:03 crusland kernel: nvidia-nvlink: Nvlink Core is being initialized, major device number 244
May 16 02:48:03 crusland kernel: nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver for UNIX platforms  390.147  Mon Dec 13 13:51:26 UTC 2021
May 16 02:48:03 crusland kernel: [drm] [nvidia-drm] [GPU ID 0x00000100] Loading driver
May 16 02:48:03 crusland kernel: [drm] Initialized nvidia-drm 0.0.0 20160202 for 0000:01:00.0 on minor 1
Here is output which you want. I gave this output after recovering Intel.
User avatar
SMG
Level 25
Level 25
Posts: 31325
Joined: Sun Jul 26, 2020 6:15 pm
Location: USA

Re: NVIDIA Driver Not Usable on LDME5

Post by SMG »

osmanakar312 wrote: Sun May 15, 2022 7:56 pm I've copied these files and it worked. I recovered Intel GPU again, Cinnamon works fine and HW is available(as direct renderer.)
Good to hear.
osmanakar312 wrote: Sun May 15, 2022 7:56 pmNVIDIA directly deleted libglamoregl.so and deleted old libglx.so file and put different file called libglx.so.390.147 and created a symbolic connection(i dont know actual name) and named with libglx.so

Original libglx.so in Live USB is 130 kb and NVIDIA's file is 15 MB.
Symbolic link (or symlink) is the technical term for symbolic connection.

Looking at the other topic I referenced (the one with roblm's quote), I see that person mentioned it was only a symbolic link as well (to libglx.so.390.141 in their situation) so they added both files.

I do not see anything listed in journalctl which indicates what might be happening with Nvidia. I will have to go back through my notes and some older topics to see if I can come up with some ideas. I will get back to you tomorrow with what I find.
Image
A woman typing on a laptop with LM20.3 Cinnamon.
User avatar
SMG
Level 25
Level 25
Posts: 31325
Joined: Sun Jul 26, 2020 6:15 pm
Location: USA

Re: NVIDIA Driver Not Usable on LDME5

Post by SMG »

What is the output of

Code: Select all

lspci -nnk | grep -EA3 "3D|VGA"
and

Code: Select all

ls /lib/modules/`uname -r`/kernel/drivers/char/drm/
Image
A woman typing on a laptop with LM20.3 Cinnamon.
osmanakar312
Level 2
Level 2
Posts: 61
Joined: Sun Mar 27, 2022 4:47 pm

Re: NVIDIA Driver Not Usable on LDME5

Post by osmanakar312 »

lspci -nnk | grep -EA3 "3D|VGA":

Code: Select all

00:02.0 VGA compatible controller [0300]: Intel Corporation 3rd Gen Core processor Graphics Controller [8086:0166] (rev 09)
	Subsystem: Samsung Electronics Co Ltd 3rd Gen Core processor Graphics Controller [144d:c652]
	Kernel driver in use: i915
	Kernel modules: i915
--
01:00.0 3D controller [0302]: NVIDIA Corporation GF117M [GeForce 610M/710M/810M/820M / GT 620M/625M/630M/720M] [10de:1140] (rev a1)
	Subsystem: Samsung Electronics Co Ltd GeForce GT 620M on NP300E5C series laptop [144d:c652]
	Kernel driver in use: nvidia
	Kernel modules: nouveau, nvidia_drm, nvidia
ls /lib/modules/`uname -r`/kernel/drivers/char/drm/ has no output, there is no file called drm.

Code: Select all

ls: '/lib/modules/5.10.0-14-amd64/kernel/drivers/char/drm/''e erişilemedi: Böyle bir dosya ya da dizin yok
Btw i can access NVIDIA packages like nvidia-smi or nvidia-settings after driver installation in NVIDIA's website.
nvidia-smi:

Code: Select all

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 390.147                Driver Version: 390.147                   |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GT 620M     Off  | 00000000:01:00.0 N/A |                  N/A |
| N/A   44C    P0    N/A /  N/A |      0MiB /   964MiB |     N/A      Default |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0                    Not Supported                                       |
+-----------------------------------------------------------------------------+
When launched "nvidia-settings" it says;

"You do not appear to be using the NVIDIA X driver. Please edit your X configuration file(just run "nvidia-xconfig" as root), and restart the X server."

Should i do?
User avatar
SMG
Level 25
Level 25
Posts: 31325
Joined: Sun Jul 26, 2020 6:15 pm
Location: USA

Re: NVIDIA Driver Not Usable on LDME5

Post by SMG »

osmanakar312 wrote: Mon May 16, 2022 1:15 am

Code: Select all

01:00.0 3D controller [0302]: NVIDIA Corporation GF117M [GeForce 610M/710M/810M/820M / GT 620M/625M/630M/720M] [10de:1140] (rev a1)
	Subsystem: Samsung Electronics Co Ltd GeForce GT 620M on NP300E5C series laptop [144d:c652]
	Kernel driver in use: nvidia
	Kernel modules: nouveau, nvidia_drm, nvidia
The Nvidia-390 is a little different from the latest drivers, but I expected to see more modules.

I just checked the link jwiz provided earlier in the topic in the hopes that maybe we are at the stage that could be used, but NvidiaXrun indicates:
To use nvidia-xrun, ensure that all appropriate Nvidia kernel modules are loaded (e.g., nvidia, nvidia_modeset, nvidia_uvm, nvidia_drm)
I will need to find the specifics for the Nvidia-390 to verify which modules should be available.
osmanakar312 wrote: Mon May 16, 2022 1:15 amls /lib/modules/`uname -r`/kernel/drivers/char/drm/ has no output, there is no file called drm.
Either that is the problem (which is possible given the lack of a drm seen until too late in the Xorg log), or it is another difference between LMDE5 and Ubuntu-based Mint.
osmanakar312 wrote: Mon May 16, 2022 1:15 amWhen launched "nvidia-settings" it says;

"You do not appear to be using the NVIDIA X driver. Please edit your X configuration file(just run "nvidia-xconfig" as root), and restart the X server."

Should i do?
If you run that, it will create an xorg.conf file that presumes you are only running with Nvidia. I am aware of a parameter that can be passed to it that in the past would create the correct file for a dual-graphics laptop, but it has been so long since anyone needed to do that that I'm not sure that still works with current drivers. Additionally, I'm not sure it would work if the correct kernel modules are not being built.

I will let you know what I find.
Image
A woman typing on a laptop with LM20.3 Cinnamon.
osmanakar312
Level 2
Level 2
Posts: 61
Joined: Sun Mar 27, 2022 4:47 pm

Re: NVIDIA Driver Not Usable on LDME5

Post by osmanakar312 »

Okay, i understand. Thanks for your effort. Please let me know if you find.
User avatar
SMG
Level 25
Level 25
Posts: 31325
Joined: Sun Jul 26, 2020 6:15 pm
Location: USA

Re: NVIDIA Driver Not Usable on LDME5

Post by SMG »

In the Nvidia documentation for the 390.147 driver Chapter 5. Listing of Installed Components indicates it should have:
  • A kernel module (/lib/modules/`uname -r`/kernel/drivers/video/nvidia-modeset.ko); this kernel module is responsible for programming the display engine of the GPU. User-mode NVIDIA driver components such as the NVIDIA X driver, OpenGL driver, and VDPAU driver communicate with nvidia-modeset.ko through the /dev/nvidia-modeset device file.
  • A kernel module (/lib/modules/`uname -r`/kernel/drivers/video/nvidia.ko); this kernel module provides low-level access to your NVIDIA hardware for all of the above components. It is generally loaded into the kernel when the X server is started, and is used by the X driver and OpenGL. nvidia.ko consists of two pieces: the binary-only core, and a kernel interface that must be compiled specifically for your kernel version. Note that the Linux kernel does not have a consistent binary interface like the X server, so it is important that this kernel interface be matched with the version of the kernel that you are using. This can either be accomplished by compiling yourself, or using precompiled binaries provided for the kernels shipped with some of the more common Linux distributions.
  • NVIDIA Unified Memory kernel module (/lib/modules/`uname -r`/kernel/drivers/video/nvidia-uvm.ko); this kernel module provides functionality for sharing memory between the CPU and GPU in CUDA programs. It is generally loaded into the kernel when a CUDA program is started, and is used by the CUDA driver on supported platforms.
Check those locations to see if you have those files. You may not need the last one, but I would think you need the first two.

In Chapter 17. Using the NVIDIA Driver with Optimus Laptops it indicates:
On systems without a mux, the NVIDIA GPU can still be useful for offscreen rendering, running CUDA applications, and other uses that don't require driving a display.

On muxless Optimus laptops, or on laptops where a mux is present, but not set to drive the internal display from the NVIDIA GPU, the internal display is driven by the integrated GPU. On these systems, it's important that the X server not be configured to use the NVIDIA X driver after the driver is installed.
I went through the Appendix B. X Config Options and I do not see the one with which I am familiar with regards to Intel/Nvidia laptops like you have. However, Chapter 33. Offloading Graphics Display with RandR 1.4 does have a sample xorg.conf file. I do not know if that will help in this situation or not.

Additionally, I see at the bottom of that page the Caveat:
Some versions of the “modesetting” driver try to load a sub-module called “glamor”, which conflicts with the NVIDIA GLX implementation. Please ensure that the libglamoregl.so X module is not installed.
which is a bit confusing since if one is running with Intel that is needed (as we already learned earlier).

Check to see if you do have those kernels modules (as the location is different from the one I originally gave you). I'm going to go back through the Debian documentation next.
Image
A woman typing on a laptop with LM20.3 Cinnamon.
User avatar
SMG
Level 25
Level 25
Posts: 31325
Joined: Sun Jul 26, 2020 6:15 pm
Location: USA

Re: NVIDIA Driver Not Usable on LDME5

Post by SMG »

I found something on the Nvidia developers forum that indicates it may be a timing issue of when the Nvidia drm is loading. I do not know if restarting X server after the desktop loads might help because I'm not sure why it is "slow" to load.

The Debian Wiki: Nvidia Driver: Configuration section indicates, "Modern Debian packages for the NVIDIA driver should not require you to do anything listed here as they handle this automatically during installation..." which is also what the Optimus page seemed to indicate. The other inxi outputs I found on the Debian forum all showed the Nvidia display driver. I do not know what might be different about your install that it is not loading properly.

Perhaps because you installed the driver from the website configuration may be needed? If so, you can try creating the xorg.conf from the Nvidia documentation and place it in /etc/X11.

I did find this topic on the Debian forum nvidia-legacy-390xx-driver not loaded and the inxi output looks like yours (Kernel drivers are loaded, but only modesetting display driver is loaded). However, that is an older topic on an older Debian base and the person ended up using Bumblebee/Optirun which is no longer the recommendation for Optimus.

I am out of ideas. :(
Image
A woman typing on a laptop with LM20.3 Cinnamon.
Tenebris_Umbra
Level 1
Level 1
Posts: 40
Joined: Thu May 23, 2019 9:23 pm

Re: NVIDIA Driver Not Usable on LDME5

Post by Tenebris_Umbra »

I too have an older nvidia card and use 390 (which eol is Dec 2022, so its still supported for now). The issue is some of the Debian builds only support kernels to a point like 390.144 only goes to 5.10. Since I'm running the liquorix kernel at 5.17.7-1, I pulled the nvidia 390.147-4 driver from sid, and it works fine and dkms builds nvidia to the 5.17 kernel
osmanakar312
Level 2
Level 2
Posts: 61
Joined: Sun Mar 27, 2022 4:47 pm

Re: NVIDIA Driver Not Usable on LDME5

Post by osmanakar312 »

SMG wrote: Mon May 16, 2022 7:22 pm I found something on the Nvidia developers forum that indicates it may be a timing issue of when the Nvidia drm is loading. I do not know if restarting X server after the desktop loads might help because I'm not sure why it is "slow" to load.

The Debian Wiki: Nvidia Driver: Configuration section indicates, "Modern Debian packages for the NVIDIA driver should not require you to do anything listed here as they handle this automatically during installation..." which is also what the Optimus page seemed to indicate. The other inxi outputs I found on the Debian forum all showed the Nvidia display driver. I do not know what might be different about your install that it is not loading properly.

Perhaps because you installed the driver from the website configuration may be needed? If so, you can try creating the xorg.conf from the Nvidia documentation and place it in /etc/X11.

I did find this topic on the Debian forum nvidia-legacy-390xx-driver not loaded and the inxi output looks like yours (Kernel drivers are loaded, but only modesetting display driver is loaded). However, that is an older topic on an older Debian base and the person ended up using Bumblebee/Optirun which is no longer the recommendation for Optimus.

I am out of ideas. :(
Yes, you are right. Restarting X Server(log-in and out) will loads nvidia drivers, but in first boot it won't work.

Code: Select all

Graphics:
  Device-1: Intel 3rd Gen Core processor Graphics driver: i915 v: kernel 
  Device-2: NVIDIA GF117M [GeForce 610M/710M/810M/820M / GT 
  620M/625M/630M/720M] 
  driver: nvidia v: 390.147 
  Device-3: Z-Star Micro WebCam SC-03FFL11739P type: USB driver: uvcvideo 
  Display: x11 server: X.Org 1.20.11 driver: 
  loaded: modesetting,nouveau,nvidia unloaded: fbdev,vesa 
  resolution: 1366x768~60Hz 
  OpenGL: renderer: Mesa DRI Intel HD Graphics 4000 (IVB GT2) 
  v: 4.2 Mesa 20.3.5
I through blacklisted nouveau but...
User avatar
SMG
Level 25
Level 25
Posts: 31325
Joined: Sun Jul 26, 2020 6:15 pm
Location: USA

Re: NVIDIA Driver Not Usable on LDME5

Post by SMG »

osmanakar312 wrote: Tue May 17, 2022 1:04 amYes, you are right. Restarting X Server(log-in and out) will loads nvidia drivers, but in first boot it won't work.
There is a way to get the nvidia modules loaded earlier on Ubuntu-based Mint, but I don't know if that would work here because in that case nouveau is loading instead of nvidia. In your case, neither nvidia nor nouveau was loading initially. Now they both are loading after restarting X Server. :?
osmanakar312 wrote: Tue May 17, 2022 1:04 amI through blacklisted nouveau but...
I recall you saying you did when you had the Debian Nvidia packages installed. Maybe it needs redone because you installed the driver from the website?
Image
A woman typing on a laptop with LM20.3 Cinnamon.
osmanakar312
Level 2
Level 2
Posts: 61
Joined: Sun Mar 27, 2022 4:47 pm

Re: NVIDIA Driver Not Usable on LDME5

Post by osmanakar312 »

I've already created a "blacklist-nvidia-nouveau.conf" file in /etc/modprobe.d and put in this:

Code: Select all

blacklist nouveau
options nouveau modeset=0
And still loads itself, in first boot no, logging off and back log in loads itself. :roll:
User avatar
SMG
Level 25
Level 25
Posts: 31325
Joined: Sun Jul 26, 2020 6:15 pm
Location: USA

Re: NVIDIA Driver Not Usable on LDME5

Post by SMG »

osmanakar312 wrote: Tue May 17, 2022 10:27 am I've already created a "blacklist-nvidia-nouveau.conf" file in /etc/modprobe.d and put in this:

Code: Select all

blacklist nouveau
options nouveau modeset=0
And you updated the initramfs and rebooted after doing that?

Code: Select all

sudo update-initramfs -u; reboot
The Debian wiki: Nvidia documentation indicates " The nouveau kernel module is blacklisted by the glx-alternative-nvidia or nvidia-kernel-common packages." so it seems no specific action would have been needed with the Debian driver (if I am interpreting the documentation correctly), but you did need to do it for the website driver.
Image
A woman typing on a laptop with LM20.3 Cinnamon.
osmanakar312
Level 2
Level 2
Posts: 61
Joined: Sun Mar 27, 2022 4:47 pm

Re: NVIDIA Driver Not Usable on LDME5

Post by osmanakar312 »

Yes. I did it.
User avatar
SMG
Level 25
Level 25
Posts: 31325
Joined: Sun Jul 26, 2020 6:15 pm
Location: USA

Re: NVIDIA Driver Not Usable on LDME5

Post by SMG »

I do not know if Xorg log might indicate from where it originates, but I will check if you want to provide the log. After you have nvidia showing as a loaded display driver run:

Code: Select all

cat /var/log/Xorg.0.log | nc termbin.com 9999
and let us know the url.
Image
A woman typing on a laptop with LM20.3 Cinnamon.
osmanakar312
Level 2
Level 2
Posts: 61
Joined: Sun Mar 27, 2022 4:47 pm

Re: NVIDIA Driver Not Usable on LDME5

Post by osmanakar312 »

Tenebris_Umbra
Level 1
Level 1
Posts: 40
Joined: Thu May 23, 2019 9:23 pm

Re: NVIDIA Driver Not Usable on LDME5

Post by Tenebris_Umbra »

Sometimes "blacklist-nvidia-nouveau.conf" doesnt work, and you need to create a /etc/modprobe.d/blacklist.conf with blacklist nouveau in it. A simple, "echo 'blacklist nouveau' | sudo tee --append /etc/modprobe.d/blacklist.conf && sudo update-initramfs -u -k all" , should work
Locked

Return to “Hardware Support”