environment modules issues [solved?]

[gostal]

There is a repo-package with binaries for environment modules so one does not have to download from SourceForge and build from scratch. The package name is: environment-modules version 4.1.1-1 and originates from Ubuntu.

There are, however, some issues:

• To enable environment modules the file /usr/share/modules/init/$shell, has to be sourced. ($shell is the name of the shell). This is supposed to be done by /etc/profile which sources all *.sh-files in /etc/profile.d where the installation of the package puts the file modules.sh which sources the initialisation file in /usr/share/modules/init/. Assuming the shell is bash this works fine in text login because then bash reads /etc/profile but apparently /etc/profile is not read during graphical login, at least not in Mate, which leaves environment modules uninitialised when a terminal window is opened as bash does not read /etc/profile because it is not invoked as a login shell.
  
  To remedy this I edited ~/.bashrc to directly source /etc/profile.d/modules.sh as ~/.bashrc is read by bash every time a terminal window is opened. Incidentally, this file is not read by bash when invoked as a login shell which is the reason why usually ~/.profile which also is read during text login sources ~/.bashrc. There is also a bunch of other *.sh-files in /etc/profile.d/ but I guess corresponding initialisations are not needed in a graphical terminal window otherwise I could of course just as easily have sourced /etc/profile in ~/.bashrc. Comments here would be appreciated. The question is if this is the best way to solve the problem. Is there perhaps some way to tell the graphical login process to read /etc/profile (or if that's not advised then only the initialisation file)? I imagine it would be good to have environment modules initialised once and for all instead of having to do it all the time.
  
  If not reading /etc/profile during graphical login is intended it would have been nice if the installation process issued a warning that environment modules might need initialisation as it took me a bit of investigation to find out what the trouble was.

• Once initialised environment modules behaves a bit odd. If the querie command module avail is entered the response is the following:

  Code: Select all

  > module avail
  ---------------------------------- /usr/share/modules/modulefiles ----------------------------------
  dot  module-git  module-info  modules  null  use.own  
  
  ---------------------------------- /usr/share/modules/modulefiles ----------------------------------
  dot  module-git  module-info  modules  null  use.own  
  
  

  i.e. it looks like the command repeats itself invisibly. The same goes for a bunch of other commands that generate output. Other than that things seem to work. I don't know if this behaviour is caused by Mint system tweaks or if the same thing would happen on Ubuntu. Does anybody know what's going on here? If so, do tell.

[gostal]

It would appear that the reason extra measures are needet to initialise Environment Modules is a change of display manager, see this viewtopic.php?t=278474#p1617505 post.

From this I gather that I might as well source /etc/profile directly rather than just /etc/profile.d/modules.sh. But I also understand that there is a point in having both a ~./profile and a ~./bashrc as the first is meant to contain stuff that's only needed to run once for every login and the latter for stuff that you wan't to initialise for every invocation. As Environment Modules by design is meant to provide a means to customize your environment i a per session way without affecting the system at all it makes sense to initialise it during login rather than doing it in my ad hoc fashion.

The conclusion seems to be to either apply the fix given in the link above or change display manager to one that does read /etc/profile. If it is at all needed anymore. Perhaps the system files have been updated so that it is no more an issue.

This does not, however, answer why the system module files are read twice ... does it? Comment would be appreciated!

Cheers,
Gösta (gostal)

[gostal]

I a newbie at Mint but not to Linux, as such. In the beginning of October 2018 I posted a message about Environment Modules. This software doesn't get initialised, the reason being that its intended initialisation assumes that /etc/profile is sourced during login. That it is, indeed, if you're using text login but not if you're logging in graphically, which is far more common. The post is here:

viewtopic.php?f=47&t=279377

In it there are some questions and thoughts that I really would appreciate comments on. So, please, please, please, read that post and get back to me. Mind you, I'm just a user, albeit fairly experienced, and not a systems guru so there's bound to be considerations I'm not aware of.

Thank you beforehand for any feedback that I really do hope you'll have.

Cheers,
Gösta (gostal)

[karlchen]

<moderator on>
Merged the threads "Doesn't anybody care?" and "environment modules issues".
If you wish to bump the thread "environment modules issues", ok, go ahead and do so.
But please do not create extra-threads, the purpose of which is only to make people read the thread, which is important to you.

Please, also consider the question whether perhaps the majority of readers may have failed to see why the current behaviour is a problem.
After all it seems be by design:
/etc/profile, /etc/profile.d/* (systemwide) and ~/.profile and ~/.bashrc (user-specific) are meant to be sourced by terminal sessions only.
</moderator off>

[gostal]

OK, point taken. Sorry about that. I didn't know the correct way to try and call for attention. I should have done my homework better. I'll try and find out now.

Thanks also for confirming that the, for me, annoying behaviour is intended. Is there a way to tell LightDM to source /etc/profile etc. or will that break something? I suppose a way to solve the problem would be to switch to text login (change runlevel) and then start X manually? It seems a bit clumsy off hand but would it messy to set up?

Cheers,
Gösta (gostal)

[gostal]

Regarding the fact that /etc/profile is not sourced by LightDM during graphical login there was quite a vivid discussion in the Ubuntu bug reports some years ago, see
https://bugs.launchpad.net/ubuntu/+sour ... bug/794315
and links therein.
By way of those links I found this page:
https://www.sitepoint.com/understanding ... n-scripts/
explaining why and also indicating that ~./xsessionrc (a file one has to create) would be a good place to source /etc/profile so that Environment Modules would be initialised the intended way, i.e., once per X session. I tried that but it didn't work . I then tried another of the suggested approaches namely to source ~/.profile and in it source the
Environment Modules initialisation file /etc/profile.d/modules.sh. No luck there. (Well, the file is not really the initialisation file, more like a pointer to it. ) So far the only working way seems to be to source /etc/profile or /etc/profile.d/modules.sh directly in ~/.bashrc but this is not very satisfactory as it means that the software is initialised every single time I open a command window even if I don't need it. So I'm back to square 1 and I still don't know how do initialise the software during graphical login.

The documentation found on the internet is confusing and different distros initialise the environment in various ways but one thing that I've read and can appreciate is that text and graphical login may require different things and therefore there is another set of startup files for X that's supposed to play a similar role that /etc/profile and files under /etc/profile.d/ does for the command line. The files I'm talking about are /etc/X11/Xsession and files under /etc/X11/Xsession.d. One of the files there is supposed to check if ~/.xsessionrc exists and if so read it but apparently this does not work in the way described in the cited web page. So the million dollar question here seem to be:
How does this work in Mint19????!!!!
I'm dying to know and the Mint documentation is no help so please, forum, come to my aid!!!!

Cheers Gösta
(gostal)

[gostal]

Now, I tried putting a dedicated script named 40environment-modules_initialise in /etc/Xsession.d/ with the content:

Code: Select all

# Initialising Environment Modules

# This file is sourced by Xsession(5), not executed. Hopefully!

#gostal 2019-04-18
. /etc/profile.d/modules.sh

Guess what! Still no luck !! Then I found this:
https://askubuntu.com/questions/598628/ ... on-lightdm
It appears then that /etc/X11/Xsession is not run at all or ...? Well, it's not the first time I have experienced entire subtrees under /etc to just be sitting there and doing nothing. So I'm not surprised but what surprises me and on top of that bugs me is that I find no relevant info though I really try looking for it,

So then I tried adding the file 70-session-setup-script.conf to /etc/lightdm/lightdm.conf.d containing:

Code: Select all

[SeatDefaults]
session-setup-script=/etc/X11/Xsession

in order that /etc/X11/Xsession really should be run according to info on this page:
https://wiki.ubuntu.com/LightDM.
Well, it had the desired effect that Environment Modules were initialised but the very unwanted and potentially dangerous side effect that X was started with user root!!! This is also proof that /etc/X11/Xsession really is not run and that scripts under /etc/X11/Xsession.d are sourced some other way if sourced at all.

Really, developers, I realise that you cannot anticipate everything but given available info it's not so far fetched that someone should do what I just did and to prevent that relevant info is needed. In particular, there should be some documentation of what to do if you want to run software which initialisation assumes that /etc/profile is sourced during graphical login.

In principle I like Mint but there is clearly room for improvement here. So, I implore you, developers, to get busy fixing this because I am really fed up with this trial-and-error shit . To begin with I would like a detailed instruction on how to construct a script that will have the desired effect and where I should put it. Also, it would be a good idea to lift this issue because it is much bigger than Environment Modules.

Please respond, in any way you want but do respond!

Cheers, Gösta
(gostal)

[gostal]

Now, I am really at my wits end! Please, help! This page:
http://www.orcus.de/main_workarounds/lm ... ofile.htm
details how you can test whether various startup files are sourced or not during login. So I put the lines:

Code: Select all

# debugging x-profiles getting (not) sourced on x-session startup
# comment or remove if not needed anymore (after debugging)
date >>/tmp/pro-$USER.txt
echo "PPID='$PPID' '$0' src '$BASH_SOURCE' for '$USER'" >>/tmp/pro-$USER.txt

at the top of
/etc/profile
/etc/profile.d/modules.sh
/etc/profile.d/vte-2.91.sh
~/.profile
~/.xsessionrc
According to the cited web page I will get output in /tmp/pro-gostal.txt if the files are sourced. ($USER=gostal)
This is the content of /tmp/pro-gostal.txt after logout and login:

Code: Select all

fre 19 apr 2019 14:30:46 CEST
PPID='19096' '/usr/sbin/lightdm-session' src '/etc/profile' for 'gostal'
fre 19 apr 2019 14:30:47 CEST
PPID='19096' '/usr/sbin/lightdm-session' src '/etc/profile.d/modules.sh' for 'gostal'
fre 19 apr 2019 14:30:47 CEST
PPID='19096' '/usr/sbin/lightdm-session' src '/etc/profile.d/vte-2.91.sh' for 'gostal'
fre 19 apr 2019 14:30:47 CEST
PPID='19096' '/usr/sbin/lightdm-session' src '/home/gostal/.profile' for 'gostal'
fre 19 apr 2019 14:30:47 CEST
PPID='19096' '/usr/sbin/lightdm-session' src '/etc/profile.d/modules.sh' for 'gostal'
fre 19 apr 2019 14:30:47 CEST
PPID='19096' '/usr/sbin/lightdm-session' src '/home/gostal/.xsessionrc' for 'gostal'

Lo and behold! Every file that I put the test in is indeed sourced!! So Environment Modules should be initialised, right? But evidently this is not the case as I
get module: command not found in a terminal session. Note that modules.sh is sourced twice but all the other files only once. I don't know why this happens, Currently all sourcing lines that I've put in ~/.profile, ~/.xsessionrc and ~/.bashrc are commented out (otherwise the module command would not fail). Could this double sourcing be the reason why the initialisation of the software gets screwed up? Perhaps this double sourcing is a manifestation of the same cause that lies behind the double report of system modules resulting from the command module avail, see the first post in this thread. Clearly, this behaviour is odd and perhaps even harmful. Do any of you Mint gurus have a clue? Perhaps the content of ~/.xsession-errors can shed some light so here it is:

Code: Select all

dbus-update-activation-environment: setting DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
dbus-update-activation-environment: setting DISPLAY=:0
dbus-update-activation-environment: setting XAUTHORITY=/home/gostal/.Xauthority
dbus-update-activation-environment: setting GTK_MODULES=gail:atk-bridge
dbus-update-activation-environment: setting QT_ACCESSIBILITY=1
dbus-update-activation-environment: setting LANG=sv_SE.UTF-8
dbus-update-activation-environment: setting GDM_LANG=sv
dbus-update-activation-environment: setting DISPLAY=:0
dbus-update-activation-environment: setting COMPIZ_CONFIG_PROFILE=mate
dbus-update-activation-environment: setting MANDATORY_PATH=/usr/share/gconf/mate.mandatory.path
dbus-update-activation-environment: setting XDG_GREETER_DATA_DIR=/var/lib/lightdm-data/gostal
dbus-update-activation-environment: setting MODULES_CMD=/usr/lib/x86_64-linux-gnu/modulecmd.tcl
dbus-update-activation-environment: setting USER=gostal
dbus-update-activation-environment: setting ENV=/usr/share/modules/init/profile.sh
dbus-update-activation-environment: setting DESKTOP_SESSION=mate
dbus-update-activation-environment: setting DEFAULTS_PATH=/usr/share/gconf/mate.default.path
dbus-update-activation-environment: setting PWD=/home/gostal
dbus-update-activation-environment: setting HOME=/home/gostal
dbus-update-activation-environment: setting QT_ACCESSIBILITY=1
dbus-update-activation-environment: setting XDG_SESSION_TYPE=x11
dbus-update-activation-environment: setting BASH_ENV=/usr/share/modules/init/bash
dbus-update-activation-environment: setting XDG_DATA_DIRS=/usr/share/mate:/home/gostal/.local/share/flatpak/exports/share:/var/lib/flatpak/exports/share:/usr/local/share:/usr/share
dbus-update-activation-environment: setting XDG_SESSION_DESKTOP=mate
dbus-update-activation-environment: setting LOADEDMODULES=
dbus-update-activation-environment: setting GTK_MODULES=gail:atk-bridge
dbus-update-activation-environment: setting SHELL=/bin/bash
dbus-update-activation-environment: setting XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0
dbus-update-activation-environment: setting IM_CONFIG_PHASE=1
dbus-update-activation-environment: setting XDG_CURRENT_DESKTOP=MATE
dbus-update-activation-environment: setting GPG_AGENT_INFO=/run/user/1000/gnupg/S.gpg-agent:0:1
dbus-update-activation-environment: setting SHLVL=1
dbus-update-activation-environment: setting LANGUAGE=sv
dbus-update-activation-environment: setting MODULEPATH=/etc/environment-modules/modules:/usr/share/modules/versions:/usr/share/modules/$MODULE_VERSION/modulefiles:/usr/share/modules/modulefiles
dbus-update-activation-environment: setting GDMSESSION=mate
dbus-update-activation-environment: setting LOGNAME=gostal
dbus-update-activation-environment: setting DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
dbus-update-activation-environment: setting XDG_RUNTIME_DIR=/run/user/1000
dbus-update-activation-environment: setting XAUTHORITY=/home/gostal/.Xauthority
dbus-update-activation-environment: setting MODULEPATH_modshare=/usr/share/modules/$MODULE_VERSION/modulefiles:1:/etc/environment-modules/modules:1:/usr/share/modules/modulefiles:1:/usr/share/modules/versions:1
dbus-update-activation-environment: setting XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session3
dbus-update-activation-environment: setting XDG_CONFIG_DIRS=/etc/xdg/xdg-mate:/etc/xdg
dbus-update-activation-environment: setting PATH=/home/gostal/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
dbus-update-activation-environment: setting MODULESHOME=/usr/share/modules
dbus-update-activation-environment: setting BASH_FUNC_module%%=() {  unset _mlre _mlIFS _mlshdbg;
 if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
 case "$-" in 
 *v*x*)
 set +vx;
 _mlshdbg='vx'
 ;;
 *v*)
 set +v;
 _mlshdbg='v'
 ;;
 *x*)
 set +x;
 _mlshdbg='x'
 ;;
 *)
 _mlshdbg=''
 ;;
 esac;
 fi;
 if [ -n "${IFS+x}" ]; then
 _mlIFS=$IFS;
 fi;
 IFS=' ';
 for _mlv in ${MODULES_RUN_QUARANTINE:-};
 do
 if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
 if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then
 _mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' ";
 fi;
 _mlrv="MODULES_RUNENV_${_mlv}";
 _mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' ";
 fi;
 done;
 if [ -n "${_mlre:-}" ]; then
 _mlre="eval ${_mlre}";
 fi;
 eval `${_mlre:-}/usr/bin/tclsh /usr/lib/x86_64-linux-gnu/modulecmd.tcl sh $*`;
 _mlstatus=$?;
 if [ -n "${_mlIFS+x}" ]; then
 IFS=$_mlIFS;
 else
 unset IFS;
 fi;
 if [ -n "${_mlshdbg:-}" ]; then
 set -$_mlshdbg;
 fi;
 unset _mlre _mlv _mlrv _mlIFS _mlshdbg;
 return $_mlstatus
}
dbus-update-activation-environment: setting BASH_FUNC_switchml%%=() {  swfound=1;
 if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
 swname='main';
 if [ -e /usr/lib/x86_64-linux-gnu/modulecmd.tcl ]; then
 swfound=0;
 unset MODULES_USE_COMPAT_VERSION;
 fi;
 else
 swname='compatibility';
 if [ -e /usr/lib/x86_64-linux-gnu/modulecmd-compat ]; then
 swfound=0;
 MODULES_USE_COMPAT_VERSION=1;
 export MODULES_USE_COMPAT_VERSION;
 fi;
 fi;
 if [ $swfound -eq 0 ]; then
 echo "Switching to Modules $swname version";
 . /usr/share/modules/init/sh;
 else
 echo "Cannot switch to Modules $swname version, command not found";
 return 1;
 fi
}
dbus-update-activation-environment: setting _=/usr/bin/dbus-update-activation-environment
dbus-update-activation-environment: warning: error sending to systemd: org.freedesktop.DBus.Error.InvalidArgs: Invalid environment assignments
mate-session[19114]: WARNING: Unable to find provider '' of required component 'dock'
Fönsterhanterarvarning: Log level 16: XPresent is not compatible with your current system configuration.

(nm-applet:19598): Gtk-WARNING **: 14:30:50.416: Can't set a parent on widget which has a parent

(mate-power-manager:19627): Gdk-CRITICAL **: 14:30:50.718: gdk_window_thaw_toplevel_updates: assertion 'window->update_and_descendants_freeze_count > 0' failed

(nm-applet:19598): Gdk-CRITICAL **: 14:30:50.720: gdk_window_thaw_toplevel_updates: assertion 'window->update_and_descendants_freeze_count > 0' failed

So, karlchen, if it is indeed by design that /etc/profile etc. are meant to be sourced in terminal sessions only, then why are they sourced at all by lightdm-session? This does not make sense and contradicts your statement. Clearly, there is something fishy going on here and I am beginning to have a mind of filing a bug report. So, system gurus, what is your take on this?

Cheers, Gösta
(gostal)

[gostal]

Regarding the double sourcing of /etc/profile.d/modules.sh. My mistake. I had forgot about the file under /etc/Xsession.d/ I had put there. That file was responsible for the second sourcing. The file is still there but I commented out the sourcing line and put in the test lines at the top as well as in the file sourcing ~/.xsessionrc.
Now the various files tested are only sourced once. /temp/pro-gostal.txt:

Code: Select all

fre 19 apr 2019 17:37:42 CEST
PPID='913' '/usr/sbin/lightdm-session' src '/etc/profile' for 'gostal'
fre 19 apr 2019 17:37:43 CEST
PPID='913' '/usr/sbin/lightdm-session' src '/etc/profile.d/modules.sh' for 'gostal'
fre 19 apr 2019 17:37:43 CEST
PPID='913' '/usr/sbin/lightdm-session' src '/etc/profile.d/vte-2.91.sh' for 'gostal'
fre 19 apr 2019 17:37:43 CEST
PPID='913' '/usr/sbin/lightdm-session' src '/home/gostal/.profile' for 'gostal'
fre 19 apr 2019 17:37:43 CEST
PPID='913' '/usr/sbin/lightdm-session' src '/etc/X11/Xsession.d/40environment-modules_initialise' for 'gostal'
fre 19 apr 2019 17:37:43 CEST
PPID='913' '/usr/sbin/lightdm-session' src '/etc/X11/Xsession.d/40x11-common_xsessionrc' for 'gostal'
fre 19 apr 2019 17:37:43 CEST
PPID='913' '/usr/sbin/lightdm-session' src '/home/gostal/.xsessionrc' for 'gostal'

But the question remains: Why is all this sourcing of startup files "forgot" when I fire up xterm? Where and how does it get wiped out?

Cheers, Gösta
(gostal)

[orcus]

hi gostal,

as your shell-script does get executed/sourced - the environment variable should be setup - which you could
simply test by writing their value to the /tmp- file like the other debug-output ...

My guessing would be somthing far simpler: you might have forgotten to export the variables = as it's not enough
to set the according value but you've to make it available to other processes too - something like:

export my-env-var=some-value

good luck - orcus

[gostal]

Hi orcus,

Thanks for responding. Save for the moderator who rightfully told me off a few posts back you're the only one so far.

But although it is conceivable that set variables are not exported this is not the case as will be demonstrated further down.

To be quite clear I installed from sources at Sourceforge to see if there was something wrong with the repo package (which by the way I removed). This is a later version than the one in the repo package but that can hardly matter regarding this issue. It turned out that there is indeed something odd with the repo package as the double report of system modules described in the first post no longer shows. So I will do a bug report regarding this.

Now to the mystery: I am running bash and this is the content of the initialisation file for bash:

Code: Select all

unset _mlshdbg;
# disable shell debugging for the run of this init file
if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
   # immediately disable debugging to echo the less number of line possible
   case "$-" in
      *v*x*) set +vx; _mlshdbg='vx' ;;
      *v*) set +v; _mlshdbg='v' ;;
      *x*) set +x; _mlshdbg='x' ;;
      *) _mlshdbg='' ;;
   esac;
fi;

# define modules runtine quarantine configuration
#export MODULES_RUN_QUARANTINE='ENVVARNAME'

# setup quarantine if defined
unset _mlre _mlIFS;
if [ -n "${IFS+x}" ]; then
   _mlIFS=$IFS;
fi;
IFS=' ';
for _mlv in ${MODULES_RUN_QUARANTINE:-}; do
   if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
      if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then
         _mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' ";
      fi;
      _mlrv="MODULES_RUNENV_${_mlv}";
      _mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' ";
   fi;
done;
if [ -n "${_mlre:-}" ]; then
   _mlre="eval ${_mlre}";
fi;

# define module command and surrounding initial environment (default value
# for MODULESHOME, MODULEPATH, LOADEDMODULES and parse of init/.modulespath)
_mlcode=`${_mlre:-}/usr/bin/tclsh /opt/Modules/libexec/modulecmd.tcl bash autoinit`
_mlret=$?

# clean temp variables used to setup quarantine
if [ -n "${_mlIFS+x}" ]; then
   IFS=$_mlIFS; unset _mlIFS;
else
   unset IFS;
fi;
unset _mlre _mlv _mlrv

# no environment alteration if the above autoinit command failed
if [ $_mlret -eq 0 ]; then
   eval "$_mlcode"

   # redefine module command if compat version has been activated
   if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
      MODULES_CMD=/opt/Modules/libexec/modulecmd-compat; export MODULES_CMD
      if [ -t 2 ]; then
         _module_raw() { eval `/opt/Modules/libexec/modulecmd-compat bash $*`; }
      else
         module() { eval `/opt/Modules/libexec/modulecmd-compat bash $*`; }
      fi
   fi

   # export functions to get them defined in sub-shells
   if [ -t 2 ]; then
      export -f _module_raw
   fi
   export -f module

   # define function to switch between C and Tcl versions of Modules
   switchml() {
      typeset swfound=1
      if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
         typeset swname='main'
         if [ -e /opt/Modules/libexec/modulecmd.tcl ]; then
            typeset swfound=0
            unset MODULES_USE_COMPAT_VERSION
         fi
      else
         typeset swname='compatibility'
         if [ -e /opt/Modules/libexec/modulecmd-compat ]; then
            typeset swfound=0
            MODULES_USE_COMPAT_VERSION=1; export MODULES_USE_COMPAT_VERSION
         fi
      fi

      # switch version only if command found
      if [ $swfound -eq 0 ]; then
         echo "Switching to Modules $swname version"
         source /opt/Modules/init/bash
      else
         echo "Cannot switch to Modules $swname version, command not found"
         return 1
      fi
   }
   export -f switchml

   # setup ENV variables to get module defined in sub-shells (works for 'sh'
   # and 'ksh' in interactive mode and 'sh' (zsh-compat), 'bash' and 'ksh'
   # (zsh-compat) in non-interactive mode.
   ENV=/opt/Modules/init/profile.sh; export ENV
   BASH_ENV=/opt/Modules/init/bash; export BASH_ENV

   # enable completion only in interactive mode
   if [ ${BASH_VERSINFO:-0} -ge 3 ] && [[ $- =~ i ]] &&
      [ -r /opt/Modules/init/bash_completion ]; then
      source /opt/Modules/init/bash_completion
   fi

   if [[ ! ":$PATH:" =~ ':/opt/Modules/bin:' ]]; then
      PATH=/opt/Modules/bin${PATH:+:}$PATH; export PATH
   fi

   # initialize MANPATH if not set with a value that preserves manpath system
   # configuration even after addition of paths to this variable by modulefiles
   if [ ! -n "${MANPATH+x}" ]; then
      MANPATH=:; export MANPATH
   fi
   if [[ ! ":`manpath 2>/dev/null`:" =~ ':/opt/Modules/share/man:' ]]; then
      if [ "$MANPATH" = ':' ] || [ "$MANPATH" = '' ]; then
         _mlpathsep=''
      else
         _mlpathsep=:
      fi
      MANPATH=/opt/Modules/share/man$_mlpathsep$MANPATH; export MANPATH
      unset _mlpathsep
   fi
fi

unset _mlcode _mlret

# restore shell debugging options if disabled
if [ -n "${_mlshdbg:-}" ]; then
   set -$_mlshdbg;
   unset _mlshdbg;
fi;

As you can see all things are exported in, as far as I can make out, a correct manner. In particular, the module command is exported with the line: export -f module. There is nothing here to indicate why I get the response: module: command not found unless I source the /etc/profile.d/modules.sh or /etc/profile either directly on the command line or in ~/.bashrc although the initialisation file clearly is sourced by lightdm-session. Moreover, the software has been around for ages and I use is also on another machine running OpenSuse Leap 42.3 and earlier versions before that. Never did I have to source the initailisation file in an interactive terminal session if the initialisation file was put under /etc/profile.d/ which is done for you if repo packages are used. Now, when I compiled from source I had to do it myself, of course.

As far as I can make out lightdm is not doing this right, at least the startup is not done in a way one would expect. So what goes wrong here?

Cheers,
gostal

[orcus]

hi gostal,

Using a mint 19(.0) cinnamon vm for testing:

I grabbed your shell-script and placed it as /etc/b-init.sh (to avoid id does get sourced directly)
additionally I did add /etc/profile.d/b-init-wrappper.sh (to be able to redirect output and errors at a single point)

Code: Select all

oadm@mint-19-cinnamon-64bit:/etc/profile.d$ cat b-init-wrappper.sh 

Code: Select all

# only used to redirect debug stdout + stderr to /tmp
# moved b-init.sh (having the actual content) at /etc to avoid it being sourced directly
. /etc/b-init.sh >/tmp/$USER-dbg.txt 2>&1
echo "### current values after source-ing ####" >>/tmp/$USER-dbg.txt
echo "MODULES_CMD:'$MODULES_CMD'" >>/tmp/$USER-dbg.txt
echo "MODULES_USE_COMPAT_VERSION:'$MODULES_USE_COMPAT_VERSION'" >>/tmp/$USER-dbg.txt
echo "PATH:'$PATH'" >>/tmp/$USER-dbg.txt
echo "MANPATH:'$MANPATH'" >>/tmp/$USER-dbg.txt

and after a reboot I got

Code: Select all

oadm@mint-19-cinnamon-64bit:/tmp$ cat oadm-dbg.txt 
/etc/b-init.sh: line 37: /usr/bin/tclsh: No such file or directory
### current values after source-ing ####
MODULES_CMD:''
MODULES_USE_COMPAT_VERSION:''
PATH:'/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games'
MANPATH:''

Hope this does help a bit ... you might want to add further variables used in the original (or just append output from the original script) to the /tmp-file too to be able to "trace" the flow inside the sourced script.

Remarks @bash-version:
I did not really check the source of the script for other issues/glitches - but If you cant sort out obvious issues (like the above missing tclsh) - this might affect you too:

The orignial-script is seemingly a bit older => there has been a version-update of bash, which does impose some unexpected changes in the behavior (breaking compatibility of scripts if using borderline statements = most regular stuff should just work):

Example - with bash 4.2/4.3 one could use a statement like cd ~/Do* which does switch into the first matching folder at the users home-folder (= Documents // having both Documents and Downloads with a regular install) -> this does not work anymore at bash 4.4 (yep - I know ... almost "nobody" did use that, but there are more (numeros) changes of that sort).

Code: Select all

oadm@mint-18-3-cinnamon-64bit ~ $ bash --version
GNU bash, version 4.3.48(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2013 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
oadm@mint-18-3-cinnamon-64bit ~ $ cd ~/Do*
oadm@mint-18-3-cinnamon-64bit ~/Documents $ 

Code: Select all

oadm@mint-19-cinnamon-64bit:~$ bash --version
GNU bash, version 4.4.19(1)-release (x86_64-pc-linux-gnu)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>

This is free software; you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
oadm@mint-19-cinnamon-64bit:~$ cd ~/Do*
bash: cd: too many arguments
oadm@mint-19-cinnamon-64bit:~$ 

long story short - if you cant find any other obvious reasons which are fixable and still stuck at it's not working:
Setup a mint 17.3 vm - do NOT update it - and test if it's working there ... so you got something to be able to compare against

good luck - orcus

[gostal]

hi orcus,

Thanks again, for responding.

However, it's not really my shell-script. It belongs to the Environment Modules software. As tcl is required to run the software my system will not show the tclshell error. I also installed tcl-devel to build from source but that's not necessary to just run the software. Moreover, since the software initialises just fine in an interactive terminal session, there should not be any problem with the setting of the variables also when initialisation is done by way of /etc/profile. The problem lies in the exporting part somehow. I am not a shell script guru so I cannot off hand say if there is anything quaint going on in the bash initialisation script but I'm pretty sure the developers try not to do any non-standard things is as usability of the software is important. However, perhaps I can utilise your error tracing constructs to explore further.

I have limited time to conduct investigations of this kind. It's also a non-critical problem as it works just fine to initialise the software on the command line or via ~/.bashrc. Perhaps my next step will be to ask the developers for help. But first I think I'll do this bug report because if the repo package works fine under Ubuntu which I suppose it does since it is Ubuntu's package then there is a compatibility issue between Mint and Ubuntu which should fall on the Mint developers to sort out. There are reports out there regarding the failure of during boot initialisation, though, but most of them are pretty old.

Best wishes,
gostal

[orcus]

yup - good luck

cheers orcus

[gostal]

I have narrowed down the initialisation problem. Using now again the repo package: environment-modules Version: 4.1.1-1 with Ubuntu origin. As previously noted the package installs the file /etc/profile.d/modules.sh which in my case sources the bash initialisation file /usr/share/modules/init/bash. As also noted all these files are indeed sourced during graphical login. All defined environment variables are properly exported:

Code:

Code: Select all

printenv|grep MODULE
MODULES_CMD=/usr/lib/x86_64-linux-gnu/modulecmd.tcl
LOADEDMODULES=
GTK_MODULES=gail:atk-bridge
MODULEPATH=/etc/environment-modules/modules:/usr/share/modules/versions:/usr/share/modules/$MODULE_VERSION/modulefiles:/usr/share/modules/modulefiles
MODULEPATH_modshare=/usr/share/modules/$MODULE_VERSION/modulefiles:1:/etc/environment-modules/modules:1:/usr/share/modules/modulefiles:1:/usr/share/modules/versions:1
MODULESHOME=/usr/share/modules

However, several commands are also defined and exported as it would seem
/usr/share/modules/init/bash snip:

Code: Select all

# define module command and surrounding initial environment (default value
# for MODULESHOME, MODULEPATH, LOADEDMODULES and parse of init/.modulespath)
eval `${_mlre:-}/usr/bin/tclsh /usr/lib/x86_64-linux-gnu/modulecmd.tcl bash autoinit`

# clean temp variables used to setup quarantine
if [ -n "${_mlIFS+x}" ]; then
   IFS=$_mlIFS; unset _mlIFS;
else
   unset IFS;
fi;
unset _mlre _mlv _mlrv

# redefine module command if compat version has been activated
if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
   MODULES_CMD=/usr/lib/x86_64-linux-gnu/modulecmd-compat; export MODULES_CMD
   if [ -t 1 ]; then
      _moduleraw() { eval `/usr/lib/x86_64-linux-gnu/modulecmd-compat bash $*`; }
   else
      module() { eval `/usr/lib/x86_64-linux-gnu/modulecmd-compat bash $*`; }
   fi
fi

# export functions to get them defined in sub-shells
if [ -t 1 ]; then
   export -f _moduleraw
fi
export -f module

# define function to switch between C and Tcl versions of Modules
switchml() {
   typeset swfound=1
   if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
      typeset swname='main'
      if [ -e /usr/lib/x86_64-linux-gnu/modulecmd.tcl ]; then
         typeset swfound=0
         unset MODULES_USE_COMPAT_VERSION
      fi
   else
      typeset swname='compatibility'
      if [ -e /usr/lib/x86_64-linux-gnu/modulecmd-compat ]; then
         typeset swfound=0
         MODULES_USE_COMPAT_VERSION=1; export MODULES_USE_COMPAT_VERSION
      fi
   fi

   # switch version only if command found
   if [ $swfound -eq 0 ]; then
      echo "Switching to Modules $swname version"
      source /usr/share/modules/init/bash
   else
      echo "Cannot switch to Modules $swname version, command not found"
      return 1
   fi
}
export -f switchml

It turns out that in my case the firstly defined command module gets redefined. It is the exporting of the shell commands that for some reason beyond me fails and is the reason why they have to be defined in the terminal session.

Before sourcing on the command line:

Code: Select all

:~$ printenv|grep BASH_FUNC
:~$

After:

Code: Select all

 . /etc/profile.d/modules.sh
gostal@ki003401-linux:~$ printenv|grep BASH_FUNC
BASH_FUNC_module%%=() {  _moduleraw "$*" 2>&1
BASH_FUNC_switchml%%=() {  typeset swfound=1;
BASH_FUNC__moduleraw%%=() {  unset _mlre _mlIFS _mlshdbg;

grep omits lines not containing BASH_FUNC. The entire definitions are:

Code: Select all

BASH_FUNC_module%%=() {  _moduleraw "$*" 2>&1
}
BASH_FUNC_switchml%%=() {  typeset swfound=1;
 if [ "${MODULES_USE_COMPAT_VERSION:-0}" = '1' ]; then
 typeset swname='main';
 if [ -e /usr/lib/x86_64-linux-gnu/modulecmd.tcl ]; then
 typeset swfound=0;
 unset MODULES_USE_COMPAT_VERSION;
 fi;
 else
 typeset swname='compatibility';
 if [ -e /usr/lib/x86_64-linux-gnu/modulecmd-compat ]; then
 typeset swfound=0;
 MODULES_USE_COMPAT_VERSION=1;
 export MODULES_USE_COMPAT_VERSION;
 fi;
 fi;
 if [ $swfound -eq 0 ]; then
 echo "Switching to Modules $swname version";
 source /usr/share/modules/init/bash;
 else
 echo "Cannot switch to Modules $swname version, command not found";
 return 1;
 fi
}
BASH_FUNC__moduleraw%%=() {  unset _mlre _mlIFS _mlshdbg;
 if [ "${MODULES_SILENT_SHELL_DEBUG:-0}" = '1' ]; then
 case "$-" in 
 *v*x*)
 set +vx;
 _mlshdbg='vx'
 ;;
 *v*)
 set +v;
 _mlshdbg='v'
 ;;
 *x*)
 set +x;
 _mlshdbg='x'
 ;;
 *)
 _mlshdbg=''
 ;;
 esac;
 fi;
 if [ -n "${IFS+x}" ]; then
 _mlIFS=$IFS;
 fi;
 IFS=' ';
 for _mlv in ${MODULES_RUN_QUARANTINE:-};
 do
 if [ "${_mlv}" = "${_mlv##*[!A-Za-z0-9_]}" -a "${_mlv}" = "${_mlv#[0-9]}" ]; then
 if [ -n "`eval 'echo ${'$_mlv'+x}'`" ]; then
 _mlre="${_mlre:-}${_mlv}_modquar='`eval 'echo ${'$_mlv'}'`' ";
 fi;
 _mlrv="MODULES_RUNENV_${_mlv}";
 _mlre="${_mlre:-}${_mlv}='`eval 'echo ${'$_mlrv':-}'`' ";
 fi;
 done;
 if [ -n "${_mlre:-}" ]; then
 _mlre="eval ${_mlre}";
 fi;
 eval `${_mlre:-}/usr/bin/tclsh /usr/lib/x86_64-linux-gnu/modulecmd.tcl bash $*`;
 _mlstatus=$?;
 if [ -n "${_mlIFS+x}" ]; then
 IFS=$_mlIFS;
 else
 unset IFS;
 fi;
 if [ -n "${_mlshdbg:-}" ]; then
 set -$_mlshdbg;
 fi;
 unset _mlre _mlv _mlrv _mlIFS _mlshdbg;
 return $_mlstatus
}

Theese are actually picked up by ~/.xsession-errors which I've posted earlier. The following lines in this file seem to be informative

Code: Select all

dbus-update-activation-environment: setting _=/usr/bin/dbus-update-activation-environment
dbus-update-activation-environment: warning: error sending to systemd: org.freedesktop.DBus.Error.InvalidArgs: Invalid environment assignments

But I have no clue whatsoever what to do about it. Ideas, anyone?

Cheers,
gostal

[gostal]

I have appended [solved?] to the thread title as the following is more of a workaround and not a fool proof solution.

I stumbled upon the reason why the Environment Module commands do not export properly during graphical login. It turns out that the standard command interpreter /bin/dash doesn't have this functionality as it seems. Some of the Environment Module shell scripts have the shebang #!/bin/sh so I got it into my head to do
ll /bin/sh
and I got
/bin/sh -> dash*
whereas I had expected
/bin/sh -> bash*
as this was the shell of my choice when I installed Mint.

To be quite frank I had no idea about dash so I googled for a bit and learned a little about Debian Almquist Shell and got the impression that bash is more competent but can be slower and that the primary reason for Debian and relatives to use dash is because it is somewhat faster than bash at the cost of far less functionality. As bash apparently has more functionality I tested to let /bin/sh point to /bin/bash instead of /bin/dash and this was all it took. Log out - log in and Environment Modules were properly initialised:

Code: Select all

printenv|grep BASH_FUNC
BASH_FUNC_module%%=() {  unset _mlre _mlIFS _mlshdbg;
BASH_FUNC_switchml%%=() {  swfound=1;

and no more module: command not found! A side effect is that apparently the module command doesn't need to be redefined.

However, replacing dash with bash in this fashion means that I have changed the basic behaviour of the system which potentially may cause other problems. I guess a safe workaround would be to replace all #!/bin/sh with #!/bin/bash so perhaps I'll do that later but there is also another softlink: /bin/sh.distrib -> dash* which I hope will save me from any serious problems so until I find myself in deep water because of my system change I'll leave it be.

Perhaps needless to say is that the double report of system modules, see the first post, is still there and now I have confirmed that the same package version (4.1.1-1) installed in an Ubuntu system does not display this behaviour so the incompatibility between Ubuntu and Mint regarding this package is now firmly established.

Edit: I didn't do my homework regarding Ubuntu. It turns out that the Environment Modules installation on my colleague's Ubuntu 18.04-machine had been tweaked to remove the double report. So Mint is not to blame in this respect. It all falls on Ubuntu. It's actually not all that hard to remove the double report. The package installs a file: /etc/environment-modules/modulespath with the following content:

Code: Select all

...
/etc/environment-modules/modules
/usr/share/modules/versions                             # location of version files
/usr/share/modules/$MODULE_VERSION/modulefiles  # Module pkg modulefiles (if versioning)
/usr/share/modules/modulefiles                          # General module files

except for comment lines. The last two of these lines come out to the same thing if the variableMODULE_VERSION is not set, which it isn't by default. echo $MODULE_VERSION gives nothing. So all that needs to be done is to remove(comment out) the line containing the variable.

I've been in contact with both the Ubuntu people:
https://lists.ubuntu.com/archives/ubunt ... html#18338
and via mail with the original Sourceforge source maintainer who investigated the issue and reproduced it on virtual Ubuntu and Mint machines. It turns out that /usr/sbin/lightdm-session is run by bash but at the end it says exec $@ which launches /bin/sh-scripts. If /bin/sh then points to /bin/dash it doesn't help if /bin/sh-scripts in the package are converted to /bin/bash-scripts since dash by design washes any defined shell functions out, module being a case in point.
End edit

Cheers,
gostal