Environment Modules Updates

Symbolic version to designate module loaded version

2021-01-05T05:59:00.001Z

When the Advanced module version specifiers is enabled, the loaded symbolic version may be used to designate the currently loaded version of specified module.

$ ml display foo@loaded
-------------------------------------------------------------------
/path/to/modulefiles/foo/1.0:
module-whatis foo/1.0
-------------------------------------------------------------------

If no version of specified module can be found loaded, an error is returned.

$ ml display foo@loaded
ERROR: No loaded version found for 'foo' module

Determining module implementation and version

2020-12-29T09:37:00.001Z

New Modules variables are introduced to determine during the evaluation of a modulefile or a modulerc what module implementation is currently in use. The ModuleTool variable corresponds to the name of the module implementation and is set to Modules for this project. The ModuleToolVersion variable corresponds to the version number of the implementation (e.g. 4.7.0).

With these new variables it is possible to precisely know what module command is in use then adapt modulefile code to handle a specific behavior or leverage a new feature.

The modulefile command versioncmp is also introduced to provide a simple way to compare two version strings and return if first version string is less than, equal to or greater than second one.

if {[info exists ModuleTool] && $ModuleTool eq {Modules}
&& [versioncmp $ModuleToolVersion 4.7] >= 0} {
# here some code specific for Modules 4.7 and later versions
}

The ModuleTool and ModuleToolVersion variables and the versioncmp modulefile command are supported by the Lmod project starting version 8.4.8.

Modules 4.6.1 released

2020-11-14T20:30:00.001Z

Modules 4.6.1 is out. This release brings a significant performance improvement for the dynamic hiding/forbidding feature and several other fixes.

The changes introduced in this bugfix release are:

Lib: implement initStateClockSeconds as a Tcl command in libtclenvmodules to provide an optimized way to retrieve current Epoch time.
Lib: implement parseDateTimeArg as a Tcl command in libtclenvmodules to provide an optimized way to convert a datetime string into an Epoch time.
When full module specification is equal to @, raise an error as no module name is provided. (fix issue #362)
Optimize internal recording of hidden module and tag specification when parsing modulerc files in order to reduce the time taken to test if a given module is hidden or if a given tag applies to it.
Script: add the ability to select the benchmark test to perform on mb utility.
Doc: add Use new features without breaking old module command cookbook recipe
Doc: rework option description for module-hide and module-forbid commands in modulefile(4) document.
Doc: describe in diff_v3_v4 document that shell special characters like backticks are escaped when used in values starting Modules 4.0. (fix issue #365)
Doc: make the ENVIRONMENT section from modulefile(4) man page point to the ENVIRONMENT section of module(1) man page.
Fix clear sub-command to unset the MODULES_LMSOURCESH environment variable. (fix issue #367)
Correctly return on avail sub-command a symbolic version defined in a global RC file when specifically searched. (fix issue #368)
Fix module hiding resolution for symbolic versions defined in a global RC file when module-hide statements are set in the modulepath where the modulefiles targeted by these symbols are located. (fix issue #369)
When a module fails to unload during a purge sub-command, preserve loaded the modules it requires to keep environment consistent. (fix issue #370)
Doc: add Hide and forbid modules cookbook recipe.

The list of all changes made in this new version are described in the NEWS document. Details on the new features introduced in the 4.6 series are provided in the MIGRATING document.

http://modules.readthedocs.io/en/stable/MIGRATING.html
http://modules.readthedocs.io/en/stable/NEWS.html

The tarball of this new version can be downloaded at:

http://downloads.sourceforge.net/modules/modules-4.6.1.tar.gz

The zipball to install this new version on Windows platform can be downloaded at:

http://downloads.sourceforge.net/modules/modules-4.6.1-win.zip

If you encounter any issue, please let us know by creating a ticket on the project bug tracker at:

https://github.com/cea-hpc/modules/issues

Special thanks to to all the people who reported issues.

Modules 4.6.0 released

2020-09-16T05:04:00.001Z

Modules 4.6.0 is here with many new stuff to handle the life cycle of your modulefiles. It is as always backward-compatible with previous version 4 releases and primarily adds features and fixes bugs. Major new features introduced in v4.6.0 since v4.5 are:

Details on these features included in this new release are provided in the MIGRATING document. The list of all changes made in this new version are described in the NEWS document.

The tarball of this new version can be downloaded at:

http://downloads.sourceforge.net/modules/modules-4.6.0.tar.gz

The zipball to install this new version on Windows platform can be downloaded at:

http://downloads.sourceforge.net/modules/modules-4.6.0-win.zip

If you encounter any issue, please let us know by creating a ticket on the project bug tracker at:

https://github.com/cea-hpc/modules/issues

Many thanks also to all of you who improve Modules by reporting bugs or sharing new ideas.

Tracing module execution

2020-09-15T04:21:00.001Z

The trace verbosity is introduced between the verbose and debug levels to report details on module searches, resolutions, selections and evaluations. Trace mode can be enabled by setting the verbosity config to the trace value or by using the -T/--trace command-line switches.

To specifically render trace messages, the tr key is added to the color palette with a default value of 2 (decreased intensity).

$ ml -T foo
Evaluate modulerc: '/path/to/modulefiles/.modulerc'
Get modules: {foo} matching 'foo' in '/path/to/modulefiles'
Resolve: 'foo' into 'bar'
Get modules: {bar bar/1.0} matching 'bar' in '/path/to/modulefiles'
Select module: 'bar/1.0' (/path/to/modulefiles/bar/1.0) matching 'bar/1.0'
Loading bar/1.0
Evaluate modulefile: '/path/to/modulefiles/bar/1.0' as 'bar/1.0'

Forbidding use of modules

2020-09-08T19:04:00.001Z

The module-forbid modulefile command is added to dynamically forbid the evaluation of modulefiles it specifies. When forbidden, a module cannot be loaded and an access error is returned when an attempt is made to evaluate it.

$ cat /path/to/modulefiles/foo/.modulerc
#%Module4.6
module-forbid foo@1:
$ ml foo/1.0
ERROR: Access to module 'foo/1.0' is denied
$ ml
No Modulefiles Currently Loaded.

module-forbid statements can be coupled with :mfcmd:module-hide statements to hide modules in addition to forbid their use. module-forbid supports the --not-user, --not-group, --before and --after options to still allow some users or forbid modules before or after a given date time.

An additional error message can be defined with the --message option to guide for instance users when they try to evaluate a forbidden module:

$ cat /path/to/modulefiles/bar/.modulerc
#%Module4.6
module-forbid --message {Software bar/1.0 is decommissioned, please now use\
bar/2.0} --after 2020-09-01 bar/1.0
$ ml bar/1.0
ERROR: Access to module 'bar/1.0' is denied
Software bar/1.0 is decommissioned, please now use bar/2.0

When an evaluated module will soon be forbidden, a message is returned to the user to warn him/her of the near limit. An additional warning message can also be defined here with the --nearly-message option to guide users.

$ cat /path/to/modulefiles/qux/.modulerc
#%Module4.6
module-forbid --nearly-message {Version 1.0 will soon expire, please now use\
version 2.0} --after 2020-09-15 qux/1.0
$ date
Tue 08 Sep 2020 06:49:43 AM CEST
$ ml qux/1.0
Loading qux/1.0
WARNING: Access to module will be denied starting '2020-09-15'
Version 1.0 will soon expire, please now use version 2.0

The range of time the nearly forbidden warning appears can be controlled with the nearly_forbidden_days configuration option, whose value equals to the number of days prior the module starts to be forbidden. This configuration is set to 14 (days) by default and this value can be controlled at configure time with --with-nearly-forbidden-days option. When the nearly_forbidden_days configuration is set through the config sub-command, the MODULES_NEARLY_FORBIDDEN_DAYS environment variable is set.

Hiding modules

2020-09-03T06:17:00.001Z

The newly introduced module-hide modulefile command enables to dynamically hide modulefiles, module aliases or symbolic versions specified to it:

$ cat /path/to/modulefiles/bar/.modulerc
#%Module4.6
module-version bar/1.0 old
# hide 'old' symbolic version
module-hide bar/old
# hide all version 2 and above
module-hide bar@2:
$ cat /path/to/modulefiles/.modulerc
#%Module4.6
# hide all versions of foo module
module-hide foo

module-hide commands should be placed in module rc files and can leverage the Advanced module version specifiers syntax as shown in the above example.

Hidden modules are excluded from available module search or module selection unless query refers to hidden module by its exact name:

$ ml av
--------------- /path/to/modulefiles ---------------
bar/1.0 bar/2.0
$ module load -v foo
ERROR: Unable to locate a modulefile for 'foo'
$ module load -v foo/1.0
Loading foo/1.0
$ module avail bar/old
--------------- /path/to/modulefiles ---------------
bar/1.0(old)

module-hide command accepts a --soft option to apply a lighter of hiding to modules:

$ cat /path/to/modulefiles/qux/.modulerc
#%Module4.6
# softly hide all qux modules
module-hide --soft qux

The soft hiding mode enables to hide modules from full availability listing yet keeping the ability to select such module for load without having to use module exact name:

$ ml av
--------------- /path/to/modulefiles ---------------
bar/1.0 bar/2.0
$ ml av qux
--------------- /path/to/modulefiles ---------------
qux/1.0 qux/2.0
$ module load -v qux
Loading qux/2.0

Alternatively, a --hard option can be set on module-hide command to ensure designated modules do not unveil even if referred by their exact name:

$ cat /path/to/modulefiles/qux/.modulerc
#%Module4.6
# softly hide all qux modules
module-hide --soft qux
# set highest version of qux hard hidden
module-hide --hard qux/3.0
$ ml av qux/3.0
$ ml qux/3.0
ERROR: Unable to locate a modulefile for 'qux/3.0'

Some users or groups can be set unaffected by hiding mechanism with the --not-user or --not-group options:

$ cat /path/to/modulefiles/quuz/.modulerc
#%Module4.6
# hiding does not apply to grp1 and grp2 groups
module-hide --not-group {grp1 grp2} quuz
$ id --groups --name
grp1 grp7
$ ml av quuz
--------------- /path/to/modulefiles ---------------
quuz/1.0 quuz/2.0
$ ml -v quuz
Loading quuz/2.0

Hiding mechanism can also be set effective only before or after a given date time with the --before and --after options. Accepted date time format is YYYY-MM-DD[THH:MM].

$ cat /path/to/modulefiles/fum/.modulerc
#%Module4.6
# hide only before a given date
module-hide --hard --before 2020-09-01T12:00 fum/1.0
# hide only after a given date
module-hide --hard --after 2020-09-01 fum/2.0
$ date
Fri 04 Sep 2020 06:21:48 AM CEST
$ ml av fum
--------------- /path/to/modulefiles ---------------
fum/1.0

Hidden modules can be included in available module searches if option --all/-a is set on avail, aliases, whatis or search sub-commands. Hard hidden modules are unaffected by this option and stay hidden.

$ ml av -a
--------------- /path/to/modulefiles ---------------
bar/1.0(old) foo/1.0 fum/1.0 quuz/2.0 qux/2.0
bar/2.0 foo/2.0 quuz/1.0 qux/1.0

Modules 4.5.3 released

2020-08-31T18:44:00.001Z

Version 4.5.3 of Modules has just been released. It provides fixes for the few issues recently spotted.

The changes introduced in this bugfix release are:

Install: take into account the --build, --host, --target, --enable-dependency-tracking and --disable-dependency-tracking configure options to transmit them to the configure scripts of Modules Tcl extension library and Modules compatibility version. (fix issue #354)
Install: ignore some regular options of an Autoconf configure script that are useless for this project but usually implied in build macros (like RPM %configure macro).
Install: ignore unsupported --enable-* and --with-* options on configure script rather raise an error and add support to define environment variable and build system type as configure script arguments to comply with GNU configuration recommendations.
Install: fix modulecmd pre-alternatives check in RPM spec file.
Install: use %make_build and %make_install macros in RPM spec file.
When module switch command is used in modulefile, do not state when processing it a conflict over switched-off module if its specification on the module switch command also matches switched-on module’s specification. Allow this way the replacement of any loaded version of a module for a specific one required by currently loading module. (fix issue #355)
Correctly report failed attempts to load module requirements expressed with advanced version specifiers. (fix issue #356)

The list of all changes made in this new version are described in the NEWS document. Details on the new features introduced in the 4.5 series are provided in the MIGRATING document.

http://modules.readthedocs.io/en/stable/MIGRATING.html
http://modules.readthedocs.io/en/stable/NEWS.html

The tarball of this new version can be downloaded at:

http://downloads.sourceforge.net/modules/modules-4.5.3.tar.gz

The zipball to install this new version on Windows platform can be downloaded at:

http://downloads.sourceforge.net/modules/modules-4.5.3-win.zip

If you encounter any issue, please let us know by creating a ticket on the project bug tracker at:

https://github.com/cea-hpc/modules/issues

Special thanks to to all the people who reported issues.

Modules 4.5.2 released

2020-07-30T12:48:00.001Z

Version 4.5.2 of Modules has just been released. It provides fixes for the few issues recently spotted.

The changes introduced in this bugfix release are:

Init: list and source sub-commands do not take available modules as argument in fish completion script.
Init: fix option list for search sub-command in bash completion script.
Fix double error counter increase when modulefile evaluation breaks.
Install: adapt configure script to pass to the configure script of Modules compatibility version only a subset of the options it supports (most commonly used options).
Install: raise an error when an unknown option is passed to configure script rather silently ignore it. (fix issue #348)
Install: enable the definition of installation directory options of configure script with the --option value syntax in addition to the --option=value syntax. (fix issue #348)
Doc: alphabetically sort sub-commands of module-info modulefile Tcl command in modulefile(4) document.
Script: clean previously built environment-modules RPMs in mrel.
Clearly separate quarantine variable definition from tclsh binary on modulecmd.tcl evaluated command call in _module_raw function for sh, bash, ksh and zsh shells. (fix issue #350)
Doc: clarify in documentation index that Environment Modules should not be confused with language-specific modules. (contribution from Rob Hurt)
Adapt conflict detection tests to ensure a module loaded by its full pathname will not detect itself as a conflict when declaring a reflexive conflict. (fix issue #352)
Adapt the mrel and mpub commands to produce new Modules release from a vZ.Y.x git branch rather than from the repository main branch.

The list of all changes made in this new version are described in the NEWS document. Details on the new features introduced in the 4.5 series are provided in the MIGRATING document.

http://modules.readthedocs.io/en/stable/MIGRATING.html
http://modules.readthedocs.io/en/stable/NEWS.html

The tarball of this new version can be downloaded at:

http://downloads.sourceforge.net/modules/modules-4.5.2.tar.gz

The zipball to install this new version on Windows platform can be downloaded at:

http://downloads.sourceforge.net/modules/modules-4.5.2-win.zip

If you encounter any issue, please let us know by creating a ticket on the project bug tracker at:

https://github.com/cea-hpc/modules/issues

Special thanks to Rob Hurt for his code contribution on this release and to all the people who reported issues.

Querying user's name and groups membership

2020-07-24T12:52:00.001Z

Two new sub-commands are introduced for the module-info modulefile command: username and usergroups. They respectively fetch the name of the user currently running modulecmd.tcl or the name of all the groups this user is member of.

These two new modulefile commands can help to adapt code to specific users or groups. Like for instance to instantiate a modulefile for each group the user is member of:

$ cat /path/to/modulefiles/foo/.modulerc
#%Module
foreach grp [module-info usergroups] {
module-virtual foo/$grp .common
}
$ id -G -n
grp1 grp2 grp3
$ module avail
--------------- /path/to/modulefiles ---------------
foo/grp1 foo/grp2 foo/grp3

username and usergroups sub-commands of module-info modulefile command are only supported on Unix platform.