Skip to content

Commit fea96e1

Browse files
jsquyresjsquyres
committed
docs: Better explain required versions of support libraries
Get rid of the table that listed the required support library versions -- the table format was too limiting. Instead, add a bunch more verbiage about the versions that are included in the Open MPI distribution tarball and the minimum required versions of each of hwloc, libevent, PMIx, and PRRTE. Pay special attention to the corner cases of building PMIx (internal and external), with and without PRRTE. Also set the minimum required versions for PMIx and PRRTE in VERSIONS: * Testing shows that Open MPI v5.0.x requires at least PMIx v4.2.0 * The minimum required version for PRRTE is v3.0.0, but we recommend in the docs that users use >=v3.0.1 so that they get a full mpirun(1) man page Finally, also show in the docs the versions of the embedded packages (hwloc, libevent, PMIx, and PRRTE). This required adding a little Python in docs/conf.py to read VERSION files and extract version numbers from tarball filenames. Signed-off-by: Jeff Squyres <jeff@squyres.com>
1 parent 993f004 commit fea96e1

File tree

4 files changed

+178
-79
lines changed

4 files changed

+178
-79
lines changed

‎VERSION

+2-2
Original file line numberDiff line numberDiff line change
@@ -25,8 +25,8 @@ mpi_standard_subversion=1
2525

2626
# OMPI required dependency versions.
2727
# List in x.y.z format.
28-
pmix_min_version=4.1.2
29-
prte_min_version=2.0.2
28+
pmix_min_version=4.2.0
29+
prte_min_version=3.0.0
3030
hwloc_min_version=1.11.0
3131
event_min_version=2.0.21
3232
automake_min_version=1.13.4

‎docs/Makefile.am

+2-2
Original file line numberDiff line numberDiff line change
@@ -1032,8 +1032,8 @@ $(ALL_MAN_BUILT):
10321032
filename=`basename $$file`; \
10331033
cp -pf $(OMPI_PRRTE_RST_CONTENT_DIR)/$$filename "$(builddir)/prrte-rst-content"; \
10341034
done
1035-
$(OMPI_V_SPHINX_HTML) OMPI_VERSION_FILE=$(top_srcdir)/VERSION $(SPHINX_BUILD) -M html "$(builddir)" "$(OUTDIR)" $(SPHINX_OPTS)
1036-
$(OMPI_V_SPHINX_MAN) OMPI_VERSION_FILE=$(top_srcdir)/VERSION $(SPHINX_BUILD) -M man "$(builddir)" "$(OUTDIR)" $(SPHINX_OPTS)
1035+
$(OMPI_V_SPHINX_HTML) OMPI_TOP_SRCDIR=$(top_srcdir) $(SPHINX_BUILD) -M html "$(builddir)" "$(OUTDIR)" $(SPHINX_OPTS)
1036+
$(OMPI_V_SPHINX_MAN) OMPI_TOP_SRCDIR=$(top_srcdir) $(SPHINX_BUILD) -M man "$(builddir)" "$(OUTDIR)" $(SPHINX_OPTS)
10371037

10381038
# A useful rule to invoke manually to ensure that all of the external
10391039
# HTML links we have are valid. Running this rule requires

‎docs/conf.py

+76-32
Original file line numberDiff line numberDiff line change
@@ -14,46 +14,85 @@
1414

1515
# -- Project information -----------------------------------------------------
1616

17+
import os
18+
import re
1719
import datetime
20+
1821
year = datetime.datetime.now().year
1922

2023
project = 'Open MPI'
2124
copyright = f'2003-{year}, The Open MPI Community'
2225
author = 'The Open MPI Community'
2326

24-
# The full version, including alpha/beta/rc tags
25-
# Read the Open MPI version from the VERSION file in the source tree
26-
# The docs/Makefile.am will set the env var OMPI_VERSION_FILE, because
27-
# we might be doing a VPATH build.
28-
filename = None
29-
if 'OMPI_VERSION_FILE' in os.environ:
30-
filename = os.environ['OMPI_VERSION_FILE']
31-
elif os.path.exists("../VERSION"):
32-
filename = '../VERSION'
33-
34-
if filename is None:
35-
print("ERROR: Could not find Open MPI source tree VERSION file")
36-
exit(1)
37-
38-
with open(filename) as fp:
39-
ompi_lines = fp.readlines()
40-
41-
ompi_data = dict()
42-
for ompi_line in ompi_lines:
43-
if '#' in ompi_line:
44-
parts = ompi_line.split("#")
45-
ompi_line = parts[0]
46-
ompi_line = ompi_line.strip()
47-
48-
if '=' not in ompi_line:
49-
continue
27+
# ---------------------------
5028

51-
ompi_key, ompi_val = ompi_line.split("=")
52-
ompi_data[ompi_key.strip()] = ompi_val.strip()
29+
# The docs/Makefile.am will set the env var OMPI_TOP_SRCDIR, because
30+
# we might be doing a VPATH build.
31+
ompi_top_srcdir = '..'
32+
if 'OMPI_TOP_SRCDIR' in os.environ:
33+
ompi_top_srcdir = os.environ['OMPI_TOP_SRCDIR']
34+
35+
# Read an Open MPI-style VERSION file
36+
def read_version_file(path):
37+
if not os.path.exists(path):
38+
print(f"ERROR: Unable to find file {path}")
39+
exit(1)
40+
41+
with open(path) as fp:
42+
version_lines = fp.readlines()
43+
44+
data = dict()
45+
for line in version_lines:
46+
if '#' in line:
47+
parts = line.split("#")
48+
line = parts[0]
49+
line = line.strip()
50+
51+
if '=' not in line:
52+
continue
53+
54+
key, val = line.split("=")
55+
data[key.strip()] = val.strip()
56+
57+
return data
58+
59+
# Look for a version string via a regular expresion of a filename in a
60+
# given directory
61+
def get_tarball_version(path, expr):
62+
if not os.path.exists(path):
63+
print(f"ERROR: Unable to find path {path}")
64+
exit(1)
65+
66+
for file in os.listdir(path):
67+
m = re.match(expr, file)
68+
if not m:
69+
continue
70+
return m.group(1)
71+
72+
return ""
73+
74+
# Read all the various versions from the source tree
75+
76+
ompi_data = read_version_file(f"{ompi_top_srcdir}/VERSION")
77+
pmix_data = read_version_file(f"{ompi_top_srcdir}/3rd-party/openpmix/VERSION")
78+
prte_data = read_version_file(f"{ompi_top_srcdir}/3rd-party/prrte/VERSION")
79+
80+
hwloc_embedded_version = get_tarball_version(f"{ompi_top_srcdir}/3rd-party/",
81+
r"hwloc-(.*).tar")
82+
event_embedded_version = get_tarball_version(f"{ompi_top_srcdir}/3rd-party/",
83+
r"libevent-(.*)-stable.tar")
84+
85+
# ---------------------------
86+
87+
# Assemble several different combinations of version strings
5388

5489
ompi_series = f"v{ompi_data['major']}.{ompi_data['minor']}.x"
5590
ompi_ver = f"v{ompi_data['major']}.{ompi_data['minor']}.{ompi_data['release']}{ompi_data['greek']}"
5691

92+
pmix_embedded_version = f"v{pmix_data['major']}.{pmix_data['minor']}.{pmix_data['release']}{pmix_data['greek']}"
93+
prte_embedded_version = f"v{prte_data['major']}.{prte_data['minor']}.{prte_data['release']}{prte_data['greek']}"
94+
prte_embedded_series = f"v{prte_data['major']}.{prte_data['minor']}"
95+
5796
pmix_min_version = f"{ompi_data['pmix_min_version']}"
5897
prte_min_version = f"{ompi_data['prte_min_version']}"
5998
hwloc_min_version = f"{ompi_data['hwloc_min_version']}"
@@ -86,7 +125,6 @@
86125
# If we're building in an RTD environment for a tag or external (i.e.,
87126
# PR), use the RTD version -- not what we just read from the VERSIONS
88127
# file.
89-
import os
90128
key = 'READTHEDOCS'
91129
if key in os.environ and os.environ[key] == 'True':
92130
print("OMPI: found ReadTheDocs build environment")
@@ -172,9 +210,6 @@
172210

173211
# -- Options for MAN output -------------------------------------------------
174212

175-
import os
176-
import re
177-
178213
# Dynamically find all the man pages and build the appropriate list of
179214
# tuples so that we don't have to manually maintain it.
180215

@@ -222,9 +257,14 @@ def _doit(topdir):
222257
.. |ompi_ver| replace:: {ompi_ver}
223258
.. |ompi_series| replace:: {ompi_series}
224259
.. |pmix_min_version| replace:: {pmix_min_version}
260+
.. |pmix_embedded_version| replace:: {pmix_embedded_version}
225261
.. |prte_min_version| replace:: {prte_min_version}
262+
.. |prte_embedded_version| replace:: {prte_embedded_version}
263+
.. |prte_embedded_series| replace:: {prte_embedded_series}
226264
.. |hwloc_min_version| replace:: {hwloc_min_version}
265+
.. |hwloc_embedded_version| replace:: {hwloc_embedded_version}
227266
.. |event_min_version| replace:: {event_min_version}
267+
.. |event_embedded_version| replace:: {event_embedded_version}
228268
.. |automake_min_version| replace:: {automake_min_version}
229269
.. |autoconf_min_version| replace:: {autoconf_min_version}
230270
.. |libtool_min_version| replace:: {libtool_min_version}
@@ -234,6 +274,10 @@ def _doit(topdir):
234274
.. |mpi_standard_minor_version| replace:: {mpi_standard_minor_version}
235275
.. |deprecated_favor| replace:: this routine is deprecated in favor of
236276
277+
.. |br| raw:: html
278+
279+
<br />
280+
237281
"""
238282

239283
# The sphinx_rtd_theme does not properly handle wrapping long lines in

‎docs/installing-open-mpi/required-support-libraries.rst

+98-43
Original file line numberDiff line numberDiff line change
@@ -3,50 +3,28 @@
33
Required support libraries
44
==========================
55

6-
Open MPI requires the following support libraries with the minimum listed versions:
7-
8-
.. list-table::
9-
:header-rows: 1
10-
11-
* - Library
12-
- Minimum version
13-
- Notes
14-
* - `Hardware Locality <https://www.open-mpi.org/projects/hwloc/>`_
15-
- |hwloc_min_version|
16-
- This library is required; Open MPI will not build without it.
17-
* - `Libevent <https://libevent.org/>`_
18-
- |event_min_version|
19-
- This library is required; Open MPI will not build without it.
20-
* - `PMIx <https://pmix.org/>`_
21-
- |pmix_min_version|
22-
- This library is required; Open MPI will not build without it.
23-
* - `PRRTE <https://github.com/openpmix/prrte>`_
24-
- |prte_min_version|
25-
- This library is optional in some environments. PRRTE provides
26-
Open MPI's full-featured ``mpirun`` / ``mpiexec`` MPI
27-
application launchers (the two are identical; they are symbolic
28-
links to the same executable).
29-
30-
* If your environment uses another MPI application launcher
31-
(e.g., Slurm users can use the ``srun`` launcher to "direct
32-
launch" Open MPI applications), then the use of PRRTE is
33-
optional.
34-
* If your environment has no other MPI application launcher, then
35-
you need to install PRRTE and build Open MPI with PRRTE
36-
support.
37-
* Open MPI can use the copy of PRRTE embedded in its source
38-
code tree, or compile/link against an external PRRTE
39-
installation. :ref:`See this section for details about how
40-
to specify each method
41-
<label-building-ompi-cli-options-required-support-libraries>`.
42-
43-
Since these support libraries are fundamental to Open MPI's operation
44-
and not universally available in all environments, they are directly
6+
7+
While Open MPI can be built with support for a wide variety of
8+
systems, a small set of support libraries are *required* in order to
9+
build Open MPI in *any* environment. Several of these packages are
10+
both fundamental to Open MPI's operation and not universally available
11+
in all environments. As such, these "fundamental" packages are both
12+
embedded in Open MPI's distribution tarballs and also directly
4513
incorporated into Open MPI's configure, build, and installation
46-
process. More on this below.
14+
process.
15+
16+
:ref:`See below
17+
<required-support-libraries-configure-discovery-label>` for a
18+
description of how Open MPI chooses whether to use the embedded
19+
versions of these packages or versions already installed on your
20+
system.
21+
22+
* `Hardware Locality <https://www.open-mpi.org/projects/hwloc/>`_
4723

48-
.. note:: The versions listed in this table are the *minimum* versions needed. In general, the Open MPI community recommends using more recent versions of both the :ref:`required support libraries <label-install-required-support-libraries>` and any other optional support libraries. This is because more recent versions typically tend to include bug fixes, sometimes affecting Open MPI functionality. As a specific example, there is a known issue with `Hardware Locality <https://www.open-mpi.org/projects/hwloc/>`_ releases older than v2.8.0 on systems with Intel Ponte Vecchio accelerators. If you run Open MPI on such systems, you need to use Hwloc v2.8.0 or newer, or you will experience undefined behavior.
49-
This effect is not unique to the Hardware Locality library; this is why the Open MPI community recommends using as recent as possible versions of all support libraries.
24+
* This library is required; Open MPI will not build without it.
25+
* **Minimum version required:** |hwloc_min_version|
26+
* **Version embedded in Open MPI distribution:**
27+
|hwloc_embedded_version|
5028

5129
.. danger:: As of |ompi_ver|, Open MPI does not yet support the
5230
Hwloc v3.x series (which may not even be available at
@@ -77,6 +55,81 @@ process. More on this below.
7755
uses Hwloc, it uses the *same* Hwloc with which Open MPI
7856
was compiled.
7957

58+
* `Libevent <https://libevent.org/>`_
59+
60+
* This library is required; Open MPI will not build without it.
61+
* **Minimum version required:** |event_min_version|
62+
* **Version embedded in Open MPI distribution:**
63+
|event_embedded_version|
64+
65+
* `PMIx <https://pmix.org/>`_
66+
67+
* This library is required; Open MPI will not build without it.
68+
* **Minimum version required when building without PRRTE:**
69+
|pmix_min_version|
70+
* **Minimum version required when building with PRRTE:** `See the
71+
PRRTE project documentation <https://docs.prrte.org/>`_.
72+
* **Version embedded in Open MPI distribution:**
73+
|pmix_embedded_version|
74+
75+
* `PRRTE <https://github.com/openpmix/prrte>`_
76+
77+
* This library is optional in some environments. See below.
78+
* **Minimum version required:** |prte_min_version|
79+
80+
.. note:: While building Open MPI with PRRTE |prte_min_version|
81+
*works*, you will not get a fully-populated
82+
``mpirun(1)`` man page. The Open MPI community
83+
recommends that you use PRRTE version 3.0.1 or higher.
84+
85+
* **Version embedded in Open MPI distribution:**
86+
|prte_embedded_version|
87+
88+
PRRTE provides Open MPI's full-featured ``mpirun`` / ``mpiexec`` MPI
89+
application launchers (the two commands are identical; they are
90+
symbolic links to the same executable).
91+
92+
.. warning:: If you are building the PRRTE that is embedded in the
93+
Open MPI |ompi_ver| distribution:
94+
95+
* If you are also building the PMIx that is embedded in
96+
the Open MPI |ompi_ver| distribution, that
97+
combination of packages is supported.
98+
99+
* If you are building against an external PMIx
100+
installation (i.e., a version of PMIx that is not
101+
embedded in the Open MPI |ompi_ver| distribution),
102+
you should check `the PRRTE project documentation
103+
<https://docs.prrte.org/>`_ to see what minimum
104+
version of PMIx is required.
105+
106+
* If your environment uses another MPI application launcher (e.g.,
107+
Slurm users can use the ``srun`` launcher to "direct launch" Open
108+
MPI applications), then the use of PRRTE is optional.
109+
* If your environment has no other MPI application launcher, then
110+
you need to install PRRTE and build Open MPI with PRRTE support.
111+
* Open MPI can use the copy of PRRTE embedded in its source code
112+
tree, or compile/link against an external PRRTE installation.
113+
:ref:`See this section for details about how to specify each
114+
method
115+
<label-building-ompi-cli-options-required-support-libraries>`.
116+
117+
.. note:: In general, the Open MPI community recommends using the most
118+
recent versions of both the :ref:`required support libraries
119+
<label-install-required-support-libraries>` and any other
120+
optional support libraries. This is because more recent
121+
versions typically tend to include bug fixes, sometimes
122+
affecting Open MPI functionality. As a specific example,
123+
there is a known issue with `Hardware Locality
124+
<https://www.open-mpi.org/projects/hwloc/>`_ releases older
125+
than v2.8.0 on systems with Intel Ponte Vecchio
126+
accelerators. If you run Open MPI on such systems, you need
127+
to use Hwloc v2.8.0 or newer, or you will experience
128+
undefined behavior. This effect is not unique to the
129+
Hardware Locality library; this is why the Open MPI
130+
community recommends using as recent as possible versions of
131+
*all* support libraries.
132+
80133
Library dependencies
81134
--------------------
82135

@@ -145,6 +198,8 @@ example |mdash| only Libevent and Hwloc, that somewhat simplifies the
145198
final Open MPI configuration, and therefore avoids some potentially
146199
erroneous configurations.
147200

201+
.. _required-support-libraries-configure-discovery-label:
202+
148203
How ``configure`` finds the required libraries
149204
----------------------------------------------
150205

@@ -264,7 +319,7 @@ on Mac OS because:
264319
tarballs).
265320
#. In MacOS, it is common for `Homebrew <https://brew.sh/>`_ or
266321
`MacPorts <https://www.macports.org/>`_ to install:
267-
322+
268323
* `Hardware Locality <https://www.open-mpi.org/projects/hwloc/>`_
269324
* `Libevent <https://libevent.org/>`_
270325

0 commit comments

Comments
 (0)