Jonathan Corbet [Wed, 13 Aug 2025 16:19:50 +0000 (10:19 -0600)]
Merge branch 'pre-install' into docs-mw
Mauro says:
that's the second version of the patch series which converts
sphinx-pre-install to Python.
The core patches are basically the same as on v1, but it has lots of
fixes over the original script after testing sphinx-install on 22
distros.
Please notice that I have a separate patch series addressing issues
that are specific to PDF generation.
Test Results Summary:
====================
PASSED - AlmaLinux release 9.6 (Sage Margay) (4 tests)
PASSED - Amazon Linux release 2023 (Amazon Linux) (4 tests)
PASSED - Arch Linux (4 tests)
PASSED - CentOS Stream release 9 (4 tests)
PASSED - Debian GNU/Linux 12 (4 tests)
PASSED - Devuan GNU/Linux 5 (4 tests)
PASSED - Fedora release 42 (Adams) (4 tests)
PASSED - Gentoo Base System release 2.17 (4 tests)
PASSED - Kali GNU/Linux 2025.2 (4 tests)
PASSED - Mageia 9 (4 tests)
PASSED - Linux Mint 22 (4 tests)
PASSED - openEuler release 25.03 (4 tests)
PARTIAL - OpenMandriva Lx 4.3 (4 tests)
ensurepip package doesn't exist there. So, venv install failed.
Installed via package worked
PASSED - openSUSE Leap 15.6 (4 tests)
PASSED - openSUSE Tumbleweed (4 tests)
PASSED - Oracle Linux Server release 9.6 (4 tests)
FAILED - Red Hat Enterprise Linux release 8.10 (Ootpa) (4 tests)
I couldn't test properly, as it requires a repository under
paywall. I suspect It should work fine
PARTIAL - Rocky Linux release 8.9 (Green Obsidian) (4 tests)
Install via package didn't work. Instaling via venv works.
PASSED - Rocky Linux release 9.6 (Blue Onyx) (4 tests)
PARTIAL - Springdale Open Enterprise Linux release 9.2 (Parma) (4 tests)
Failed to install ImageMagick (affects pdf only)
PASSED - Ubuntu 24.04.2 LTS (4 tests)
PASSED - Ubuntu 25.04 (4 tests)
In short, I expect that, for all the above, the script will properly
recommend the right packages to have sphinx-build working.
A more detailed list of tests that passed/failed and detected Sphinx
versions can be seeing below:
AlmaLinux release 9.6 (Sage Margay):
------------------------------------
PASSED: OS detection: AlmaLinux release 9.6 (Sage Margay)
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 7.4.7
PASSED: Sphinx package: Sphinx 3.4.3
Amazon Linux release 2023 (Amazon Linux):
-----------------------------------------
PASSED: OS detection: Amazon Linux release 2023 (Amazon Linux)
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 7.4.7
PASSED: Sphinx package: Sphinx 3.4.3
Arch Linux:
-----------
PASSED: OS detection: Arch Linux
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.2.3
CentOS Stream release 9:
------------------------
PASSED: OS detection: CentOS Stream release 9
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 7.4.7
PASSED: Sphinx package: Sphinx 3.4.3
Debian GNU/Linux 12:
--------------------
PASSED: OS detection: Debian GNU/Linux 12
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 5.3.0
Devuan GNU/Linux 5:
-------------------
PASSED: OS detection: Devuan GNU/Linux 5
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 5.3.0
Fedora release 42 (Adams):
--------------------------
PASSED: OS detection: Fedora release 42 (Adams)
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.1.3
Gentoo Base System release 2.17:
--------------------------------
PASSED: OS detection: Gentoo Base System release 2.17
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.2.3
Kali GNU/Linux 2025.2:
----------------------
PASSED: OS detection: Kali GNU/Linux 2025.2
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.1.3
Mageia 9:
---------
PASSED: OS detection: Mageia 9
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 6.1.3
PASSED: Sphinx package: Sphinx 6.1.3
Linux Mint 22:
--------------
PASSED: OS detection: Linux Mint 22
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.1.3
PASSED: Sphinx package: Sphinx 4.3.2
openEuler release 25.03:
------------------------
PASSED: OS detection: openEuler release 25.03
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.1.3
OpenMandriva Lx 4.3:
--------------------
PASSED: OS detection: OpenMandriva Lx 4.3
FAILED: System packages: Error: Unable to find a match: ensurepip
FAILED: Sphinx on venv: Installation failed
PASSED: Sphinx package: Sphinx 4.3.2
openSUSE Leap 15.6:
-------------------
PASSED: OS detection: openSUSE Leap 15.6
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 7.2.6
openSUSE Tumbleweed:
--------------------
PASSED: OS detection: openSUSE Tumbleweed
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.2.3
Oracle Linux Server release 9.6:
--------------------------------
PASSED: OS detection: Oracle Linux Server release 9.6
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 7.4.7
PASSED: Sphinx package: Sphinx 3.4.3
Red Hat Enterprise Linux release 8.10 (Ootpa):
----------------------------------------------
PASSED: OS detection: Red Hat Enterprise Linux release 8.10 (Ootpa)
FAILED: System packages: Error: Unable to find a match: google-noto-sans-cjk-ttc-fonts librsvg2-tools
texlive-amscls texlive-amsfonts texlive-amsmath texlive-anyfontsize texlive-capt-of texlive-cmap
texlive-collection-fontsrecommended texlive-collection-latex texlive-ec texlive-eqparbox texlive-euenc
texlive-fancybox texlive-fancyvrb texlive-float texlive-fncychap texlive-framed texlive-luatex85
texlive-mdwtools texlive-multirow texlive-needspace texlive-oberdiek texlive-parskip texlive-polyglossia
texlive-psnfss texlive-tabulary texlive-threeparttable texlive-titlesec texlive-tools texlive-ucs
texlive-upquote texlive-wrapfig texlive-xecjk texlive-xetex-bin
FAILED: Sphinx on venv: No Sphinx version detected
FAILED: Sphinx package: No Sphinx version detected
Rocky Linux release 8.9 (Green Obsidian):
-----------------------------------------
PASSED: OS detection: Rocky Linux release 8.9 (Green Obsidian)
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 7.4.7
FAILED: Sphinx package: No Sphinx version detected
Rocky Linux release 9.6 (Blue Onyx):
------------------------------------
PASSED: OS detection: Rocky Linux release 9.6 (Blue Onyx)
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 7.4.7
PASSED: Sphinx package: Sphinx 3.4.3
Springdale Open Enterprise Linux release 9.2 (Parma):
-----------------------------------------------------
PASSED: OS detection: Springdale Open Enterprise Linux release 9.2 (Parma)
FAILED: System packages: Error: Problem: package ImageMagick-6.9.13.25-1.el9.x86_64 requires
libMagickCore-6.Q16.so.7()(64bit), but none of the providers can be installed - package
ImageMagick-6.9.13.25-1.el9.x86_64 requires libMagickWand-6.Q16.so.7()(64bit), but none of the providers can
be installed - package ImageMagick-6.9.13.25-1.el9.x86_64 requires ImageMagick-libs(x86-64) =
6.9.13.25-1.el9, but none of the providers can be installed - conflicting requests - nothing provides
libraw_r.so.23()(64bit) needed by ImageMagick-libs-6.9.13.25-1.el9.x86_64
PASSED: Sphinx on venv: Sphinx 7.4.7
PASSED: Sphinx package: Sphinx 3.4.3
Ubuntu 24.04.2 LTS:
-------------------
PASSED: OS detection: Ubuntu 24.04.2 LTS
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 7.2.6
Ubuntu 25.04:
-------------
PASSED: OS detection: Ubuntu 25.04
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.1.3
AlmaLinux release 9.6 (Sage Margay):
------------------------------------
PASSED: OS detection: AlmaLinux release 9.6 (Sage Margay)
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 7.4.7
PASSED: Sphinx package: Sphinx 3.4.3
Amazon Linux release 2023 (Amazon Linux):
-----------------------------------------
PASSED: OS detection: Amazon Linux release 2023 (Amazon Linux)
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 7.4.7
PASSED: Sphinx package: Sphinx 3.4.3
Arch Linux:
-----------
PASSED: OS detection: Arch Linux
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.2.3
CentOS Stream release 9:
------------------------
PASSED: OS detection: CentOS Stream release 9
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 7.4.7
PASSED: Sphinx package: Sphinx 3.4.3
Debian GNU/Linux 12:
--------------------
PASSED: OS detection: Debian GNU/Linux 12
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 5.3.0
Devuan GNU/Linux 5:
-------------------
PASSED: OS detection: Devuan GNU/Linux 5
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 5.3.0
Fedora release 42 (Adams):
--------------------------
PASSED: OS detection: Fedora release 42 (Adams)
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.1.3
Gentoo Base System release 2.17:
--------------------------------
PASSED: OS detection: Gentoo Base System release 2.17
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.2.3
Kali GNU/Linux 2025.2:
----------------------
PASSED: OS detection: Kali GNU/Linux 2025.2
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.1.3
Mageia 9:
---------
PASSED: OS detection: Mageia 9
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 6.1.3
PASSED: Sphinx package: Sphinx 6.1.3
Linux Mint 22:
--------------
PASSED: OS detection: Linux Mint 22
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.1.3
PASSED: Sphinx package: Sphinx 4.3.2
openEuler release 25.03:
------------------------
PASSED: OS detection: openEuler release 25.03
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.1.3
OpenMandriva Lx 4.3:
--------------------
PASSED: OS detection: OpenMandriva Lx 4.3
FAILED: System packages: Error: Unable to find a match: ensurepip
PARTIAL: Sphinx on venv: Installation failed
PASSED: Sphinx package: Sphinx 4.3.2
openSUSE Leap 15.6:
-------------------
PASSED: OS detection: openSUSE Leap 15.6
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 7.2.6
openSUSE Tumbleweed:
--------------------
PASSED: OS detection: openSUSE Tumbleweed
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.2.3
Oracle Linux Server release 9.6:
--------------------------------
PASSED: OS detection: Oracle Linux Server release 9.6
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 7.4.7
PASSED: Sphinx package: Sphinx 3.4.3
Red Hat Enterprise Linux release 8.10 (Ootpa):
----------------------------------------------
PASSED: OS detection: Red Hat Enterprise Linux release 8.10 (Ootpa)
FAILED: System packages: Error: Unable to find a match: google-noto-sans-cjk-ttc-fonts librsvg2-tools
texlive-amscls texlive-amsfonts texlive-amsmath texlive-anyfontsize texlive-capt-of texlive-cmap
texlive-collection-fontsrecommended texlive-collection-latex texlive-ec texlive-eqparbox texlive-euenc
texlive-fancybox texlive-fancyvrb texlive-float texlive-fncychap texlive-framed texlive-luatex85
texlive-mdwtools texlive-multirow texlive-needspace texlive-oberdiek texlive-parskip texlive-polyglossia
texlive-psnfss texlive-tabulary texlive-threeparttable texlive-titlesec texlive-tools texlive-ucs
texlive-upquote texlive-wrapfig texlive-xecjk texlive-xetex-bin
PARTIAL: Sphinx on venv: No Sphinx version detected
PARTIAL: Sphinx package: No Sphinx version detected
Rocky Linux release 8.9 (Green Obsidian):
-----------------------------------------
PASSED: OS detection: Rocky Linux release 8.9 (Green Obsidian)
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 7.4.7
PARTIAL: Sphinx package: No Sphinx version detected
Rocky Linux release 9.6 (Blue Onyx):
------------------------------------
PASSED: OS detection: Rocky Linux release 9.6 (Blue Onyx)
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 7.4.7
PASSED: Sphinx package: Sphinx 3.4.3
Springdale Open Enterprise Linux release 9.2 (Parma):
-----------------------------------------------------
PASSED: OS detection: Springdale Open Enterprise Linux release 9.2 (Parma)
FAILED: System packages: Error: Problem: package ImageMagick-6.9.13.25-1.el9.x86_64 requires
libMagickCore-6.Q16.so.7()(64bit), but none of the providers can be installed - package
ImageMagick-6.9.13.25-1.el9.x86_64 requires libMagickWand-6.Q16.so.7()(64bit), but none of the providers can
be installed - package ImageMagick-6.9.13.25-1.el9.x86_64 requires ImageMagick-libs(x86-64) =
6.9.13.25-1.el9, but none of the providers can be installed - conflicting requests - nothing provides
libraw_r.so.23()(64bit) needed by ImageMagick-libs-6.9.13.25-1.el9.x86_64
PASSED: Sphinx on venv: Sphinx 7.4.7
PASSED: Sphinx package: Sphinx 3.4.3
Ubuntu 24.04.2 LTS:
-------------------
PASSED: OS detection: Ubuntu 24.04.2 LTS
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 7.2.6
Ubuntu 25.04:
-------------
PASSED: OS detection: Ubuntu 25.04
PASSED: System packages: Packages installed
PASSED: Sphinx on venv: Sphinx 8.2.3
PASSED: Sphinx package: Sphinx 8.1.3
scripts: sphinx-pre-install: some adjustments related to venv
While nothing was really needed for virtualenv to work on most
distros, we had an issue with OpenMandriva.
While checking for it, it was noticed that there was no check if
python-virtualenv was installed.
This didn't solve the issues we faced there: at least with
the half-broken OpenMandriva Lx 4.0 docker container we used,
ensurepip was not available anywhere, causing venv to fail.
Add a distro-specific note about that.
Note: at least at the time we did our tests, OpenMandriva Lx 4.0
docker was shipped with wrong dnf repositories. Also, there
was no repos available for it anymore. So, we had to do some
hacks to upgrade to 4.3 before being able to run any tests.
This program is somewhat complex. Add some docstring documentation,
explaining what each function and class is supposed to do.
Most of the focus here were to describe the ancillary functions used
to detect dependency needs.
The main SphinxDependencyChecker still requires a lot of care,
and probably need to be reorganized to clearly split the 4 types
of output it produces:
- Need to upgrade Python binary;
- System install needs;
- Virtual env install needs;
- Python install needs via system packages, to run Sphinx
natively.
Yet, for now, I'm happy of having it a lot better documented
than its Perl version.
-
While here, rename a parameter to have its usage better
documented.
The code at get_system_release() is actually a helper function,
independent from the actual Sphinx verification checker. Move
it to MissingCheckers class, where other checkers are present.
With that, the entire distro-specific handler logic, with
all its complexity is confined at SphinxDependencyChecker
class.
scripts: sphinx-pre-install: add more generic checkers on a class
Better organize the code by moving the more generic methods
to MissingCheckers. Such class contain only binary and package
dependent missing checkers, but no distro-specific data or code.
All distro-specific data/code remains at SphinxDependencyChecker
class.
Better implement support for RHEL-based distros. While here,
get rid of a Fedora 28 support which cause troubles with
server distros. Also, get rid of yum, as RHEL8 already
suppords dnf, and this is not the minimal version we may
still support.
It took me a lot of time, but I guess understand now what it
takes to install a package on Gentoo.
Handling dependencies is a nightmare, as Gentoo refuses to emerge
some packages if there's no package.use file describing them.
To make it worse, compilation flags shall also be present there
for some packages. If USE is not perfect, error/warning messages
like those are shown:
!!! The following binary packages have been ignored due to non matching USE:
=media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_13 qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf python_single_target_python3_12 -python_single_target_python3_13 qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_10 qt6 svg
=media-gfx/graphviz-12.2.1-r1 X pdf -python_single_target_python3_10 python_single_target_python3_12 -python_single_target_python3_13 qt6 svg
=media-fonts/noto-cjk-20190416 X
=app-text/texlive-core-2024-r1 X cjk -xetex
=app-text/texlive-core-2024-r1 X -xetex
=app-text/texlive-core-2024-r1 -xetex
=dev-libs/zziplib-0.13.79-r1 sdl
If emerge is allowed, it will simply ignore the above packages,
creating an incomplete installation, which will later fail when
one tries to build docs with images or build PDFs.
After the fix, command line commands to produce the needed USE
chain will be emitted, if they don't exist yet.
sudo su -c 'echo "media-gfx/graphviz" > /etc/portage/package.use/graphviz'
sudo su -c 'echo "media-gfx/imagemagick" > /etc/portage/package.use/imagemagick'
sudo su -c 'echo "media-libs/harfbuzz icu" > /etc/portage/package.use/media-libs'
sudo su -c 'echo "media-fonts/noto-cjk" > /etc/portage/package.use/media-fonts'
sudo su -c 'echo "app-text/texlive-core xetex" > /etc/portage/package.use/texlive'
sudo su -c 'echo "dev-libs/zziplib sdl" > /etc/portage/package.use/zziblib'
The new logic tries to be smart enough to detect for missing files
and missing arguments. Yet, as Gentoo seems to require users to
manage those package.use files by hand, the logic isn't perfect:
users may still need to verify for conflicts on different use
files.
scripts: sphinx-pre-install: fix Leap support for rsvg-convert
There is a test logic meant to be for Leap, renaming rsvg-convert
package. Well, at least on latest Leap releases, this is wrong,
causing install to fail. Drop it.
scripts: sphinx-pre-install: add a missing f-string marker
I forgot one f-string marker, with turned to be affecting 3
lines, because of cut-and-paste ;-)
Use the proper f-string marker to print Sphinx version at
the hint lines. Yet, we don't want to print as a tuple, so
call ver_str() for it.
Ideally, we would be placing it directly at the f-string, but
Python 3.6 f-string support was pretty much limited. Only
3.12 (PEP 701) makes it similar to Perl, allowing expressions
inside it. It sounds that function call itself was introduced
on 3.7.
As we explicitly want this one to run on 3.6, as latest Leap
comes with it, we can't use function calls on f-string.
Rhel < 8.0 is not supported anymore. Drop support for it.
Rhel 8 is problematic: at least on the tests I did with a
docker repo, it didn't work, but it could be due to the issue
that it is actually different than a real One.
scripts: sphinx-pre-install: Address issues with OpenSUSE Leap 15.x
On openSUSE Leap 15.6, which is the current LTS version, has two
Sphinx packages. The normal one requires Python 3.6, which we
don't support anymore. However, it also has Python 3.11 with a
newer Sphinx version (7.2.6).
Suggest the newer version:
Detected OS: openSUSE Leap 15.6.
ERROR: at least python 3.7 is required to build the kernel docs
Warning: python version is not supported.
Warning: better to also install "convert".
Warning: better to also install "dot".
ERROR: please install "yaml", otherwise, build won't work.
You should run:
Please note that Sphinx >= 3.0 will currently produce false-positive
warning when the same name is used for more than one type (functions,
structs, enums,...). This is known Sphinx bug. For more details, see:
https://github.com/sphinx-doc/sphinx/pull/8313
Can't build as 2 mandatory dependencies are missing
scripts: sphinx-pre-install: run on a supported version
The scripts/sphinx-pre-install is used to detect problems at
the system environment and adjust it to build the Kernel
documentation. If the version is too old, it won't run, though.
Check if the version which started the script is valid. If not,
seek for a new one that is compatible with documentation
build.
scripts: sphinx-pre-install: Make it compatible with Python 3.6
The minimal version requirements we have is 3.9. Yet, the
script which detects it is this one. So, let's try supporting
an old version here, as we may want to suggest to upgrade
Python version to build the docs.
changeset f061c9f7d058 ("Documentation: Document each netlink family")
added a logic which generates *.rst files inside $(srctree). This is bad
when O=<BUILDDIR> is used.
A recent change renamed the yaml files used by Netlink, revealing a bad
side effect: as "make cleandocs" don't clean the produced files and symbols
appear duplicated for people that don't build the kernel from scratch.
This series adds an yaml parser extension and uses an index file with glob for
*. We opted to write such extension in a way that no actual yaml conversion
code is inside it. This makes it flexible enough to handle other types of yaml
files in the future. The actual yaml conversion logic were placed at
netlink_yml_parser.py.
As requested by YNL maintainers, this version has netlink_yml_parser.py
inside tools/net/ynl/pyynl/ directory. I don't like mixing libraries with
binaries, nor to have Python libraries spread all over the Kernel. IMO,
the best is to put all of them on a common place (scripts/lib, python/lib,
lib/python, ...) but, as this can be solved later, for now let's keep it this
way.
breaks line parsing. As an alternative, I tested a variant of it:
rst_parser.parse(result, document)
but still line number was not preserved. As Donald noted,
standard Parser classes don't have a direct mechanism to preserve
line numbers from ViewList().
So, instead, let's use a mechanism similar to what we do already at
kerneldoc.py: call the statemachine mechanism directly there.
I double-checked when states and statemachine were introduced:
both were back in 2002. I also tested doc build with docutils 0.16
and 0.21.2. It worked with both, so it seems to be stable enough
for our needs.
MAINTAINERS: add netlink_yml_parser.py to linux-doc
The documentation build depends on the parsing code
at ynl_gen_rst.py. Ensure that changes to it will be c/c
to linux-doc ML and maintainers by adding an entry for
it. This way, if a change there would affect the build,
or the minimal version required for Python, doc developers
may know in advance.
docs: netlink: remove obsolete .gitignore from unused directory
The previous code was generating source rst files
under Documentation/networking/netlink_spec/. With the
Sphinx YAML parser, this is now gone. So, stop ignoring
*.rst files inside netlink specs directory.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
docs: use parser_yaml extension to handle Netlink specs
Instead of manually calling ynl_gen_rst.py, use a Sphinx extension.
This way, no .rst files would be written to the Kernel source
directories.
We are using here a toctree with :glob: property. This way, there
is no need to touch the netlink/specs/index.rst file every time
a new Netlink spec is added/renamed/removed.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
tools: ynl_gen_rst.py: Split library from command line tool
As we'll be using the Netlink specs parser inside a Sphinx
extension, move the library part from the command line parser.
While here, change the code which generates an index file
to parse inputs from both .rst and .yaml extensions. With
that, the tool can easily be tested with:
Having :doc: references with relative paths doesn't always work,
as it may have troubles when O= is used. Also that's hard to
maintain, and may break if we change the way rst files are
generated from yaml. Better to use instead a reference for
the netlink family.
So, replace them by Sphinx cross-reference tag that are
created by ynl_gen_rst.py.
Signed-off-by: Mauro Carvalho Chehab <mchehab+huawei@kernel.org> Reviewed-by: Donald Hunter <donald.hunter@gmail.com>
docs: filesystems: sysfs: Recommend sysfs_emit() for new code only
The advantages of converting existing sysfs show() methods to sysfs_emit()
and sysfs_emit_at() do not outweigh the risk of introducing bugs. Hence
recommend sysfs_emit() and sysfs_emit_at() only for new implementations of
show() methods.
Cc: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Cc: James Bottomley <James.Bottomley@HansenPartnership.com> Cc: Martin K. Petersen <martin.petersen@oracle.com> Signed-off-by: Bart Van Assche <bvanassche@acm.org> Reviewed-by: Martin K. Petersen <martin.petersen@oracle.com> Acked-by: Greg Kroah-Hartman <gregkh@linuxfoundation.org> Signed-off-by: Jonathan Corbet <corbet@lwn.net> Link: https://lore.kernel.org/r/20250724153449.2433395-1-bvanassche@acm.org
docs: admin-guide: update to current minimum pipe size default
The pipe size limit used when the fs.pipe-user-pages-soft sysctl value
is reached was increased from one to two pages in commit 46c4c9d1beb7;
update the documentation to match the new reality.
Kim Tae Hyun [Sat, 2 Aug 2025 12:58:01 +0000 (21:58 +0900)]
docs: update physical memory documentation by adding N_GENERIC_INITIATOR to enum node_states
While reading physical_memory.rst, I noticed that N_GENERIC_INITIATOR has not been update
from the node_states list, even though it's already added in commit 894c26a1c274b8eafbb4b1dad67e70e51a106061.
docs: kernel-doc: avoid script crash on ancient Python
While we do need at least 3.6 for kernel-doc to work, and at least
3.7 for it to output functions and structs with parameters at the
right order, let the python binary be compatible with legacy
versions.
The rationale is that the Kernel build nowadays calls kernel-doc
with -none on some places. Better not to bail out when older
versions are found.
With that, potentially this will run with python 2.7 and 3.2+,
according with vermin:
The minimal version I could check was version 3.4
(using anaconda). Anaconda doesn't support 3.2 or 3.3
anymore, and 3.2 doesn't even compile (I tested compiling
Python 3.2 on Fedora 42 and on Fedora 32 - no show).
With 3.4, the script didn't crash and emitted the right warning:
$ conda create -n py34 python=3.4
$ conda activate py34
python --version
Python 3.4.5
$ python ./scripts/kernel-doc --none include/media
Error: Python 3.6 or later is required by kernel-doc
$ conda deactivate
$ python --version
Python 3.13.5
$ python ./scripts/kernel-doc --none include/media
(no warnings and script ran properly)
Supporting 2.7 is out of scope, as it is EOL for 5 years, and
changing shebang to point to "python" instead of "python3"
would have a wider impact.
I did some extra checks about the differences from 3.2 and
3.4, and didn't find anything that would cause troubles:
Jonathan Corbet [Mon, 11 Aug 2025 16:29:07 +0000 (10:29 -0600)]
Merge branch 'dump-struct' into docs-mw
In my ongoing effort to truly understand our new kernel-doc, I continue to
make changes to improve the code, and to try to make the understanding task
easier for the next person. These patches focus on dump_struct() in
particular, which starts out at nearly 300 lines long - to much to fit into
my little brain anyway. Hopefully the result is easier to manage.
Jonathan Corbet [Thu, 7 Aug 2025 21:16:36 +0000 (15:16 -0600)]
docs: kdoc: Some rewrite_struct_members() commenting
Add comments to rewrite_struct_members() describing what it is actually
doing, and reformat/comment the main struct_members regex so that it is
(more) comprehensible to humans.
Jonathan Corbet [Thu, 7 Aug 2025 21:16:34 +0000 (15:16 -0600)]
docs: kdoc: rework the rewrite_struct_members() main loop
Adopt a more Pythonic form for the main loop of this function, getting rid
of the "while True:" construction and making the actual loop invariant
explicit.
Jonathan Corbet [Thu, 7 Aug 2025 21:16:33 +0000 (15:16 -0600)]
docs: kdoc: split struct-member rewriting out of dump_struct()
The massive loop that massages struct members shares no data with the rest
of dump_struct(); split it out into its own function. Code movement only,
no other changes.
Jonathan Corbet [Thu, 7 Aug 2025 21:16:31 +0000 (15:16 -0600)]
docs: kdoc: move the prefix transforms out of dump_struct()
dump_struct is one of the longest functions in the kdoc_parser class,
making it hard to read and reason about. Move the definition of the prefix
transformations out of the function, join them with the definition of
"attribute" (which was defined at the top of the file but only used here),
and reformat the code slightly for shorter line widths.
Jonathan Corbet [Thu, 7 Aug 2025 21:16:30 +0000 (15:16 -0600)]
docs: kdoc: backslashectomy in kdoc_parser
A lot of the regular expressions in this file have extraneous backslashes
that may have been needed in Perl, but aren't helpful here. Take them out
to reduce slightly the visual noise.
Escaping of (){}[] has been left in place, even when unnecessary, for
visual clarity.
Jonathan Corbet [Thu, 7 Aug 2025 21:16:28 +0000 (15:16 -0600)]
docs: kdoc: consolidate the stripping of private struct/union members
There were two locations duplicating the logic of stripping private members
and associated comments; coalesce them into one, and add some comments
describing what's going on.
Output change: we now no longer add extraneous white space around macro
definitions.
Linus Torvalds [Sun, 10 Aug 2025 05:51:37 +0000 (08:51 +0300)]
Merge tag 'smp_urgent_for_v6.17_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull smp fixes from Borislav Petkov:
- Remove an obsolete comment and fix spelling
* tag 'smp_urgent_for_v6.17_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
cpu: Remove obsolete comment from takedown_cpu()
smp: Fix spelling in on_each_cpu_cond_mask()'s doc-comment
Linus Torvalds [Sun, 10 Aug 2025 05:46:47 +0000 (08:46 +0300)]
Merge tag 'irq_urgent_for_v6.17_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull irq fixes from Borislav Petkov:
- Fix a wrong ioremap size in mvebu-gicp
- Remove yet another compile-test case for a driver which needs an
additional dependency
- Fix a lock inversion scenario in the IRQ unit test suite
- Remove an impossible flag situation in gic-v5
- Do not iounmap resources in gic-v5 which are managed by devm
- Make sure stale, left-over interrupts in mvebu-gicp are cleared on
driver init
- Fix a reference counting mishap in msi-lib
- Fix a dereference-before-null-ptr-check case in the riscv-imsic
irqchip driver
* tag 'irq_urgent_for_v6.17_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
irqchip/mvebu-gicp: Use resource_size() for ioremap()
irqchip: Build IMX_MU_MSI only on ARM
genirq/test: Resolve irq lock inversion warnings
irqchip/gic-v5: Remove IRQD_RESEND_WHEN_IN_PROGRESS for ITS IRQs
irqchip/gic-v5: iwb: Fix iounmap probe failure path
irqchip/mvebu-gicp: Clear pending interrupts on init
irqchip/msi-lib: Fix fwnode refcount in msi_lib_irq_domain_select()
irqchip/riscv-imsic: Don't dereference before NULL pointer check
Linus Torvalds [Sun, 10 Aug 2025 05:15:32 +0000 (08:15 +0300)]
Merge tag 'x86_urgent_for_v6.17_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip
Pull x86 fixes from Borislav Petkov:
- Fix an interrupt vector setup race which leads to a non-functioning
device
- Add new Intel CPU models *and* a family: 0x12. Finally. Yippie! :-)
* tag 'x86_urgent_for_v6.17_rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tip/tip:
x86/irq: Plug vector setup race
x86/cpu: Add new Intel CPU model numbers for Wildcatlake and Novalake
Len Brown [Sun, 10 Aug 2025 01:08:26 +0000 (21:08 -0400)]
tools/power turbostat: version 2025.09.09
Probe and display L3 Cache topology
Add ability to average an added counter
(useful for pre-integrated "counters", such as Watts)
Break the limit of 64 built-in counters.
Assorted bug fixes and minor feature tweaks
/sys/devices/system/cpu/intel_uncore_frequency/package_X_die_Y/
may be readable by all, but
/sys/devices/system/cpu/intel_uncore_frequency/package_X_die_Y/current_freq_khz
may be readable only by root.
Non-root turbostat users see complaints in this scenario.
Fail probe of the interface if we can't read current_freq_khz.
Reported-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com> Original-patch-by: Zhang Rui <rui.zhang@intel.com> Signed-off-by: Len Brown <len.brown@intel.com>
Zhang Rui [Wed, 11 Jun 2025 06:50:26 +0000 (14:50 +0800)]
tools/power turbostat: Fix DMR support
Together with the RAPL MSRs, there are more MSRs gone on DMR, including
PLR (Perf Limit Reasons), and IRTL (Package cstate Interrupt Response
Time Limit) MSRs. The configurable TDP info should also be retrieved
from TPMI based Intel Speed Select Technology feature.
Remove the access of these MSRs for DMR. Improve the DMR platform
feature table to make it more readable at the same time.
Fixes: 83075bd59de2 ("tools/power turbostat: Add initial support for DMR") Signed-off-by: Zhang Rui <rui.zhang@intel.com> Signed-off-by: Len Brown <len.brown@intel.com>
Michael Hebenstreit [Fri, 8 Aug 2025 19:57:53 +0000 (15:57 -0400)]
tools/power turbostat: add format "average" for external attributes
External atributes with format "raw" are not printed in summary lines
for nodes/packages (or with option -S). The new format "average"
behaves like "raw" but also adds the summary data
Signed-off-by: Michael Hebenstreit <michael.hebenstreit@intel.com> Signed-off-by: Len Brown <len.brown@intel.com>
Len Brown [Sat, 12 Jul 2025 20:16:56 +0000 (16:16 -0400)]
tools/power turbostat: Support more than 64 built-in-counters
We have out-grown the ability to use a 64-bit memory location
to inventory every possible built-in counter.
Leverage the the CPU_SET(3) macros to break this barrier.
Also, break the Joules & Watts counters into two,
since we can no longer 'or' them together...
Linus Torvalds [Sat, 9 Aug 2025 15:12:23 +0000 (18:12 +0300)]
Merge tag 'tty-6.16-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty
Pull TTY fix from Greg KH:
"Here is a single revert of one of the previous patches that went in
the last tty/serial merge that is breaking userspace on some platforms
(specifically powerpc, probably a few others.)
It accidentially changed the ioctl values of some tty ioctls, which
breaks xorg.
The revert has been in linux-next all this week with no reported
issues"
* tag 'tty-6.16-rc1-2' of git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/tty:
Revert "tty: vt: use _IO() to define ioctl numbers"
Linus Torvalds [Sat, 9 Aug 2025 05:47:28 +0000 (08:47 +0300)]
Merge tag 'block-6.17-20250808' of git://git.kernel.dk/linux
Pull more block updates from Jens Axboe:
- MD pull request via Yu:
- mddev null-ptr-dereference fix, by Erkun
- md-cluster fail to remove the faulty disk regression fix, by
Heming
- minor cleanup, by Li Nan and Jinchao
- mdadm lifetime regression fix reported by syzkaller, by Yu Kuai
- MD pull request via Christoph
- add support for getting the FDP featuee in fabrics passthru path
(Nitesh Shetty)
- add capability to connect to an administrative controller
(Kamaljit Singh)
- fix a leak on sgl setup error (Keith Busch)
- initialize discovery subsys after debugfs is initialized
(Mohamed Khalfella)
- fix various comment typos (Bjorn Helgaas)
- remove unneeded semicolons (Jiapeng Chong)
- nvmet debugfs ordering issue fix
- Fix UAF in the tag_set in zloop
- Ensure sbitmap shallow depth covers entire set
- Reduce lock roundtrips in io context lookup
- Move scheduler tags alloc/free out of elevator and freeze lock, to
fix some lockdep found issues
- Improve robustness of queue limits checking
- Fix a regression with IO priorities, if no io context exists
* tag 'block-6.17-20250808' of git://git.kernel.dk/linux: (26 commits)
lib/sbitmap: make sbitmap_get_shallow() internal
lib/sbitmap: convert shallow_depth from one word to the whole sbitmap
nvmet: exit debugfs after discovery subsystem exits
block, bfq: Reorder struct bfq_iocq_bfqq_data
md: make rdev_addable usable for rcu mode
md/raid1: remove struct pool_info and related code
md/raid1: change r1conf->r1bio_pool to a pointer type
block: ensure discard_granularity is zero when discard is not supported
zloop: fix KASAN use-after-free of tag set
block: Fix default IO priority if there is no IO context
nvme: fix various comment typos
nvme-auth: remove unneeded semicolon
nvme-pci: fix leak on sgl setup error
nvmet: initialize discovery subsys after debugfs is initialized
nvme: add capability to connect to an administrative controller
nvmet: add support for FDP in fabrics passthru path
md: rename recovery_cp to resync_offset
md/md-cluster: handle REMOVE message earlier
md: fix create on open mddev lifetime regression
block: fix potential deadlock while running nr_hw_queue update
...
projects / users / hch / misc.git / log