Mark sync/no-HDLC PPP tests as XFAIL for all CI images

See https://gitlab.com/openconnect/openconnect/-/issues/287#note_641198529 for discussions.

Across all CI images, non-HDLC PPP tests are consistently failing (this is
described as "synchronous" framing in the '90s-era terminology of pppd, and
is supported by 'pppd sync').

    FAIL: ppp-over-tls
    ==================
    Testing PPP ...
    [...]
    Starting PPP peer (sync/no-HDLC, IPv4+IPv6, DNS, extraneous VJ and CCP)... started in 0 seconds
    2021/07/31 20:54:18 socat[10622] E waitpid(): child 10625 exited with status 1
    Connecting to it with openconnect --protocol=nullppp... failed (after 0 seconds)
    [...]
    ===== START pppd log =====
    Couldn't set tty to PPP discipline: Invalid argument

The 'pppd sync' support has always appeared to be a fairly marginal part of
pppd capabilities, brittle and not well-tested, and I've run into other
problems with it before (see eaabbb09 for example).

This is frustrating because non-HDLC/pre-framed PPP is the version that is
(and should be!) used in all modern implementations of PPP, including F5 and
Fortinet's implementations.

This patch splits the sync/no-HDLC PPP test into a separate script
(ppp-over-tls-sync), and marks it as XFAIL for all CI runs, so that we can
continue to test it by default when running locally, and to fail on the
other PPP tests (which use async mode aka “HDLC-like” framing).

Signed-off-by: Daniel Lenski <dlenski@gmail.com>

diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml
index 2db1fa829a218efd68c3a98e0b48e0f9754e9211..907119be881dfeb76573e4c4c63a4cce78da4e74 100644 (file)
--- a/.gitlab-ci.yml
+++ b/.gitlab-ci.yml
@@ -28,7 +28,8 @@ CentOS7/GnuTLS:
 # we don't want pppd to invoke any actual connection scripts
   - mv /etc/ppp /etc/ppp.DISABLED
 # auth-nonascii: UTF-8 support is not available in distro's ocserv package
-  - make VERBOSE=1 XFAIL_TESTS="auth-nonascii" -j4 check
+# ppp-over-tls-sync: https://gitlab.com/openconnect/openconnect/-/issues/287#note_641198529)
+  - make VERBOSE=1 XFAIL_TESTS="auth-nonascii ppp-over-tls-sync" -j4 check
   tags:
   - shared
   except:
@@ -57,7 +58,8 @@ CentOS7/OpenSSL:
   - mv /etc/ppp /etc/ppp.DISABLED
 # auth-nonascii: UTF-8 support is not available in distro's ocserv package
 # dtls-psk: communication with ocserv fails for undetermined reasons marking as xfail until issue is identified
-  - make VERBOSE=1 XFAIL_TESTS="auth-nonascii dtls-psk" -j4 check
+# ppp-over-tls-sync: https://gitlab.com/openconnect/openconnect/-/issues/287#note_641198529)
+  - make VERBOSE=1 XFAIL_TESTS="auth-nonascii dtls-psk ppp-over-tls-sync" -j4 check
   tags:
   - shared
   except:
@@ -82,7 +84,8 @@ CentOS8/GnuTLS:
 # we don't want pppd to invoke any actual connection scripts
   - mv /etc/ppp /etc/ppp.DISABLED
 # auth-nonascii: UTF-8 support is not available in distro's ocserv package
-  - make VERBOSE=1 XFAIL_TESTS="auth-nonascii" -j4 check
+# ppp-over-tls-sync: https://gitlab.com/openconnect/openconnect/-/issues/287#note_641198529)
+  - make VERBOSE=1 XFAIL_TESTS="auth-nonascii ppp-over-tls-sync" -j4 check
   tags:
   - shared
   except:
@@ -108,7 +111,8 @@ CentOS8/OpenSSL:
   - mv /etc/ppp /etc/ppp.DISABLED
 # auth-nonascii: UTF-8 support is not available in distro's ocserv package
 # obsolete-server-crypto: OpenSSL 1.1.0 disables 3DES and RC4 by default (https://www.openssl.org/blog/blog/2016/08/24/sweet32/)
-  - make VERBOSE=1 XFAIL_TESTS="auth-nonascii obsolete-server-crypto" -j4 check
+# ppp-over-tls-sync: https://gitlab.com/openconnect/openconnect/-/issues/287#note_641198529)
+  - make VERBOSE=1 XFAIL_TESTS="auth-nonascii obsolete-server-crypto ppp-over-tls-sync" -j4 check
   tags:
   - shared
   except:
@@ -131,7 +135,8 @@ CentOS6/OpenSSL:
 # we don't want pppd to invoke any actual connection scripts
   - mv /etc/ppp /etc/ppp.DISABLED
 # auth-nonascii: UTF-8 support is not available in distro's ocserv package
-  - make VERBOSE=1 XFAIL_TESTS="bad_dtls_test auth-nonascii" -j4 check
+# ppp-over-tls-sync: https://gitlab.com/openconnect/openconnect/-/issues/287#note_641198529)
+  - make VERBOSE=1 XFAIL_TESTS="bad_dtls_test auth-nonascii ppp-over-tls-sync" -j4 check
   tags:
   - shared
   except:
@@ -296,7 +301,8 @@ Fedora/GnuTLS:
 # obsolete-server-crypto: system-wide crypto policy prevents --allow-insecure-crypto from working, and the
 #   library-reinitialization hack does not reliably work (https://gitlab.com/openconnect/openconnect/-/issues/243#note_576194663)
 #   XFAIL until we get a solution for https://bugzilla.redhat.com/show_bug.cgi?id=1960763
-  - make VERBOSE=1 OPENSSL_ia32cap=~0x4000000000000000 XFAIL_TESTS="obsolete-server-crypto" -j4 check
+# ppp-over-tls-sync: https://gitlab.com/openconnect/openconnect/-/issues/287#note_641198529)
+  - make VERBOSE=1 OPENSSL_ia32cap=~0x4000000000000000 XFAIL_TESTS="obsolete-server-crypto ppp-over-tls-sync" -j4 check
   tags:
   - shared
   except:
@@ -329,7 +335,8 @@ Fedora/GnuTLS/ibmtss:
 # obsolete-server-crypto: system-wide crypto policy prevents --allow-insecure-crypto from working, and the
 #   library-reinitialization hack does not reliably work (https://gitlab.com/openconnect/openconnect/-/issues/243#note_576194663)
 #   XFAIL until we get a solution for https://bugzilla.redhat.com/show_bug.cgi?id=1960763
-  - make VERBOSE=1 OPENSSL_ia32cap=~0x4000000000000000 XFAIL_TESTS="obsolete-server-crypto" -j4 check
+# ppp-over-tls-sync: https://gitlab.com/openconnect/openconnect/-/issues/287#note_641198529)
+  - make VERBOSE=1 OPENSSL_ia32cap=~0x4000000000000000 XFAIL_TESTS="obsolete-server-crypto ppp-over-tls-sync" -j4 check
   tags:
   - shared
   except:
@@ -361,7 +368,8 @@ Fedora/GnuTLS/clang:
 # obsolete-server-crypto: system-wide crypto policy prevents --allow-insecure-crypto from working, and the
 #   library-reinitialization hack does not reliably work (https://gitlab.com/openconnect/openconnect/-/issues/243#note_576194663)
 #   XFAIL until we get a solution for https://bugzilla.redhat.com/show_bug.cgi?id=1960763
-  - make VERBOSE=1 OPENSSL_ia32cap=~0x4000000000000000 XFAIL_TESTS="obsolete-server-crypto" -j4 check
+# ppp-over-tls-sync: https://gitlab.com/openconnect/openconnect/-/issues/287#note_641198529)
+  - make VERBOSE=1 OPENSSL_ia32cap=~0x4000000000000000 XFAIL_TESTS="obsolete-server-crypto ppp-over-tls-sync" -j4 check
   tags:
   - shared
   except:
@@ -394,6 +402,8 @@ Fedora/OpenSSL:
 # auth-nonascii: UTF-8 support is not available in distro's ocserv package
 # obsolete-server-crypto: OpenSSL 1.1.0 disables 3DES and RC4 by default (https://www.openssl.org/blog/blog/2016/08/24/sweet32/)
   - make VERBOSE=1 XFAIL_TESTS="obsolete-server-crypto" -j4 check
+# ppp-over-tls-sync: https://gitlab.com/openconnect/openconnect/-/issues/287#note_641198529)
+  - make VERBOSE=1 XFAIL_TESTS="obsolete-server-crypto ppp-over-tls-sync" -j4 check
   tags:
   - shared
   except:
@@ -425,6 +435,8 @@ Fedora/OpenSSL/clang:
   - mv /etc/ppp /etc/ppp.DISABLED
 # obsolete-server-crypto: OpenSSL 1.1.0 disables 3DES and RC4 by default (https://www.openssl.org/blog/blog/2016/08/24/sweet32/)
   - make VERBOSE=1 XFAIL_TESTS="obsolete-server-crypto" -j4 check
+# ppp-over-tls-sync: https://gitlab.com/openconnect/openconnect/-/issues/287#note_641198529)
+  - make VERBOSE=1 XFAIL_TESTS="obsolete-server-crypto ppp-over-tls-sync" -j4 check
   tags:
   - shared
   except:
@@ -453,7 +465,8 @@ Ubuntu18.04/GnuTLS:
 # cause PPP tests using IPv6 to fail. So we must explicitly enable IPv6:
   - sysctl net.ipv6.conf.all.disable_ipv6=0
 # auth-nonascii: UTF-8 support is not available in distro's ocserv package
-  - make VERBOSE=1 XFAIL_TESTS="auth-nonascii" -j4 check
+# ppp-over-tls-sync: https://gitlab.com/openconnect/openconnect/-/issues/287#note_641198529)
+  - make VERBOSE=1 XFAIL_TESTS="auth-nonascii ppp-over-tls-sync" -j4 check
   tags:
   - shared
   except:
@@ -482,7 +495,8 @@ Ubuntu18.04/OpenSSL:
   - sysctl net.ipv6.conf.all.disable_ipv6=0
 # auth-nonascii: UTF-8 support is not available in distro's ocserv package
 # obsolete-server-crypto: OpenSSL 1.1.0 disables 3DES and RC4 by default (https://www.openssl.org/blog/blog/2016/08/24/sweet32/)
-  - make VERBOSE=1 XFAIL_TESTS="auth-nonascii obsolete-server-crypto" -j4 check
+# ppp-over-tls-sync: https://gitlab.com/openconnect/openconnect/-/issues/287#note_641198529)
+  - make VERBOSE=1 XFAIL_TESTS="auth-nonascii obsolete-server-crypto ppp-over-tls-sync" -j4 check
   tags:
   - shared
   except:

diff --git a/tests/Makefile.am b/tests/Makefile.am
index e7c8392c88200aa0fa323c7f55c89da6da5a2619..e94faa3c133d6f5d642b564255a4318b11048363 100644 (file)
--- a/tests/Makefile.am
+++ b/tests/Makefile.am
@@ -78,9 +78,9 @@ if HAVE_NETNS
 TESTS += dtls-psk sigterm
 endif
 
-dist_check_SCRIPTS += ppp-over-tls
+dist_check_SCRIPTS += ppp-over-tls ppp-over-tls-sync
 if TEST_PPP
-TESTS += ppp-over-tls
+TESTS += ppp-over-tls ppp-over-tls-sync
 endif
 
 dist_check_SCRIPTS += auth-username-pass auth-certificate auth-nonascii cert-fingerprint \

diff --git a/tests/ppp-over-tls b/tests/ppp-over-tls
index 4b9a572ffcbb3ea17eb4b19d7e3459b8d92090f1..a0f316affc71b8b335dec4ebd48d66c412d0deb7 100755 (executable)
--- a/tests/ppp-over-tls
+++ b/tests/ppp-over-tls
@@ -45,7 +45,7 @@ IPV6_SUCCESS_1="rcvd [IPV6CP ConfAck "
 IPV6_SUCCESS_2="sent [IPV6CP ConfAck "
 TIMEOUT_3S_IDLE="idle 3"
 
-echo "Testing PPP ... "
+echo "Testing PPP with 'HDLC-like framing' (RFC1662)... "
 
 echo -n "Starting PPP peer (HDLC/RFC1662, IPv4+IPv6, DNS, extraneous VJ and CCP)... "
 start=$(date +%s)
@@ -89,27 +89,6 @@ fi
 
 cleanup
 
-echo -n "Starting PPP peer (sync/no-HDLC, IPv4+IPv6, DNS, extraneous VJ and CCP)... "
-start=$(date +%s)
-launch_simple_pppd $CERT $KEY $HDLC_NO $IPV4_YES $OFFER_DNS $IPV6_YES 2>&1
-echo "started in $(( $(date +%s) - start )) seconds"
-wait_server "$PID"
-echo -n "Connecting to it with openconnect --protocol=nullppp... "
-start=$(date +%s)
-LD_PRELOAD=libsocket_wrapper.so $OPENCONNECT -q --protocol=nullppp $ADDRESS:443 -u test $FINGERPRINT --cookie "term" -Ss '' >/dev/null 2>&1
-took=$(( $(date +%s) - start ))
-if grep -qF "$IPV4_SUCCESS_1" $LOGFILE && grep -qF "$IPV4_SUCCESS_2" $LOGFILE && grep -qF "$IPV6_SUCCESS_1" $LOGFILE && grep -qF "$IPV6_SUCCESS_2" $LOGFILE; then
-    echo "ok (took $took seconds)"
-else
-    echo "failed (after $took seconds)"
-    echo "Log from pppd"; echo "===== START pppd log ====="
-    cat $LOGFILE
-    echo "===== END pppd log ====="
-    fail "$PID" "Did not negotiate IPCP and IP6CP successfully."
-fi
-
-cleanup
-
 echo -n "Starting PPP peer (HDLC/RFC1662, IPv4 only)... "
 start=$(date +%s)
 launch_simple_pppd $CERT $KEY $HDLC_YES $NO_JUNK_COMP $IPV4_YES $IPV6_NO 2>&1

diff --git a/tests/ppp-over-tls-sync b/tests/ppp-over-tls-sync
new file mode 100755 (executable)
index 0000000..8216b3b
--- /dev/null
+++ b/tests/ppp-over-tls-sync
@@ -0,0 +1,71 @@
+#!/bin/sh
+#
+# Copyright © 2021 Daniel Lenski
+#
+# This file is part of openconnect.
+#
+# This is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public License
+# as published by the Free Software Foundation; either version 2.1 of
+# the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>
+
+srcdir=${srcdir:-.}
+top_builddir=${top_builddir:-..}
+
+. `dirname $0`/common.sh
+
+FINGERPRINT="--servercert=d66b507ae074d03b02eafca40d35f87dd81049d3"
+CERT=$certdir/server-cert.pem
+KEY=$certdir/server-key.pem
+
+# pppd is very poorly designed for mocking and testing in isolation, and running as non-root.
+# See launch_simple_pppd() in common.sh for a number of caveats about using it for these
+# purposes.
+
+IPV4_NO="noip"
+IPV4_YES="'169.254.1.1:169.254.128.128'" # needs single-quotes to escape for socat
+IPV6_NO="noipv6"
+IPV6_YES="+ipv6"
+OFFER_DNS="ms-dns 1.1.1.1 ms-dns 8.8.8.8"
+NO_HDR_COMP="nopcomp noaccomp"
+NO_JUNK_COMP="novj noccp"
+HDLC_YES=""
+HDLC_NO="sync"
+IPV4_SUCCESS_1="rcvd [IPCP ConfAck "
+IPV4_SUCCESS_2="sent [IPCP ConfAck "
+IPV6_SUCCESS_1="rcvd [IPV6CP ConfAck "
+IPV6_SUCCESS_2="sent [IPV6CP ConfAck "
+TIMEOUT_3S_IDLE="idle 3"
+
+echo "Testing PPP with 'synchronous' framing (plain RFC1661)... "
+
+echo -n "Starting PPP peer (sync/no-HDLC/plain-RFC1661, IPv4+IPv6, DNS, extraneous VJ and CCP)... "
+start=$(date +%s)
+launch_simple_pppd $CERT $KEY $HDLC_NO $IPV4_YES $OFFER_DNS $IPV6_YES 2>&1
+echo "started in $(( $(date +%s) - start )) seconds"
+wait_server "$PID"
+echo -n "Connecting to it with openconnect --protocol=nullppp... "
+start=$(date +%s)
+LD_PRELOAD=libsocket_wrapper.so $OPENCONNECT -q --protocol=nullppp $ADDRESS:443 -u test $FINGERPRINT --cookie "term" -Ss '' >/dev/null 2>&1
+took=$(( $(date +%s) - start ))
+if grep -qF "$IPV4_SUCCESS_1" $LOGFILE && grep -qF "$IPV4_SUCCESS_2" $LOGFILE && grep -qF "$IPV6_SUCCESS_1" $LOGFILE && grep -qF "$IPV6_SUCCESS_2" $LOGFILE; then
+    echo "ok (took $took seconds)"
+else
+    echo "failed (after $took seconds)"
+    echo "Log from pppd"; echo "===== START pppd log ====="
+    cat $LOGFILE
+    echo "===== END pppd log ====="
+    fail "$PID" "Did not negotiate IPCP and IP6CP successfully."
+fi
+
+cleanup
+
+exit 0