Skip to content

Require the peer fingerprint before completing a dtls_gw handshake - #182

Merged
sobomax merged 4 commits into
sippy:masterfrom
arfa-21:dtls-require-peer-fingerprint
Nov 15, 2026
Merged

sobomax merged 4 commits into
sippy:masterfrom
arfa-21:dtls-require-peer-fingerprint

Conversation

@arfa-21

@arfa-21 arfa-21 commented Nov 7, 2026

Copy link
Copy Markdown
Contributor

Repro: U ... && M4:1 S, then, before the A/P for that leg arrives, send a DTLS ClientHello with any self-signed certificate to the returned port. The handshake completes and SRTP is keyed with that peer. The real A is then refused with "cannot change mode from 0 to 2 when in the 2 state", so relayed media keeps going to (and being accepted from) whoever got there first.
Cause: rtpp_dtls_conn_dtls_recv() answers the handshake in RDC_INIT before any peer fingerprint is set, and the check after the handshake is skipped when pvt->fingerprint is empty.
Fix: drop DTLS packets until A/P has supplied the fingerprint (the peer retransmits its ClientHello), and make the comparison unconditional.

The dtls scenario used by tests/dtls_gw depended on the old behaviour: in the second IPv6 round fwd's gena-facing endpoint is created with S and never gets gena's fingerprint, because the A goes to the geno-facing stream via L6. It now gets it with U6 before gena starts the handshake.

I validated the change by linking the real rtpp_dtls_conn.c/rtpp_dtls.c against an OpenSSL DTLS client. Before the change, a handshake made before A completes and rtp_send protects with its keys. After it, the early ClientHello is ignored and the retransmit completes only once A carries the matching fingerprint; a mismatching fingerprint still fails as before. I could only do this on macOS, where rtpproxy itself doesn't build, so tests/dtls_gw with the adjusted scenario has not been run locally.

An endpoint created by the S command has no remote fingerprint until
the A or P command supplies one, but rtpp_dtls_conn_dtls_recv() still
answered a ClientHello in that state and, once the handshake finished,
skipped the fingerprint check because pvt->fingerprint was empty. Any
host that reached the advertised port between S and A/P could complete
the handshake with a throwaway self-signed certificate: the SRTP
contexts were keyed with that peer, the A/P command that followed was
refused ("cannot change mode ... when in the 2 state") and relayed
media stayed encrypted to, and accepted from, that peer.

Drop DTLS packets while no peer fingerprint is configured, the peer
retransmits its ClientHello and the handshake completes once A/P has
arrived. The check after the handshake no longer has a no-fingerprint
case.

The dtls scenario used by tests/dtls_gw relied on this: in the second
gena's fingerprint (the A went to the geno-facing stream through L6).
Give it gena's fingerprint with U6 before gena starts the handshake.

Validated by driving the real rtpp_dtls_conn object with an OpenSSL
DTLS client: before the change a handshake made before A completes and
keys SRTP, after it the handshake only completes once A has supplied
the matching fingerprint, and a mismatching one still fails.
@sobomax
sobomax force-pushed the dtls-require-peer-fingerprint branch from cf581aa to 1b072ce Compare November 10, 2026 15:31
@sobomax

sobomax commented Nov 10, 2026

Copy link
Copy Markdown
Member

@arfa-21 thanks for submission, this indeed makes sense. However, it looks like this made dtls_gw test flaky:

Getting session stats...
Failed action: ".eval: %%G_STATS4%%"
	expanded: "npkts_ina=400 npkts_ino=2400 nrelayed=2799 ndropped=1"
Traceback (most recent call last):
  File "/opt/rtpproxy/basic/python/RPTL/rptl_run.py", line 794, in setvar
    output_var.handler(self, value)
  File "/opt/rtpproxy/basic/python/RPTL/rptl_run.py", line 384, in str_compare
    raise AssertionError(F'"{res}" != "{expected}"')
AssertionError: "npkts_ina=400 npkts_ino=2400 nrelayed=2799 ndropped=1" != "npkts_ina=400 npkts_ino=2400 nrelayed=2800 ndropped=0"

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/opt/rtpproxy/basic/python/RPTL/rptl_run.py", line 992, in <module>
    raise srun.ex_info.exception
  File "/opt/rtpproxy/basic/python/RPTL/rptl_run.py", line 468, in guarded
    func(*args)
  File "/opt/rtpproxy/basic/python/RPTL/rptl_run.py", line 508, in _issue_next_cmd
    self.internal_ops[sname](cmd, command)
  File "/opt/rtpproxy/basic/python/RPTL/rptl_run.py", line 574, in handle_eval
    self.setvar(cmd.output_var, command)
  File "/opt/rtpproxy/basic/python/RPTL/rptl_run.py", line 796, in setvar
    raise Exception(F'"{value}" -> {output_var.name}{output_var.args}: failed') from ex
Exception: "npkts_ina=400 npkts_ino=2400 nrelayed=2799 ndropped=1" -> str_compare('G_STATS_CHK_1',): failed
FAILED: (rc=1) running DTLS gateway RPTL script
FAIL: dtls_gw/basic
======================================
Testsuite summary

Looks like one packet arrived earlier that the dtls session has been setup. Any ideas how to fix?

@arfa-21

arfa-21 commented Nov 13, 2026

Copy link
Copy Markdown
Contributor Author

I don't think this one comes from the patch itself. The check that failed is the first IPv4 round (G_STATS_CHK_1), and that leg never reaches either changed path: fwd is the active side there and gets geno's fingerprint from the P on its L before it sends the ClientHello, and geno already has fwd's from the A on its U. So nothing arrives in RDC_INIT without a fingerprint, and the strcmp gives the same answer as before (same for the IPv6 leg in that round). The same commit also went green on all 27 functional jobs on 7 Nov, before the rebase.

Going through the scenario, it looks like a plain timing race. The IPv4 handshake starts on fwd's L ... P, and the first streamed packet only comes a handful of commands and a .sleep: 0.1 later. On a busy runner with the debug build the handshake can overrun that slightly, and the first packet hits rtp_send()/srtp_recv() while the conn isn't RDC_UP yet, which is exactly one ndropped. The same window exists before the second streaming phase, and my reordering does make it a bit tighter there for IPv6, since gena's U6 ... P is now the last command before the sleep.

Simplest fix is a longer sleep before both DTLS streaming phases. 1s leaves plenty of headroom and is small next to the 25s of traffic. A deterministic version would need the module to expose the handshake state so the script can .until on it, since nothing in Qv reports it at the moment. I can push the sleep change here, or as a separate commit if you'd rather keep it out of this PR.

Add the Q module subcommand to report none, init, connecting, up or dead
without allocating or reconfiguring an endpoint. Read the state under
the existing connection lock; up means the fingerprint is verified and
both SRTP contexts are installed. Document query usage and stream
selection.
Use the new state query to replace handshake sleeps with bounded polling
of both IPv4/IPv6 peers, including the oversize test. Preserve exact
packet-count assertions. Add a standalone security regression using real
ClientHello retransmissions: reject pre-fingerprint responses, then
require recovery with the matching fingerprint. Cover unconfigured,
connecting and rejected endpoints. The regression fails on both address
families when the PR security changes, or only the early-packet gate,
are reverted.
@sobomax
sobomax force-pushed the dtls-require-peer-fingerprint branch from 9cae0bf to 77bd6ad Compare November 14, 2026 23:28
@sobomax
sobomax merged commit 0c50700 into sippy:master Nov 15, 2026
188 of 195 checks passed
@sobomax

sobomax commented Nov 15, 2026

Copy link
Copy Markdown
Member

Merged via rebase. Thank you, @arfa-21, for finding the early-DTLS handshake issue, providing a clear reproducer, and contributing the fingerprint-gating fix! We added readiness polling and IPv4/IPv6 regression coverage that catches the previous behavior and verifies recovery after the matching fingerprint is supplied. The GitHub Actions run passed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants