Since a few weeks I get a lot of errors from your pubsub node due to none or an expired certificate on your side.
Jul 03 17:10:52 s2sin564fb8faed10 warn No certificate provided by pubsub.beta.kontalk.net
Jul 03 17:10:52 mod_s2s warn Forbidding insecure connection to/from pubsub.beta.kontalk.net
Were there any changes to your pubsub node? If there never was any certificate for pubsub.beta.kontalk.net, could you deploy one or is there anything why you do not want a cert there?
Hello and welcome!
Our pubsub node is an internal node living in the server beta.kontalk.net. You can’t really access it directly through pubsub.beta.kontalk.net, that’s a non-existant hostname.
But maybe I understand what you mean. Probably your server can’t handle this use case. What XMPP server are you using?
I am using prosody in latest version (trunk). I am connecting to users on kontalk via normal xmpp, so maybe that is why there are pubsub connections?
Exactly, but Prosody shouldn’t connect directly to pubsub.beta.kontalk.net, it should instead go through beta.kontalk.net. You should look up something in Prosody for this, maybe a configuration entry or a bug - Openfire had I bug like this IIRC.
It looks like your server tries to connect, not the other way around
Sep 04 20:14:58 s2sin55f013025110 debug Incoming s2s received <stream:stream from='pubsub .beta. kontalk .net' to='domain. org' version='1.0' xmlns='http:// etherx. jabber. org/streams'>
Sep 04 20:14:58 s2sin55f013025110 warn No certificate provided by pubsub .beta .kontalk .net
Sep 04 20:14:58 domain .org:watchuntrusted debug Checking certificate...
Sep 04 20:14:58 mod_s2s warn Forbidding insecure connection to/from pubsub. beta .kontalk .net
Sep 04 20:14:58 s2sin55f013025110 debug Sending[s2sin_unauthed]: <?xml version='1.0'?>
Sep 04 20:14:58 s2sin55f013025110 debug Sending[s2sin_unauthed]: <stream:stream xmlns='jabber:server' xmlns:stream='http ://etherx.j abb er. org/ streams' xml:lang='en' from='domain.o rg' id='68a0fdde-464b-4121-bb98-87c24805c28e' version='1.0' to='pubsub.beta.kontalk. net' xmlns:db='jabber:server:dialback'>
Sep 04 20:14:58 s2sin55f013025110 debug Disconnecting 46.101.92.252[s2sin_unauthed], <strea m:error> is: <stream:er ror><not-authorized xmlns='urn:ietf:params:xml:ns:xmpp-streams'/><text xmlns='urn:ietf:params:xml:ns:xmpp-streams'>Your server's certificate is invalid, expired, or not trusted by domain. org</text></stream:error>
Sep 04 20:14:58 s2sin55f013025110 debug Sending[s2sin_unauthed]: <stream:err or>
Sep 04 20:14:58 s2sin55f013025110 debug Sending[s2sin_unauthed]: </stream:stream>
Sep 04 20:14:58 s2sin55f013025110 info Incoming s2s stream pubsub.beta.kontalk.net->domain. org closed: Your server's certificate is invalid, expired, or not trusted by domain. org
Sep 04 20:14:58 s2sin55f013025110 debug Destroying incoming session pubsub.beta.kontalk.net->domain. org: Your server's certificate is invalid, expired, or not trusted by domain. org
Sep 04 20:14:58 s2sin55f013025110 debug s2s disconnected: <nil>-><nil> (connection closed)
That’s a dialback connection, because your server requested it. Do you have previous logs when your server decides to contact the pubsub host?
Seems like I missed your reply, sorry. Sadly I cannot provide prosody logs anymore as I migrated to ejabberd (needed for two auth sources at same time).
ejabberd debug logs with this “bug” (i use pastebin this time, shitty “link limit” in codeblock):
https://bin.disroot.org/?29cfbd86b400d92a#GTIVujkgVvZWpAvUTyiHs0NSU5XlQJYAHpG6l92E5JA=
https://bin.disroot.org/?37b6fa6683f10352#4HyLI8Gvlh+HVkep3aTymjgdOMB73t4edlveCyr2Ulk=
Sorry for the late reply. Could you try again now? I think I’ve fixed it somehow.
Yes, the last time the dialback thing happened was on 2019-11-04 21:14:44
Only this is left:
2019-11-04 21:39:40.596 [warning] <0.15687.26>@ejabberd_s2s_out:handle_auth_failure:219 (tls|<0.15687.26>) Failed outbound s2s EXTERNAL authentication domain.org -> beta.kontalk.net (46.101.92.252): Authentication failed: Peer provided no SASL mechanisms; most likely it doesn't accept our certificate
Out of interest, can you please explain what was wrong and how you fixed it? I saw that pubsub.beta.kontalk.net is now a valid hostname and resolves to the same IP as beta.kontalk.net does.
This happens again since a few weeks. I collected new debug logs with Prosody trunk nightly build 1253 (2020-04-14, 144a1ee24a4e) on a fresh start.
Apr 17 09:15:48 runnerPGipGubq debug creating new coroutine
Apr 17 09:15:49 socket debug server.lua: accepted new client connection from 46.101.92.252:57192 to 5269
Apr 17 09:15:49 s2sin55de1f28bb90 debug Incoming s2s connection
Apr 17 09:15:49 runner7SpVUdo5 debug creating new coroutine
Apr 17 09:15:49 s2sin55de1f28bb90 debug Incoming s2s received <stream:stream from='pubsub.beta.kontalk.net' to='domain.de' version='1.0' xmlns='http://etherx.jabber.org/streams'>
Apr 17 09:15:49 s2sin55de1f28bb90 debug Sending[s2sin_unauthed]: <?xml version='1.0'?>
Apr 17 09:15:49 s2sin55de1f28bb90 debug Sending[s2sin_unauthed]: <stream:stream xmlns='jabber:server' xmlns:stream='http://etherx.jabber.org/streams' xml:lang='en' from='domain.de' id='75dc148a-157b-4c6a-8b27-68254e63f8e4' version='1.0' to='pubsub.beta.kontalk.net' xmlns:db='jabber:server:dialback'>
Apr 17 09:15:49 mod_s2s debug Sending stream features: <stream:features><starttls xmlns='urn:ietf:params:xml:ns:xmpp-tls'><required/></starttls><dialback xmlns='urn:xmpp:features:dialback'/></stream:features>
Apr 17 09:15:49 s2sin55de1f28bb90 debug Sending[s2sin_unauthed]: <stream:features>
Apr 17 09:15:49 s2sin55de1f28bb90 debug Received[s2sin_unauthed]: <starttls xml:lang='en' xmlns='urn:ietf:params:xml:ns:xmpp-tls'>
Apr 17 09:15:49 s2sin55de1f28bb90 debug Sending[s2sin_unauthed]: <proceed xmlns='urn:ietf:params:xml:ns:xmpp-tls'>
Apr 17 09:15:49 socket debug server.lua: we need to do tls, but delaying until send buffer empty
Apr 17 09:15:49 s2sin55de1f28bb90 debug TLS negotiation started for s2sin_unauthed...
Apr 17 09:15:49 socket debug server.lua: attempting to start tls on tcp{client}: 0x55de1f20cd48
Apr 17 09:15:49 socket debug server.lua: ssl handshake done
Apr 17 09:15:49 s2sin55de1f28bb90 info Stream encrypted (TLSv1.2 with AES256-GCM-SHA384)
Apr 17 09:15:49 s2sin55de1f28bb90 debug Incoming s2s received <stream:stream from='pubsub.beta.kontalk.net' to='domain.de' version='1.0' xmlns='http://etherx.jabber.org/streams'>
Apr 17 09:15:49 s2sin55de1f28bb90 debug certificate chain validation result: invalid
Apr 17 09:15:49 s2sin55de1f28bb90 debug certificate error(s) at depth 0: EE certificate key too weak, self signed certificate
I’m sorry I totally forgot about your request.
I simply added the hostname as to the DNS records. The certificate stays the same. I probably should add pubsub.beta.kontalk.net as an alternative subject.
About the errors in the last few weeks: this is probably due to “loosing up” the hardening setting in Tigase to allow connections by some “older” Android devices. Unfortunately it is not possible to set two different hardening settings for client and server connections. I need to make some tests, I’ll get back to you. I was wrong sorry, hardened mode is already enabled.
Reading better your error… my certificate is issued by Let’s Encrypt, is the authority trusted by your configuration? I mean the level 0 one: O=Digital Signature Trust Co., CN=DST Root CA X3 which is self-signed. It should be included in the trust store that Prosody reads from.
It is, otherwise all other servers which use Let’s Encrypt certificates (most of them do) would throw an error too.
Ok. You showed me the error about the pubsub host, does it mean it works with the main host (beta.kontalk.net) but not the pubsub host? That would be even more weird since it’s the same server responding.
P.S. it might be related to the pubsub hostname not present in the certificate… I can fix that in the meantime.
P.P.S. May I have hostname and s2s port of a server using Let’s Encrypt that you successfully connect to?
Yes it works with beta .kontalk. net, I can chat, I can even see status (online, away) and also the status message a user can configure.
Most xmpp servers use Let’s Encrypt certificates, examples which I just work: 5222 .de, dismail .de
it might be related to the pubsub hostname not present in the certificate… I can fix that in the meantime.
i expect so, if you added it let me know, i will check again
I’ve added the pubsub hostname to the certificate. Connecting to beta.kontalk.net now returns this certificate:
Certificate:
Data:
Version: 3 (0x2)
Serial Number:
03:e7:0e:2b:bc:e1:27:84:42:4b:0a:4d:ac:7f:e7:53:b4:27
Signature Algorithm: sha256WithRSAEncryption
Issuer: C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
Validity
Not Before: Apr 21 16:50:24 2020 GMT
Not After : Jul 20 16:50:24 2020 GMT
Subject: CN = beta.kontalk.net
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
RSA Public-Key: (2048 bit)
Modulus:
00:c0:cb:9c:54:9f:ce:d9:02:99:e0:d4:5f:57:c0:
7b:96:59:f9:5c:0b:47:1d:47:39:fe:14:98:2c:d2:
75:07:e8:77:8d:a7:9f:a1:0f:db:b9:88:b4:3a:ec:
dd:ec:ac:2c:78:39:80:16:e8:69:c3:90:2a:06:34:
b7:67:4a:82:dd:29:c9:ed:5d:f1:ee:56:77:df:15:
ee:9d:0e:0a:5f:44:fe:cc:f7:49:57:21:fd:71:e9:
3a:d9:72:3c:6e:47:ee:98:1e:11:5a:7d:ca:5d:a0:
08:29:80:10:43:18:c2:48:f8:3a:d1:e2:c8:b7:5b:
ef:a3:81:2e:e2:03:38:36:7e:c1:a4:1a:52:25:23:
e7:70:00:98:8f:0d:89:f0:10:15:de:e7:93:d7:ec:
d5:2b:da:e5:a8:c7:76:8e:2b:3c:4a:39:25:a1:67:
9d:8b:eb:7f:c8:33:bc:1e:d9:29:8e:ee:96:35:18:
d8:2b:ec:fd:9c:40:2a:2c:3b:55:83:20:d3:d0:f2:
1e:1b:f7:5a:d2:59:b1:f5:12:e0:e6:18:27:6b:75:
dd:cd:d2:16:39:73:d6:e9:41:ba:06:f9:28:c5:19:
5a:0e:6e:7c:1c:7b:94:b1:45:4d:d8:6a:54:ad:40:
2f:7c:90:48:81:95:cf:0a:c5:ad:e7:df:72:81:c2:
0c:b7
Exponent: 65537 (0x10001)
X509v3 extensions:
X509v3 Key Usage: critical
Digital Signature, Key Encipherment
X509v3 Extended Key Usage:
TLS Web Server Authentication, TLS Web Client Authentication
X509v3 Basic Constraints: critical
CA:FALSE
X509v3 Subject Key Identifier:
D3:FC:45:02:7C:A6:89:03:21:85:4F:A9:E6:B6:D9:CD:74:4B:B5:AB
X509v3 Authority Key Identifier:
keyid:A8:4A:6A:63:04:7D:DD:BA:E6:D1:39:B7:A6:45:65:EF:F3:A8:EC:A1
Authority Information Access:
OCSP - URI:http://ocsp.int-x3.letsencrypt.org
CA Issuers - URI:http://cert.int-x3.letsencrypt.org/
X509v3 Subject Alternative Name:
DNS:beta.kontalk.net, DNS:pubsub.beta.kontalk.net
X509v3 Certificate Policies:
Policy: 2.23.140.1.2.1
Policy: 1.3.6.1.4.1.44947.1.1.1
CPS: http://cps.letsencrypt.org
CT Precertificate SCTs:
Signed Certificate Timestamp:
Version : v1 (0x0)
Log ID : 6F:53:76:AC:31:F0:31:19:D8:99:00:A4:51:15:FF:77:
15:1C:11:D9:02:C1:00:29:06:8D:B2:08:9A:37:D9:13
Timestamp : Apr 21 17:50:24.195 2020 GMT
Extensions: none
Signature : ecdsa-with-SHA256
30:45:02:21:00:A1:E6:0C:21:AD:6C:3A:33:DF:14:39:
66:0F:2E:72:28:E8:FF:4B:47:F9:0A:50:EF:38:98:37:
B3:8C:86:B3:4D:02:20:63:A3:60:14:43:62:98:60:D6:
D3:AB:99:1D:6C:AE:0E:52:3E:5B:09:28:29:73:FC:91:
73:EF:50:50:3B:96:01
Signed Certificate Timestamp:
Version : v1 (0x0)
Log ID : 07:B7:5C:1B:E5:7D:68:FF:F1:B0:C6:1D:23:15:C7:BA:
E6:57:7C:57:94:B7:6A:EE:BC:61:3A:1A:69:D3:A2:1C
Timestamp : Apr 21 17:50:24.208 2020 GMT
Extensions: none
Signature : ecdsa-with-SHA256
30:45:02:21:00:E1:56:37:F9:1E:A7:1A:9A:87:37:B8:
52:9F:12:A7:1D:5E:C8:3E:40:41:9C:A8:2B:EF:81:3E:
0C:07:B5:84:CF:02:20:6C:B8:03:13:FF:19:88:5C:92:
84:AF:FD:21:90:5F:EA:64:12:41:45:46:BE:16:CB:68:
2D:36:6C:5F:22:A3:AB
Signature Algorithm: sha256WithRSAEncryption
4d:1d:e7:b1:8b:ef:29:8a:4d:66:5f:5c:c2:de:79:af:5f:0c:
be:79:21:11:50:56:83:d8:b5:b1:85:6d:ef:37:f5:8a:19:2c:
52:38:be:ce:c4:25:98:da:95:e3:cb:93:29:20:f5:f0:30:5f:
ef:71:62:90:cd:50:23:d9:f2:71:7d:7d:41:f5:b3:13:86:00:
9e:cf:6a:98:14:08:10:78:5e:0e:ca:01:ae:52:ae:be:f8:79:
18:84:9e:c4:0c:e4:cb:20:55:ab:f0:bf:4d:11:46:3d:46:64:
73:c5:2c:b7:ca:b1:f2:9d:90:46:45:89:af:60:db:7c:cb:ad:
73:b0:60:1b:fe:04:a0:b0:f2:e9:9c:ad:15:80:05:70:8a:97:
b8:b7:8f:40:66:14:1f:ea:a2:8d:fb:bf:ff:46:4a:49:ad:f9:
54:be:9f:41:3a:ec:86:61:2b:43:bb:76:71:c8:17:3b:f2:69:
f0:93:bb:b7:e5:cc:e1:eb:43:48:ac:f3:b8:e1:23:69:c9:7c:
70:75:62:c6:0c:7d:f1:bd:bd:0e:ad:04:86:68:72:4b:17:8b:
c9:64:3b:c9:2a:a3:2c:c2:4c:08:a8:ee:00:4c:67:37:bc:bb:
a7:d3:54:13:f8:e8:a5:63:83:9e:9b:47:27:9c:74:bd:28:1c:
04:97:69:e5
Hmm, the error is gone now but I also cannot see any connection attempts from pubsub .beta .kontalk. net anymore. You probably restarted / reloaded the server on your side(?), maybe that is why.
I will report back if I spot a connection or error.
Yes I had to restart Tigase after generating the certificate.
It is happening again.
May 07 07:56:54 socket debug server.lua: accepted new client connection from 46.101.92.252:33288 to 5269
May 07 07:56:54 s2sin55d5c7c68220 debug Incoming s2s connection
May 07 07:56:54 runnerHlA6EwsL debug creating new coroutine
May 07 07:56:54 s2sin55d5c7c68220 debug Incoming s2s received <stream:stream from='pubsub.beta.kontalk.net' to='domain.de' version='1.0' xmlns='http://etherx.jabber.org/streams'>
May 07 07:56:54 s2sin55d5c7c68220 debug Sending[s2sin_unauthed]: <?xml version='1.0'?>
May 07 07:56:54 s2sin55d5c7c68220 debug Sending[s2sin_unauthed]: <stream:stream xmlns='jabber:server' xmlns:stream='http://etherx.jabber.org/streams' xml:lang='en' from='domain.de' id='82d682ae-7928-43df-9c76-fa1dacf8b03a' version='1.0' to='pubsub.beta.kontalk.net' xmlns:db='jabber:server:dialback'>
May 07 07:56:54 mod_s2s debug Sending stream features: <stream:features><dialback xmlns='urn:xmpp:features:dialback'/><starttls xmlns='urn:ietf:params:xml:ns:xmpp-tls'><required/></starttls></stream:features>
May 07 07:56:54 s2sin55d5c7c68220 debug Sending[s2sin_unauthed]: <stream:features>
May 07 07:56:55 s2sin55d5c7c68220 debug Received[s2sin_unauthed]: <starttls xml:lang='en' xmlns='urn:ietf:params:xml:ns:xmpp-tls'>
May 07 07:56:55 s2sin55d5c7c68220 debug Sending[s2sin_unauthed]: <proceed xmlns='urn:ietf:params:xml:ns:xmpp-tls'>
May 07 07:56:55 socket debug server.lua: we need to do tls, but delaying until send buffer empty
May 07 07:56:55 s2sin55d5c7c68220 debug TLS negotiation started for s2sin_unauthed...
May 07 07:56:55 socket debug server.lua: attempting to start tls on tcp{client}: 0x55d5c7c65be8
May 07 07:56:55 socket debug server.lua: ssl handshake done
May 07 07:56:55 s2sin55d5c7c68220 info Stream encrypted (TLSv1.2 with AES256-GCM-SHA384)
May 07 07:56:55 s2sin55d5c7c68220 debug Incoming s2s received <stream:stream from='pubsub.beta.kontalk.net' to='domain.de' version='1.0' xmlns='http://etherx.jabber.org/streams'>
May 07 07:56:55 s2sin55d5c7c68220 debug certificate chain validation result: invalid
May 07 07:56:55 s2sin55d5c7c68220 debug certificate error(s) at depth 0: EE certificate key too weak, self signed certificate
May 07 07:56:55 s2sin55d5c7c68220 warn Forbidding insecure connection to/from pubsub.beta.kontalk.net because its certificate is self-signed
May 07 07:56:55 s2sin55d5c7c68220 debug Sending[s2sin_unauthed]: <?xml version='1.0'?>
May 07 07:56:55 s2sin55d5c7c68220 debug Sending[s2sin_unauthed]: <stream:stream xmlns='jabber:server' xmlns:stream='http://etherx.jabber.org/streams' xml:lang='en' from='domain.de' id='b4ecb221-acfe-43f8-8436-98aaa6d9d0c0' version='1.0' to='pubsub.beta.kontalk.net' xmlns:db='jabber:server:dialback'>
May 07 07:56:55 s2sin55d5c7c68220 debug Disconnecting pubsub.beta.kontalk.net->domain.de[s2sin_unauthed], <stream:error> is: table: 0x55d5c7bd6410
May 07 07:56:55 s2sin55d5c7c68220 debug Sending[s2sin_unauthed]: <stream:error>
May 07 07:56:55 s2sin55d5c7c68220 debug Sending[s2sin_unauthed]: </stream:stream>
May 07 07:56:55 s2sin55d5c7c68220 info Incoming s2s stream pubsub.beta.kontalk.net->domain.de closed: Your server's certificate is self-signed
May 07 07:56:55 s2sin55d5c7c68220 debug Destroying incoming session pubsub.beta.kontalk.net->domain.de: Your server's certificate is self-signed
May 07 07:56:55 s2sin55d5c7c68220 debug s2s disconnected: <nil>-><nil> (connection closed)
May 07 07:56:55 socket debug server.lua: closed client handler and removed socket from list
The only difference I found is the certificate chain
---
Certificate chain
0 s:CN = beta.kontalk.net
i:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
1 s:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
2 s:C = US, O = Internet Security Research Group, CN = ISRG Root X1
i:C = US, O = Internet Security Research Group, CN = ISRG Root X1
---
vs
---
Certificate chain
0 s:CN = 5222.de
i:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
1 s:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
i:O = Digital Signature Trust Co., CN = DST Root CA X3
---
---
Certificate chain
0 s:CN = dismail.de
i:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
1 s:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
i:O = Digital Signature Trust Co., CN = DST Root CA X3
---
That’s a significant difference 
Ok I think I understand what is going on.
Let’s Encrypt intermediate CA is cross-signed by two different CAs. They say the ISRG root (which I’m currently providing in the chain the server sends to clients) should be widely accepted by now, but it seems some servers and clients are not up to date yet.
So my server is now providing the other chain (the DST one), like other servers are doing.
Could you check now please?
Still the same error
May 07 11:51:54 s2sin5611f9f38d10 debug certificate chain validation result: invalid
May 07 11:51:54 s2sin5611f9f38d10 debug certificate error(s) at depth 0: EE certificate key too weak, self signed certificate
May 07 11:51:54 s2sin5611f9f38d10 warn Forbidding insecure connection to/from pubsub.beta.kontalk.net because its certificate is self-signed
I wonder why it works fine with beta.kontalk.net but not with the pubsub subsubdomain.
Via openssl I now get the following - still different compared to the others
---
Certificate chain
0 s:CN = beta.kontalk.net
i:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
1 s:C = US, O = Let's Encrypt, CN = Let's Encrypt Authority X3
i:O = Digital Signature Trust Co., CN = DST Root CA X3
2 s:O = Digital Signature Trust Co., CN = DST Root CA X3
i:O = Digital Signature Trust Co., CN = DST Root CA X3
---
The last one is signed by itself?
They say the ISRG root should be widely accepted by now
That is interesting, since I found this issue on 3 different machines, Debian 9 and 10 + Ubuntu 18.04