Comment by 🚀 ingrix
@stack @skyjake I can get to spell.ddns.net with my own pxc client while using client certificates, but my certs are all self-signed. skyjake, are you using a CA-signed cert? try with a self-signed one and see if you get the same error. the cert-checking code is probably too aggressive and I'll need to relax it.
Sep 03 · 3 months ago
33 Later Comments ↓
@stack also if you set the global config option 'index_file=index.gmi' it will do what skyjake recommended re automatically routing games/ to games/index.gmi
@stack
I updated polluxd to libpxd-d49c92d-20250903, which has more relaxed client cert checking and will pass the cert's subject as REMOTE_USER to cgi scripts. Let me know how that does for you.
@ingrix This is a Lagrange-generated self-signed certificate. It is only version 1 which may cause some issues with modern TLS:
Certificate:
Data:
Version: 1 (0x0)
Serial Number: 0 (0x0)
Signature Algorithm: sha256WithRSAEncryption
Issuer: CN=skyjake
Validity
Not Before: Feb 24 03:17:13 2021 GMT
Not After : Dec 31 22:00:00 2099 GMT
Subject: CN=skyjake, emailAddress=jaakko.keranen@iki.fi
Subject Public Key Info:
Public Key Algorithm: rsaEncryption
Public-Key: (2048 bit)
Modulus: (...)
Exponent: 65537 (0x10001)
@ingrix: Thank you, will try that out tomorrow.
xxxxThe examples cannot be built as is, btw. Include and lib directories do not match current source tree.xxxx
Sorry, I see they are built from the main Makefile
@ingrix, the provided code (libpxd-d49c92f) crashes on first (non-cgi) request!
Importantly, when I request a simple .gmi file, for some reason the output is:
info polluxd.c 451: accepted connection on socket 10 from listening socket 9 error polluxd.c: 519: assumption violation, gemini step didn't clean path info polluxd.cgi_c: 86: cgi hepler 81651: cgi request socket closed ... (5 more, one per process I guess...)
@stack
Thanks for the report, that's the first I've seen that issue, I'll try to track down the cause. From your other post it sounds like you're not really interested in continuing to use this one, but would you mind telling me what OS you're running on, and if you recall seeing any warnings when building the software? I am running the same code on ingrix.info without this issue so that info would be helpful for the development effort.
Also, the 'cgi helper' output is from the cgi helper processes dying after the main server asserted out in polluxd.c:519
I will rebuild it tomorrow and send you the details.
The other post is more of a future direction; I would love to use your server, and appreciate your help.
@ingrix, I rebuilt it with no issues, and started it on port 10965.
As before, I got:
info polluxd.c: 451: accepted connection on socket 10 from listening socket 9 error polluxd.c: 519: assumption violation, gemini step didn't clean path info polluxd_cgi.c: 86: cgi helper 86698: cgi request socket closed info polluxd_cgi.c: 86: cgi helper 86696: cgi request socket closed info polluxd_cgi.c: 86: cgi helper 86694: cgi request socket closed info polluxd_cgi.c: 86: cgi helper 86692: cgi request socket closed info polluxd_cgi.c: 86: cgi helper 86690: cgi request socket closed ...
My system:
.-.
.-'``(|||)
,`\ \ `-`. 88 88
/ \ '``-. ` 88 88
.-. , `___: 88 88 88,888, 88 88 ,88888, 88888 88 88
(:::) : ___ 88 88 88 88 88 88 88 88 88 88 88
`-` ` , : 88 88 88 88 88 88 88 88 88 88 88
\ / ,..-` , 88 88 88 88 88 88 88 88 88 88 88
`./ / .-.` '88888' '88888' '88888' 88 88 '8888 '88888'
`-..-( )
`-`
Linux Version 6.14.0-1012-aws, Compiled #12~24.04.1-Ubuntu SMP Fri Aug 15 00:16:
Two 2.5GHz Intel Xeon Cascadelake X Processors, 914M RAM, 10k Bogomips
ip-172-31-39-9
@stack thanks for the additional info! I'll spin up an ubuntu vm and see if I can reproduce what you're seeing. I have still had no luck on my Gentoo or OpenBSD machines, so maybe it's a compiler or library versioning issue. If I can't reproduce it out-of-the-box I may need more info about your polluxd.conf and/or directory hierarchy, but let me see what I can do first.
Oh, I guess it shuts down all processes. My browser displays 'incomplete header'.
@ingrix, is it possible to run two or more instances of polluxd? I tried renaming the binary but I think it daemonizes using 'polluxd' no matter what, so I am only running the one on port 1965 for now, as there is not much to see on 10965 (incomplete header, when it's running).
@stack copying or symlinking the binary should work just fine to run it with a different name as long as you don't have both instances running on the same listening address/port. I tested this briefly with no issues. If you run CGI then you'll see a lot of helper processes with the same name as the main polluxd.
Re: it killing all processes, yes, the 'error' message from polluxd.c:519 is actually fatal (I'll fix the name in the next patch). The server only uses a single process with async IO, the cgi helper processes shut down when the main server dies.
@stack This is interesting. To confirm, you have one instance running on 1965 normally and when you try running on 10965 you get the failures/crashes you're describing? That's an interesting data point if so, I'll try some more tests with that consideration.
@ingrix -- were you able to reproduce the error?
It seems pretty odd, given that no cgi is invoked but the cgi subsystem crashes.
The conf file :
drop_user=stack
drop_group=stack
host=spell.ddns.net
listen_addr=any
port=10965
cert_file=gemkey/my.crt
key_file=gemkey/my.key
#chroot_dir=/home/stack/public_gemini
docroot=/home/stack/public_gemini
location /{
action = file
}
location /games/spell/cgi{
action = cgi
}
location /games/wordo/cgi{
action = cgi
}
location /games/ensemble/game{
action = cgi
}
location /games/wall/cgi{
action = cgi
}
location /games/weather{
action = cgi
}
location /games/env.cgi{
action = cgi
}
Not seeing any errors so far but the leaderboard seems gone? Anyway glad I found this post, the tilde hosted games are down regularly these days 😭
@stack no luck yet, even on the ubuntu vm. You may want to double-check that your instance of polluxd is using the correct libpxd.so - I have seen strange errors kind of like yours when testing when the testing version of polluxd tried to use the system-wide libpxd.so instead of a freshly built testing version. Otherwise, your configuration file will be helpful, thank you! I'll use that one verbatim and see if there's anything that shows up.
I will be on an airplane for most of the day but I'll report back what I find.
Oh, that is probably what happened. When I built the original version I installed it in /usr/bin and /usr/lib! Thanks for the tip, it will probably cure the problem.
@ingrix, that was it! I totally forgot about the library, started up the updated server in its own directory, and as it started (apparently) fine, totally spaced on the lib.
Thank you -- at this point I think I have a working system.
The only minor point I have is whether null input is allowed. The previous 3 (or was it 4) servers that hosted SpellBinding allowed a null-input URL "...?", which I used to shuffle. It appears polluxd does not like it and resends client request. I am not clear on what the spec intends after glancing over it just now.
In any case it does not interfere with the operation of the system.
@ingrix,
Null responses
It looks like (correct me if I am wrong), upon receivng a request with a ? but nothing following, polluxd does not place the null response into the environment variable. To a CGI, an empty response never arrives, it is as if the user never responded to the query, forcing a re-issued query.
I would argue that it is a bug, and sending a null response is different than sending no response.
Other servers I've used allowed the CGI to see the null and act accordingly.
My games relied on that behaviour in a minor way, and are a bit wacky (especially first time around, upon entry because my urls often end in a ? to request a gamefield redraw)
I use blank input to shuffle or redraw puzzles, and it adds a keystroke or you wind up in a loop because the game never sees the null input and assumes there was no input at all.
@stack I couldn't respond earlier but saw your reply, and then noticed exactly what you say when playing SpellBinding - I couldn't get the letters to shuffle. I agree with your analysis and I also agree that it is a bug. I won't be able to fix it tonight, but can probably get to it tomorrow. I'll let you know when I post the patch.
Thanks again for the feedback.
@ingrix thank you, whenever you can. I truly appreciate your help and was at the end of my wits before you offered help.
Also as pointed out by others, is it possible to serve index.gmi when a bare directory is requested?
@stack See comment above:
@stack
I reworked the URL parsing code. This version should report present-but-empty queries - at least I can see empty queries when requesting CGI, anyway. It hasn't been tested exhaustively yet to make sure the URL parsing changes didn't violate some assumption I made elsewhere in the main library (tests are broken and need to be fixed), so if you see any strange behavior please let me know. I'll also do more testing tonight. If any important bugfixes are made I'll let you know about that too.
@ingrix, just deployed it. Looks ok so far. Thanks again
@ingrix, I found the server dead this morning.
unk px connection.c: 370: overflow, buffer sizes need adjustment info c polluxd_cgi.c: 86: cgi helper 113038: cgi request ...
@ingrix, the server has been running for a couple of days (since you provided the latest tarball) prior to crashing.
I set up a cron entry to restart it daily in the meantime.
Let me know if I can do more logging or something that helps.
Looking at the source, it seems to be converting address to string, so kind of strange.
You may want to not abort on things like that, but just close the connection?
@stack thanks for the report, you're probably right that closing the connection makes more sense. I'll get a fix out with that and more asap
@ingrix - is there something like a more traditional log, with a line per request containing time, client address, and full request url? As I have it, the server spits maybe 10 lines of mostly-unhelpful stuff to stderr (I think)
@stack
That has a quick-fix for the issue you're encountering, and the "buffers" have been adjusted to be large enough to hopefully avoid issue. Now it will print "WARNING: peer string is longer than expected..." if address strings exceed expectations - if you can, please let me know if you see any of those so I can figure out what weird case you managed to run in to (I thought I had the math right on those originally but I guess not).
@stack I have the logging set up that way because it helps me figure out what happened if things go wrong in the process, but that's probably less useful for end-users.... The logging does support multiple levels of verbosity so I can make a change to print what you request instead. I worry a bit about spitting out the whole request because on other applications the request may contain e.g. passwords, but I'll figure something out.
@ingrix - perhaps the configuration file can keep an opt-in parameter for logging entire request, defaulting to trim query
@stack I think something like that is a good way to go, let's see how far we get with it. It'll probably take a day or so to get all of the changes put in place the right way, but stay tuned.
@ingrix: I deployed libpxd-c0144d1
No great rush - things that crash are bad, but things like logging format - not so much...
Original Post
New SpellBinding Server — I am happy to announce that, finally, SpellBinding and other games have a dedicated server. After moving the games from tilde to tilde I am looking forward to 100% uptime. Soft open to iron out any remaining issues [gemini link] Let me know if there are any dangling links or anything else ungodly.