B ^m@sdZddlmZddlZddlZddlZddlZddlZddlZddl Z ddl Z ddl Z ddl Z ddl Z ddlZddlZddlZddlZddlZejdkreddZdZdZd Zd Zd ed ZGd ddZGdddZejGdddZejGdddZ GdddZ!Gddde!Z"GdddZ#GdddZ$dddd d!Z%d0dd#d#d#d#d$d%d&d'Z&d(d)d*d+Z'd,d)d-d.Z(e)d/kre(dS)1a Jetforce, an experimental Gemini server. Overview -------- GeminiServer: An asynchronous TCP server built on top of python's asyncio stream abstraction. This is a lightweight class that accepts incoming requests, logs them, and sends them to a configurable request handler to be processed. GeminiRequestHandler: The request handler manages the life of a single gemini request. It exposes a simplified interface to read the request URL and write the gemini response status line and body to the socket. The request URL and other server information is stuffed into an ``environ`` dictionary that encapsulates the request at a low level. This dictionary, along with a callback to write the response data, and passed to a configurable "application" function or class. JetforceApplication: This is a base class for writing jetforce server applications. It doesn't anything on its own, but it does provide a convenient interface to define custom server endpoints using route decorators. If you want to utilize jetforce as a library and write your own server in python, this is the class that you want to extend. The examples/ directory contains some examples of how to accomplish this. StaticDirectoryApplication: This is a pre-built application that serves files from a static directory. It provides an "out-of-the-box" gemini server without needing to write any lines of code. This is what is invoked when you launch jetforce from the command line. ) annotationsN)z*Fatal Error: jetforce requires Python 3.7+z0.2.2zJetforce Gemini Serverz Michael LazarzFloodgap Free Software Licensez(c) 2020 Michael Lazara  You are now riding on... _________ _____________ ______ /______ /___ __/_______________________ ___ _ /_ _ \ __/_ /_ _ __ \_ ___/ ___/ _ \ / /_/ / / __/ /_ _ __/ / /_/ / / / /__ / __/ \____/ \___/\__/ /_/ \____//_/ \___/ \___/ An Experimental Gemini Server, vz+ https://github.com/michael-lazar/jetforce c@sdeZdZdZdZdZdZdZdZdZ dZ d Z d Z d Z d Zd ZdZdZdZdZdZdZdZdZdZdS)Statusz' Gemini response status codes. ()*+,2345;<=>?@AN)__name__ __module__ __qualname____doc__INPUTSUCCESSZSUCCESS_END_OF_SESSIONREDIRECT_TEMPORARYREDIRECT_PERMANENTTEMPORARY_FAILUREZSERVER_UNAVAILABLE CGI_ERRORZ PROXY_ERRORZ SLOW_DOWNPERMANENT_FAILURE NOT_FOUNDZGONEPROXY_REQUEST_REFUSED BAD_REQUESTZCLIENT_CERTIFICATE_REQUIREDZTRANSIENT_CERTIFICATE_REQUESTEDAUTHORISED_CERTIFICATE_REQUIREDZCERTIFICATE_NOT_ACCEPTEDZFUTURE_CERTIFICATE_REJECTEDZEXPIRED_CERTIFICATE_REJECTEDr*r* /var/gopher/jetforce/jetforce.pyrLs,rc@seZdZdZddddZdS)RequestzM Object that encapsulates information about a single gemini request. dict)environcCs~||_|d|_tj|j}|js,td|js:d|_n|j|_|j|_|j|_|j |_ |j |_ tj |j |_ |j |_ dS)N GEMINI_URLz"URL must contain a `hostname` partgemini)r.urlurllibparseurlparsehostname ValueErrorschemeportpathparamsunquotequeryZfragment)selfr. url_partsr*r*r+__init__rs zRequest.__init__N)rrrrr?r*r*r*r+r,msr,c@s.eZdZUdZded<ded<dZded<dS) ResponsezN Object that encapsulates information about a single gemini response. intstatusstrmetaNz6typing.Union[None, bytes, str, typing.Iterator[bytes]]body)rrrr__annotations__rEr*r*r*r+r@s r@c@sjeZdZUdZdZded<dZded<dZded <d Zd ed <d Z d ed <dZ d ed<dddddZ dS) RoutePatternzF A pattern for matching URLs with a single endpoint or route. z.*rCr9r0r7Nztyping.Optional[str]r5Tboolstrict_hostname strict_portFstrict_trailing_slashr,ztyping.Optional[re.Match])requestreturncCs|jdkr|jd}n|j}t|jd}|jr>|j|kr>dS|jr\|jdk r\|j|kr\dS|jrr|j|jkrrdS|jr|j}n |j d}t |j|S)zL Check if the given request URL matches this route pattern. NHOSTNAME SERVER_PORT/) r5r.rArIrJr8r7rKr9rstripre fullmatch)r=rLZserver_hostnameZ server_portZ request_pathr*r*r+matchs    zRoutePattern.match) rrrrr9rFr7r5rIrJrKrTr*r*r*r+rGs       rGc@sTeZdZdZddZdddddd ZddddddddddZdddddZd S)JetforceApplicationa Base Jetforce application class with primitive URL routing. This is a base class for writing jetforce server applications. It doesn't anything on its own, but it does provide a convenient interface to define custom server endpoints using route decorators. If you want to utilize jetforce as a library and write your own server in python, this is the class that you want to extend. The examples/ directory contains some examples of how to accomplish this. cCs g|_dS)N)routes)r=r*r*r+r?szJetforceApplication.__init__r-ztyping.Callableztyping.Iterator[bytes])r. send_statusrMccsy t|}Wn tk r,|tjddSXx.|jdddD]\}}||r@Pq@W|j}||}||j|jt |j t r|j Vn,t |j t r|j Vn|j r|j EdHdS)NzUnrecognized URL format)r, Exceptionrr(rVrTdefault_callbackrBrD isinstancerEbytesrCencode)r=r.rWrL route_patterncallbackresponser*r*r+__call__s"      zJetforceApplication.__call__.*r0NTFrCztyping.Optional[str]rH)r9r7r5rIrKrMcs*t|||||dddfdd }|S)a Decorator for binding a function to a route based on the URL path. app = JetforceApplication() @app.route('/my-path') def my_path(request): return Response(Status.SUCCESS, 'text/plain', 'Hello world!') ztyping.Callable)funcrMcsj|f|S)N)rVappend)rc)r^r=r*r+wrapsz'JetforceApplication.route..wrap)rG)r=r9r7r5rIrKrer*)r^r=r+routeszJetforceApplication.router,r@)rLrMcCs ttjdS)z? Set the error response based on the URL type. z Not Found)r@rr%)r=rLr*r*r+rZsz$JetforceApplication.default_callback)rbr0NTF)rrrrr?rarfrZr*r*r*r+rUs rUcseZdZdZdddddfdd Zd d d d d Zddd dddZdddddZddddddZdddddZ d d d ddZ Z S) StaticDirectoryApplicationa Application for serving static files & CGI over gemini. This is a pre-built application that serves files from a static directory. It provides an "out-of-the-box" gemini server without needing to write any lines of code. This is what is invoked when you launch jetforce from the command line. If a directory contains a file with the name "index.gmi", that file will be returned when the directory path is requested. Otherwise, a directory listing will be auto-generated. /var/gemini index.gmicgi-binrC)root_directory index_file cgi_directorycsrt|jt|jft|jdd|_ | dd|_ ||_ t |_ |j dd|j dddS)NT)strictrPz text/geminiz.gmiz.gemini)superr?rVrdrGserve_static_filepathlibPathresolverootstriprmrl mimetypesZ MimeTypesZadd_type)r=rkrlrm) __class__r*r+r?s  z#StaticDirectoryApplication.__init__r,r@)rLrMc Cst|jd}ttjt|}|s@t|j drLt t j dS|j |}yt|tjsrt t j dSWntk rt t j dSX|rt| |j}t|tj}|r|r|||jS||j}||}t t j||S|r|jds:tj|j} | j|jdd} t t j | !S||j"} | #rf|| }t t jd|S|$||}t t jd|St t j dSdS)z Convert a URL into a filesystem path, and attempt to serve the file or directory that is represented at that path. rPz..z Not Found)r9z text/geminiN)%rqrrr9ruosnormpathrCZ is_absolutename startswithr@rr&rtaccessR_OKOSErroris_filermX_OKrun_cgi_scriptr.guess_mimetype load_filer is_direndswithr2r3r4r1_replacer"Zgeturlrlexistslist_directory) r=rLurl_pathfilenamefilesystem_pathZis_cgiZis_exeZmimetype generatorr>rlr*r*r+rp%s<         z,StaticDirectoryApplication.serve_static_filez pathlib.Pathr-)rr.rMc Cst|}|}d|d<||d<tj|gtj|dddd}|j}|jdd}t |d ksl|d  sxt t j d S|\}} tj|jd dd } t t|| | S)zu Execute the given file as a CGI script and return the script's stdout stream to the client. zGCI/1.1ZGATEWAY_INTERFACEZ SCRIPT_NAMETsurrogateescape)stdoutenvbufsizeZuniversal_newlineserrors)maxsplitrzUnexpected Errorzutf-8)encodingr)rCcopy subprocessPopenPIPErreadlinerusplitlen isdecimalr@rr$codecs iterencoderA) r=rr.Z script_nameZcgi_envoutZ status_lineZ status_partsrBrDrEr*r*r+rVs$  z)StaticDirectoryApplication.run_cgi_scriptztyping.Iterator[bytes])rrMc cs>|d*}|d}x|r.|V|d}qWWdQRXdS)zZ Load a file using a generator to allow streaming data to the TCP socket. rbiN)openread)r=rfpdatar*r*r+rvs   z$StaticDirectoryApplication.load_file)rrrMccsd|dV|j|kr0d|jdVxnt|D]^}|jdrRq>q>|r|d||jd|jdVq>d||jd|jdVq>Wd S) z` Auto-generate a text/gemini document based on the contents of the file system. z Directory: /z z=>/z .. .z/ z/  N)r]parentsortedZiterdirrzr{r)r=rrfiler*r*r+rs  "z)StaticDirectoryApplication.list_directory)rrMcCs.|j|\}}|r"|d|S|p(dSdS)zK Guess the mimetype of a file based on the file extension. z ; charset=z text/plainN)rvZ guess_type)r=rZmimerr*r*r+rsz)StaticDirectoryApplication.guess_mimetypecCsd|jdkrttjdS|j|jdkr2ttjdS|jrT|j|jdkrTttjdSttjdSdS)z Since the StaticDirectoryApplication only serves gemini URLs, return a proxy request refused for suspicious URLs. r0z)This server does not allow proxy requestsrNrOz Not FoundN)r7r@rr'r5r.r8r&)r=rLr*r*r+rZs z+StaticDirectoryApplication.default_callback)rhrirj) rrrrr?rprrrrrZ __classcell__r*r*)rwr+rgs  1   rgc@seZdZUdZdZded<ded<ded<d ed <d ed <d ed <ded<d ed<d ed<ded<ddddddZddddddZddddZddd d!Z dd dd"d#d$Z d%dd&d'd(Z ddd)d*Z ddd+d,Z ddd-d.Zd/S)0GeminiRequestHandleraV Handle a single Gemini Protocol TCP request. The request handler manages the life of a single gemini request. It exposes a simplified interface to read the request URL and write the gemini response status line and body to the socket. The request URL and other server information is stuffed into an ``environ`` dictionary that encapsulates the request at a low level. This dictionary, along with a callback to write the response data, and passed to a configurable "application" function or class. This design borrows heavily from the standard library's HTTP request handler (http.server.BaseHTTPRequestHandler). However, I did not make any attempts to directly emulate the existing conventions, because Gemini is an inherently simpler protocol than HTTP and much of the boilerplate could be removed. z%d/%b/%Y:%H:%M:%S %zzasyncio.StreamReaderreaderzasyncio.StreamWriterwriterztime.struct_timereceived_timestamprC remote_addrr- client_certr1rArBrDresponse_buffer response_size GeminiServerztyping.CallableNone)serverapprMcCs||_||_d|_dS)Nr)rrr)r=rrr*r*r+r?szGeminiRequestHandler.__init__)rrrMcs||_||_|dd|_|d|_t|_y|IdHWn,t k rp| t j d| IdHSXzby8|}|||j }x|D]}||IdHqWWn$t k r| t jdYnXWd| IdHXdS)a Main method for the request handler, performs the following: 1. Read the request bytes from the reader stream 2. Parse the request and generate response data 3. Write the response bytes to the writer stream ZpeernamerZpeercertNzMalformed requestzAn unexpected error occurred)rrZget_extra_inforrtime localtimer parse_headerrY write_statusrr(close_connection build_environr write_bodyr$)r=rrr.rrr*r*r+handles&     zGeminiRequestHandler.handleztyping.Dict[str, typing.Any])rMc Cstj|j}|j|jj|j|j|j|j|jjt |jj ddt d }|j rt dd|j dD}|d|dd |j d |j d |j d d |S)z Construct a dictionary that will be passed to the application handler. Variable names conform to the CGI spec defined in RFC 3875. ZGEMINIz jetforce/) r/rNZ PATH_INFOZ QUERY_STRINGZ REMOTE_ADDRZ REMOTE_HOSTZ SERVER_NAMErOZSERVER_PROTOCOLZSERVER_SOFTWAREcss|]}|dVqdS)rNr*).0xr*r*r+ sz5GeminiRequestHandler.build_environ..subjectZ CERTIFICATEZ commonNameZ notBeforeZnotAfterZ serialNumber)Z AUTH_TYPE REMOTE_USERZTLS_CLIENT_NOT_BEFOREZTLS_CLIENT_NOT_AFTERTLS_CLIENT_SERIAL_NUMBER)r2r3r4r1rr5r9r<rrCr8 __version__rr-updateget)r=r>r.rr*r*r+rs(  z"GeminiRequestHandler.build_environcs@|jdIdH}|dd}t|dkr2td||_dS)zq Parse the gemini header line. The request is a single UTF-8 line formatted as: s Niz$URL exceeds max length of 1024 bytes)rZ readuntilrr6decoder1)r=rr*r*r+rs   z!GeminiRequestHandler.parse_header)rBrDrMcCs"||_||_|d|d|_dS)aJ Write the gemini status line to an internal buffer. The status line is a single UTF-8 line formatted as: If the response status is 2, the meta field will contain the mimetype of the response data sent. If the status is something else, the meta will contain a descriptive message. The status is not written immediately, it's added to an internal buffer that must be flushed. This is done so that the status can be updated as long as no other data has been written to the stream yet. rz N)rBrDr)r=rBrDr*r*r+r*sz!GeminiRequestHandler.write_statusr\)rrMcs@|IdH|jt|7_|j||jIdHdS)z: Write bytes to the gemini response body. N) flush_statusrrrwritedrain)r=rr*r*r+r=s zGeminiRequestHandler.write_bodycsN|jrD|jsD|j}|jt|7_|j||jIdHd|_dS)zV Flush the status line from the internal buffer to the socket stream. Nr)rrr]rrrr)r=rr*r*r+rFs    z!GeminiRequestHandler.flush_statuscs*|IdH||jIdHdS)zA Flush any remaining bytes and close the stream. N)r log_requestrr)r=r*r*r+rQsz%GeminiRequestHandler.close_connectionc CsbyH|j|jdt|j|jd|jd|jd|j d|j Wnt k r\YnXdS)zY Log a gemini request using a format derived from the Common Log Format. z [z] "z" z "N) r log_messagerrstrftimeTIMESTAMP_FORMATrr1rBrDrAttributeError)r=r*r*r+rYs Bz GeminiRequestHandler.log_requestN)rrrrrrFr?rrrrrrrrr*r*r*r+rs* ""   rc@s^eZdZdZeZddddd dd d d d Zd dddZddd dddZdd dddZ dS)rz An asynchronous TCP server that uses the asyncio stream abstraction. This is a lightweight class that accepts incoming requests, logs them, and sends them to a configurable request handler to be processed. 127.0.0.1N localhostztyping.CallablerCrAzssl.SSLContextr)rhostr8 ssl_contextr5rMcCs"||_||_||_||_||_dS)N)rr8r5rr)r=rrr8rr5r*r*r+r?us zGeminiServer.__init__)rMc s|ttj|j|j|j|jdIdH}|d|jxV|j D]L}| ^}}}|j t j krz|d|d|qD|d|d|qDW|4IdH|IdHWdQIdHRXdS)z4 The main asynchronous server loop. )sslNzServer hostname is z Listening on :zListening on [z]:)rABOUTasyncioZ start_serveraccept_connectionrr8rr5Zsockets getsocknamefamilysocketAF_INETZ serve_forever)r=rsockZsock_ipZ sock_port_r*r*r+runs   zGeminiServer.runzasyncio.StreamReaderzasyncio.StreamWriter)rrrMcs4|||j}z|||IdHWd|XdS)zU Hook called by the socket server when a new connection is accepted. N)request_handler_classrrclose)r=rrZrequest_handlerr*r*r+rszGeminiServer.accept_connection)messagerMcCst|tjddS)z2 Log a diagnostic server message. )rN)printsysstderr)r=rr*r*r+rszGeminiServer.log_message)rrNr) rrrrrrr?rrrr*r*r*r+rks  rrCztyping.Tuple[str, str])r5rMcCstt|d}tt|d}|r@|sttd|tjd|d|d|dgddd t|t|fS) z Utility function to generate a self-signed SSL certificate key pair if one isn't provided. Results may vary depending on your version of OpenSSL. z.crtz.keyz"Writing ad hoc TLS certificate to z,openssl req -newkey rsa:2048 -nodes -keyout z -nodes -x509 -out z -subj "/CN="T)shellcheck) rqrrtempfileZ gettempdirrrrrrC)r5certfilekeyfiler*r*r+generate_ad_hoc_certificatesrrztyping.Optional[str]zssl.SSLContext)r5rrcafilecapathrMcCsZ|dkrt|\}}t}tj|_||||sJ|sJ|jtjjdn | |||S)ay Generate a sane default SSL context for a Gemini server. For more information on what these variables mean and what values they can contain, see the python standard library documentation: https://docs.python.org/3/library/ssl.html#ssl-contexts verify_mode: ssl.CERT_OPTIONAL A client certificate request is sent to the client. The client may either ignore the request or send a certificate in order perform TLS client cert authentication. If the client chooses to send a certificate, it is verified. Any verification error immediately aborts the TLS handshake. N)purpose) rrZ SSLContextZ CERT_OPTIONALZ verify_modeZload_cert_chainZload_default_certsZPurposeZ CLIENT_AUTHZload_verify_locations)r5rrrrcontextr*r*r+make_ssl_contexts   rzargparse.ArgumentParser)rMcCstjddtjd}|jddddtd|jd d d d |jd dtdd|jdddd |jddddd|jddddd|jddddd|jdd d!d"d|S)#z Construct the default argument parser when launching the server from the command line. These are meant to be application-agnostic arguments that could apply to any subclass of the JetforceApplication. jetforcez&An Experimental Gemini Protocol Server)prog descriptionZformatter_classz-Vz --versionversionz jetforce )actionrz--hostzServer address to bind toz 127.0.0.1)helpdefaultz--portzServer port to bind toi)rtyperz --hostnamezServer hostnamerz--tls-certfilerzServer TLS certificate fileFILE)destrmetavarz --tls-keyfilerzServer TLS private key filez --tls-cafilerz'A CA file to use for validating clientsz --tls-capathrz6A directory containing CA files for validating clientsDIR)argparseArgumentParserZArgumentDefaultsHelpFormatter add_argumentrrA)parserr*r*r+command_line_parsers< rrcCst}|jddddd|jddddd|jd d d d d|}t|j|j|j}t|j|j |j |j |j }t |j|j||j|d }t|dS)z> Entry point for running the static directory server. z--dirz)Root directory on the filesystem to servez /var/geminir)rrrz --cgi-dirz=CGI script directory, relative to the server's root directoryzcgi-binz --index-filezpIf a directory contains a file with this name, that file will be served instead of auto-generating an index pagez index.gmir)rr8rr5rN)rr parse_argsrgdirrlZcgi_dirrr5rrrrrrr8rr)rargsrrrr*r*r+ run_servers6r __main__)rNNNN)*r __future__rrrrZ dataclassesrvrxrqrRrrrrrrtypingZ urllib.parser2 version_infoexitrZ __title__ __author__Z __license__Z __copyright__rrr,Z dataclassr@rGrUrgrrrrrr rr*r*r*r+"sZ    ! (K07@",(