I am using Sphinx to document a webservice that will be deployed in different servers. The documentation is full of URL examples for the user to click and they should just work. My problem is that the host, port and deployment root will vary and the documentation will have to be re-generated for every deployment.
I tried defining substitutions like this:
|base_url|/path
.. |base_url| replace:: http://localhost:8080
But the generated HTML is not what I want (doesn't include "/path" in the generated link):
http://localhost:8080/path
Does anybody know how to work around this?
New in Sphinx v1.0:
sphinx.ext.extlinks – Markup to shorten external links
https://www.sphinx-doc.org/en/master/usage/extensions/extlinks.html
The extension adds one new config value:
extlinks
This config value must be a dictionary of external sites, mapping unique short alias names to a base URL and a prefix. For example, to create an alias for the above mentioned issues, you would add
extlinks = {'issue':
('http://bitbucket.org/birkenfeld/sphinx/issue/%s', 'issue ')}
Now, you can use the alias name as a new role, e.g. :issue:`123`. This then inserts a link to http://bitbucket.org/birkenfeld/sphinx/issue/123. As you can see, the target given in the role is substituted in the base URL in the place of %s.
The link caption depends on the second item in the tuple, the prefix:
If the prefix is None, the link caption is the full URL.
If the prefix is the empty string, the link caption is the partial URL given in the role content (123 in this case.)
If the prefix is a non-empty string, the link caption is the partial URL, prepended by the prefix – in the above example, the link caption would be issue 123.
You can also use the usual “explicit title” syntax supported by other roles that generate links, i.e. :issue:`this issue <123>`. In this case, the prefix is not relevant.
I had a similar problem where I needed to substitute also URLs in image targets.
The extlinks do not expand when used as a value of image :target: attribute.
Eventually I wrote a custom sphinx transformation that rewrites URLs that start with a given prefix, in my case, http://mybase/. Here is a relevant code for conf.py:
from sphinx.transforms import SphinxTransform
class ReplaceMyBase(SphinxTransform):
default_priority = 750
prefix = 'http://mybase/'
def apply(self):
from docutils.nodes import reference, Text
baseref = lambda o: (
isinstance(o, reference) and
o.get('refuri', '').startswith(self.prefix))
basetext = lambda o: (
isinstance(o, Text) and o.startswith(self.prefix))
base = self.config.mybase.rstrip('/') + '/'
for node in self.document.traverse(baseref):
target = node['refuri'].replace(self.prefix, base, 1)
node.replace_attr('refuri', target)
for t in node.traverse(basetext):
t1 = Text(t.replace(self.prefix, base, 1), t.rawsource)
t.parent.replace(t, t1)
return
# end of class
def setup(app):
app.add_config_value('mybase', 'https://en.wikipedia.org/wiki', 'env')
app.add_transform(ReplaceMyBase)
return
This expands the following rst source to point to English wikipedia.
When conf.py sets mybase="https://es.wikipedia.org/wiki" the links would point to the Spanish wiki.
* inline link http://mybase/Helianthus
* `link with text <http://mybase/Helianthus>`_
* `link with separate definition`_
* image link |flowerimage|
.. _link with separate definition: http://mybase/Helianthus
.. |flowerimage| image:: https://upload.wikimedia.org/wikipedia/commons/f/f1/Tournesol.png
:target: http://mybase/Helianthus
Ok, here's how I did it. First, apilinks.py (the Sphinx extension):
from docutils import nodes, utils
def setup(app):
def api_link_role(role, rawtext, text, lineno, inliner, options={},
content=[]):
ref = app.config.apilinks_base + text
node = nodes.reference(rawtext, utils.unescape(ref), refuri=ref,
**options)
return [node], []
app.add_config_value('apilinks_base', 'http://localhost/', False)
app.add_role('apilink', api_link_role)
Now, in conf.py, add 'apilinks' to the extensions list and set an appropriate value for 'apilinks_base' (otherwise, it will default to 'http://localhost/'). My file looks like this:
extensions = ['sphinx.ext.autodoc', 'apilinks']
# lots of other stuff
apilinks_base = 'http://host:88/base/'
Usage:
:apilink:`path`
Output:
http://host:88/base/path
You can write a Sphinx extension that creates a role like
:apilink:`path`
and generates the link from that. I never did this, so I can't help more than giving this pointer, sorry. You should try to look at how the various roles are implemented. Many are very similar to what you need, I think.
Related
I'm a computer science student. Recently we were tasked to develop a static HTTP server from scratch without using any HTTP modules, solely depending on socket programming. So this means that I had to write all the logic for HTTP message parsing, extracting headers, parsing URLs, etc.
However, I'm stuck with some confusion. As I'm somewhat experienced in web development before, I'm used to using URLs in places like anchor tags like this "/about", and "/articles/article-1".However, I've seen people sometimes people to relative paths according to their folder structure like this. "./about.html", "../contact.html".This always seemed to be a bad idea to me. However, I realized that even though in my code I'm not supporting these kinds of URLs explicitly, it seems to work anyhow.
Following is the python code I'm using to get the path from the HTTP message and then get the corresponding path in the file system.
def get_http_url(self, raw_request_headers: list[str]):
"""
Method to get HTTP url by parsing request headers
"""
if len(raw_request_headers) > 0:
method_and_path_header = raw_request_headers[0]
method_and_path_header_segments = method_and_path_header.split(" ")
if len(method_and_path_header_segments) >= 2:
"""
example: GET / HTTP/1.1 => ['GET', '/', 'HTTP/1.1] => '/'
"""
url = method_and_path_header_segments[1]
return url
return False
def get_resource_path_for_url(self, path: str | Literal[False]):
"""
Method to get the resource path based on url
"""
if not path:
return False
else:
if path.endswith('/'):
# Removing trailing '/' to make it easy to parse the url
path = path[0:-1]
# Split to see if the url also includes the file extension
parts = path.split('.')
if path == '':
# if the requested path is "/"
path_to_resource = os.path.join(
os.getcwd(), "htdocs", "index.html")
else:
# Assumes the user entered a valid url with resources file extension as well, ex: http://localhost:2728/pages/about.html
if len(parts) > 1:
path_to_resource = os.path.join(
os.getcwd(), "htdocs", path[1:]) # Get the abslute path with the existing file extension
else:
# Assumes user requested a url without an extension and as such is hoping for a html response
path_to_resource = os.path.join(
os.getcwd(), "htdocs", f"{path[1:]}.html") # Get the absolute path to the corresponding html file
return path_to_resource
So in my code, I'm not explicitly adding any logic to handle that kind of relative path. But somehow, when I use things like ../about.html in my test HTML files, it somehow works?
Is this the expected behavior? As of now (I would like to know where this behavior is implemented), I'm on Windows if that matters. And if this is expected, can I depend on this behavior and conclude that it's safe to refer to HTML files and other assets with relative paths like this on my web server?
Thanks in advance for any help, and I apologize if my question is not clear or well-formed.
I have a HTTP endpoint in Google Functions named process_me:
def process(request: Request) -> str:
return process_message(request)
which generates /process_me traces in GCP.
Now to trace a bit further I add OpenTelemetry:
def process(request: Request) -> str:
with TRACER.start_as_current_span("process_message") as span:
return process_message(request)
which in turn leads to traces no longer starting with /process_me. Instead they are named/have URL process_message.
Everything that is traced underneath the span is properly attached to process_message, though.
So I wonder how do I add Span process_message to Span /process_me (instead of replacing it)?
Notice that /process_me is the trace entry of the Loadbalancer. So I want this to go beyond Network Services.
The best I could come up with was:
def _build_trace_links(headers: EnvironHeaders) -> List[Link]:
"""
Tries to extract GCP trace context from headers.
Example header would be:
```
X-Cloud-Trace-Context: TRACE_ID/SPAN_ID;o=TRACE_TRUE
```
"""
trace_header: str = headers.get("X-Cloud-Trace-Context", "")
if not trace_header:
return []
ids, opts = trace_header.split(";", 1)[0]
b = opts.split("=", 1)[1]
if b.upper() != "TRACE_TRUE":
return []
trace_id, span_id = ids.split("/", 1)
parent_context = SpanContext(
trace_id=int(trace_id, 16),
span_id=int(span_id),
is_remote=True,
)
return [Link(context=parent_context)]
This way you can extract the parent and add at least a link to process_message.
I could see you found some relevant information already but it is not parent-child relationship in the trace and you are doing your own parsing and linking. I believe what you need here is the propagator that does all the work for you. I am not really super familiar with Google Cloud Function but I think this should help.
Install the propagator
python -m pip install opentelemetry-propagator-gcp
And the set the global text map propagator
...
from opentelemetry.propagate import set_global_textmap
from opentelemetry.propagators.cloud_trace_propagator import CloudTraceFormatPropagator
...
set_global_textmap(CloudTraceFormatPropagator())
...
In my documentation, I often refer to XML tags. So I try to write a small extension to simplify adding and referencing tags to the documentation. The documentation shall look like this:
My text, referencing to :xmltag-ref:`ExampleTag` here.
The :xmltag:`ExampleTag` Explained in Detail
--------------------------------------------
This section describes the tag...
I wrote a simple extension like this:
import re
from docutils import nodes
def xmltag(name, rawtext, text, lineno, inliner, options={}, content=[]):
literal_node = nodes.literal(text=text, classes=['xmltag'])
# <--- HERE: How to add target?
result_nodes = [literal_node]
return result_nodes, []
def xmltag_ref(name, rawtext, text, lineno, inliner, options={}, content=[]):
source = f'tag-{text}'
reference_node = nodes.reference(classes=['xmltag-ref'], text=text)
reference_node['refid'] = source
result_nodes = [reference_node]
return result_nodes, []
def setup(app):
app.add_role("xmltag-ref", xmltag_ref)
app.add_role("xmltag", xmltag)
return {
'version': '1.0',
'parallel_read_safe': True,
'parallel_write_safe': True,
}
If I add a reference manually, everything works as expected, but I could not find a way how to add a label automatically if the Tag is defined in a title.
I like how the automatic reference works, e.g. using the option directive:
.. option:: --example
text...
This automatically creates a label, which can be referenced using :option:'--example'. I would like to have something similar, but with the flexibility to use it in a title, which also creates an entry in the TOC.
How can I add a label in a role element?
Or, is there another way, how this should be solved correctly?
Using intersphinx and autodoc, having:
:param stores: Array of objects
:type stores: list[dict[str,int]]
Would result in an entry like:
stores (list[dict[str,int]]) - Array of objects.
Is there a way to convert list[dict[str,int]] outside of the autodoc :param: derivative (or others like :rtype:) with raw RST (within the docstring) or programatically given a 'list[dict[str,int]]' string?
Additionally, is it possible to use external links within the aforementioned example?
Example
Consider a script.py file:
def some_func(arg1):
"""
This is a head description.
:param arg1: The type of this param is hyperlinked.
:type arg1: list[dict[str,int]]
Is it possible to hyperlink this, here: dict[str,list[int]]
Or even add custom references amongst the classes: dict[int,ref]
Where *ref* links to a foreign, external source.
"""
Now in the Sphinx conf.py file add:
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.intersphinx'
]
intersphinx_mapping = {
'py': ('https://docs.python.org/3', None),
}
In your index.rst, add:
Title
=====
.. toctree::
:maxdepth: 2
.. autofunction:: script.some_func
And now simply make the html for the page.
The list[dict[str,int]] next to :type arg1: will be hyperlinked as shown at the beginning of this question, but dict[str,list[int]] obviously won't. Is there a way to make the latter behave like the former?
I reached a solution after digging around sphinx's code.
Injecting External References (:param:)
I created a custom extension that connects to the missing-reference event and attempts to resolve unknown references.
Code of reflinks.py:
import docutils.nodes as nodes
_cache = {}
def fill_cache(app):
_cache.update(app.config.reflinks)
def missing_reference(app, env, node, contnode):
target = node['reftarget']
try:
uri = _cache[target]
except KeyError:
return
newnode = nodes.reference('', '', internal = False, refuri = uri)
if not node.get('refexplicit'):
name = target.replace('_', ' ') # style
contnode = contnode.__class__(name, name)
newnode.append(contnode)
return newnode
def setup(app):
app.add_config_value('reflinks', None, False)
app.connect('builder-inited', fill_cache)
app.connect('missing-reference', missing_reference, priority = 1000)
Explanation
I consulted intersphinx's methodology for resolving unknown references and connected the function with high priority so it's hopefully only consulted as a last result.
Followup
Include the extenion.
Adding to conf.py:
reflinks = {'google': 'https://google.com'}
Allowed for script.py:
def some_func(arg1):
"""
:param arg1: Google homepages.
:type arg1: dict[str, google]
"""
Where dict[str, google] are now all hyperlinks.
Formatting Nested Types
There were instances where I wanted to use type structures like list[dict[str,myref]] outside of fields like :param:, :rtype:, etc. Another short extension did the trick.
Code of nestlinks.py:
import sphinx.domains.python as domain
import docutils.parsers.rst.roles as roles
_field = domain.PyTypedField('class')
def handle(name, rawtext, text, lineno, inliner, options = {}, content = []):
refs = _field.make_xrefs('class', 'py', text)
return (refs, [])
def setup(app):
roles.register_local_role('nref', handle)
Explanation
After reading this guide on roles, and digging here and here I realised that all I needed was a dummy field to handle the whole reference-making work and pretend like it's trying to reference classes.
Followup
Include the extension.
Now script.py:
def some_func(arg1):
"""
:param arg1: Google homepages.
:type arg1: dict[str, google]
Now this :nref:`list[dict[str,google]]` is hyperlinked!
"""
Notes
I am using intersphinx and autodoc to link to python's types and document my function's docstrings.
I am not well-versed in Sphinx's underlying mechanisms so take my methodology with a grain of salt.
The examples are provided are adjusted for the sake of being re-usable and generic and have not been tested.
The usability of such features is obviously questionable and only necessary when libraries like extlinks don't cover your needs.
Say that one wishes to convert all absolute svn:externals URLS to relative URLS throughout their repository.
Alternatively, if heeding the tip in the svn:externals docs ("You should seriously consider using explicit revision numbers..."), one might find themselves needing to periodically pull new revisions for externals in many places throughout the repository.
What's the best way to programmatically update a large number of svn:externals properties?
My solution is posted below.
Here's my class to extract parts from a single line of an svn:externals property:
from urlparse import urlparse
import re
class SvnExternalsLine:
'''Consult https://subversion.apache.org/docs/release-notes/1.5.html#externals for parsing algorithm.
The old svn:externals format consists of:
<local directory> [revision] <absolute remote URL>
The NEW svn:externals format consists of:
[revision] <absolute or relative remote URL> <local directory>
Therefore, "relative" remote paths always come *after* the local path.
One complication is the possibility of local paths with spaces.
We just assume that the remote path cannot have spaces, and treat all other
tokens (except the revision specifier) as part of the local path.
'''
REVISION_ARGUMENT_REGEXP = re.compile("-r(\d+)")
def __init__(self, original_line):
self.original_line = original_line
self.pinned_revision_number = None
self.repo_url = None
self.local_pathname_components = []
for token in self.original_line.split():
revision_match = self.REVISION_ARGUMENT_REGEXP.match(token)
if revision_match:
self.pinned_revision_number = int(revision_match.group(1))
elif urlparse(token).scheme or any(map(lambda p: token.startswith(p), ["^", "//", "/", "../"])):
self.repo_url = token
else:
self.local_pathname_components.append(token)
# ---------------------------------------------------------------------
def constructLine(self):
'''Reconstruct the externals line in the Subversion 1.5+ format'''
tokens = []
# Update the revision specifier if one existed
if self.pinned_revision_number is not None:
tokens.append( "-r%d" % (self.pinned_revision_number) )
tokens.append( self.repo_url )
tokens.extend( self.local_pathname_components )
if self.repo_url is None:
raise Exception("Found a bad externals property: %s; Original definition: %s" % (str(tokens), repr(self.original_line)))
return " ".join(tokens)
I use the pysvn library to iterate recursively through all of the directories possessing the svn:externals property, then split that property value by newlines, and act upon each line according to the parsed SvnExternalsLine.
The process must be performed on a local checkout of the repository. Here's how pysvn (propget) can be used to retrieve the externals:
client.propget( "svn:externals", base_checkout_path, recurse=True)
Iterate through the return value of this function, and and after modifying the property on each directory,
client.propset("svn:externals", new_externals_property, path)