Hi all,

For the impatient: Below the longish text is a fully self-contained Python example that illustrates my problem.

I'm struggling to understand xml.etree's handling of namespaces. I'm trying to parse an Inkscape document which uses several namespaces. From etree's documentation:

If the XML input has namespaces, tags and attributes with prefixes in the
form prefix:sometag get expanded to {uri}sometag where the prefix is
replaced by the full URI.

Which means that given an Element e, I cannot directly access its attributes using e.get() because in order to do that I need to know the URI of the namespace. So rather than doing this (see example below):

label = e.get('inkscape:label')

I need to do this:

label = e.get('{' + uri_inkscape_namespace + '}label')

...which is the method mentioned in etree's docs:

One way to search and explore this XML example is to manually add the URI
to every tag or attribute in the xpath of a find() or findall().
[...]
A better way to search the namespaced XML example is to create a
dictionary with your own prefixes and use those in the search functions.

Good idea! Better yet, that dictionary or rather, its reverse, already exists, because etree has used it to unnecessarily mangle the namespaces in the first place. The documentation doesn't mention where it can be found, but we can
just use the 'xmlns:' attributes of the <svg> root element to rebuild it. Or
so I thought, until I found out that etree deletes exactly these attributes before handing the <svg> element to the user.

I'm really stumped here. Apart from the fact that I think XML is bloated shit anyway and has no place outside HTML, I just don't get the purpose of etree's way of working:

1) Evaluate 'xmlns:' attributes of the <svg> element
2) Use that info to replace the existing prefixes by {uri}
3) Realizing that using {uri} prefixes is cumbersome, suggest to
the user to build their own prefix -> uri dictionary
to undo the effort of doing 1) and 2)
4) ...but witholding exactly the information that existed in the original
document by deleting the 'xmlns:' attributes from the <svg> tag

Why didn't they leave the whole damn thing alone? Keep <svg> intact and keep the attribute 'prefix:key' literally as they are. For anyone wanting to use
the {uri} prefixes (why would they) they could have thrown in a helper
function for the prefix->URI translation.

I'm assuming that etree's designers knew what they were doing in order to make my life easier when dealing with XML. Maybe I'm missing the forest for the trees. Can anybody enlighten me? Thanks!


#### self-contained example
import xml.etree.ElementTree as ET

def test_svg(xml):
root = ET.fromstring(xml)
for e in root.iter():
print(e.tag) # tags are shown prefixed with {URI}
if e.tag.endswith('svg'):
# Since namespaces are defined inside the <svg> tag, let's use the info
# from the 'xmlns:' attributes to undo etree's URI prefixing
print('Element <svg>:')
for k, v in e.items():
print(' %s: %s' % (k, v))
# ...but alas: the 'xmlns:' attributes have been deleted by the parser

xml = '''<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->

<svg
width="210mm"
height="297mm"
viewBox="0 0 210 297"
version="1.1"
id="svg285"
inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
sodipodi:docname="test.svg"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns="http://www.w3.org/2000/svg"
xmlns:svg="http://www.w3.org/2000/svg">
<sodipodi:namedview
id="namedview287"
pagecolor="#ffffff"
bordercolor="#000000"
borderopacity="0.25"
inkscape:showpageshadow="2"
inkscape:pageopacity="0.0"
inkscape:pagecheckerboard="0"
inkscape:deskcolor="#d1d1d1"
inkscape:document-units="mm"
showgrid="false"
inkscape:zoom="0.2102413"
inkscape:cx="394.78447"
inkscape:cy="561.25984"
inkscape:window-width="1827"
inkscape:window-height="1177"
inkscape:window-x="85"
inkscape:window-y="-8"
inkscape:window-maximized="1"
inkscape:current-layer="layer1" />
<defs
id="defs282" />
<g
inkscape:label="Ebene 1"
inkscape:groupmode="layer"
id="layer1">
<rect
style="fill:#aaccff;stroke-width:0.264583"
id="rect289"
width="61.665253"
height="54.114403"
x="33.978813"
y="94.38559" />
</g>
</svg>
'''

if __name__ == '__main__':
test_svg(xml)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

On 2022-10-19, Robert Latest <boblatest@yahoo.com> wrote:

If the XML input has namespaces, tags and attributes with prefixes
in the form prefix:sometag get expanded to {uri}sometag where the
prefix is replaced by the full URI.

Which means that given an Element e, I cannot directly access its attributes using e.get() because in order to do that I need to know the URI of the namespace.

That's because you *always* need to know the URI of the namespace,
because that's its only meaningful identifier. If you assume that a
particular namespace always uses the same prefix then your code will be completely broken. The following two pieces of XML should be understood identically:

<svg xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape">
<g inkscape:label="Ebene 1" inkscape:groupmode="layer" id="layer1">

and:

<svg xmlns:epacskni="http://www.inkscape.org/namespaces/inkscape">
<g epacskni:label="Ebene 1" epacskni:groupmode="layer" id="layer1">

So you can see why e.get('inkscape:label') cannot possibly work, and why e.get('{http://www.inkscape.org/namespaces/inkscape}label') makes sense.

The xml.etree author obviously knew that this was cumbersome, and
hence you can do something like:

namespaces = {'inkspace': 'http://www.inkscape.org/namespaces/inkscape'}
element = root.find('inkspace:foo', namespaces)

which will work for both of the above pieces of XML.

But unfortunately as far as I can see nobody's thought about doing the
same for attributes rather than tags.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

I mean, it's worth to look at BeautifulSoup source how do they do that.
With BS I work with attributes exactly as you want, and I explicitly
tell BS to use lxml parser.

Axy.

On 19/10/2022 14:25, Robert Latest via Python-list wrote:

Hi all,

For the impatient: Below the longish text is a fully self-contained Python example that illustrates my problem.

I'm struggling to understand xml.etree's handling of namespaces. I'm trying to
parse an Inkscape document which uses several namespaces. From etree's documentation:

If the XML input has namespaces, tags and attributes with prefixes in the
form prefix:sometag get expanded to {uri}sometag where the prefix is
replaced by the full URI.

Which means that given an Element e, I cannot directly access its attributes using e.get() because in order to do that I need to know the URI of the namespace. So rather than doing this (see example below):

label = e.get('inkscape:label')

I need to do this:

label = e.get('{' + uri_inkscape_namespace + '}label')

...which is the method mentioned in etree's docs:

One way to search and explore this XML example is to manually add the URI
to every tag or attribute in the xpath of a find() or findall().
[...]
A better way to search the namespaced XML example is to create a
dictionary with your own prefixes and use those in the search functions.

Good idea! Better yet, that dictionary or rather, its reverse, already exists,
because etree has used it to unnecessarily mangle the namespaces in the first place. The documentation doesn't mention where it can be found, but we can just use the 'xmlns:' attributes of the <svg> root element to rebuild it. Or so I thought, until I found out that etree deletes exactly these attributes before handing the <svg> element to the user.

I'm really stumped here. Apart from the fact that I think XML is bloated shit anyway and has no place outside HTML, I just don't get the purpose of etree's way of working:

1) Evaluate 'xmlns:' attributes of the <svg> element
2) Use that info to replace the existing prefixes by {uri}
3) Realizing that using {uri} prefixes is cumbersome, suggest to
the user to build their own prefix -> uri dictionary
to undo the effort of doing 1) and 2)
4) ...but witholding exactly the information that existed in the original
document by deleting the 'xmlns:' attributes from the <svg> tag

Why didn't they leave the whole damn thing alone? Keep <svg> intact and keep the attribute 'prefix:key' literally as they are. For anyone wanting to use the {uri} prefixes (why would they) they could have thrown in a helper function for the prefix->URI translation.

I'm assuming that etree's designers knew what they were doing in order to make
my life easier when dealing with XML. Maybe I'm missing the forest for the trees. Can anybody enlighten me? Thanks!

#### self-contained example
import xml.etree.ElementTree as ET

def test_svg(xml):
root = ET.fromstring(xml)
for e in root.iter():
print(e.tag) # tags are shown prefixed with {URI}
if e.tag.endswith('svg'):
# Since namespaces are defined inside the <svg> tag, let's use the info
# from the 'xmlns:' attributes to undo etree's URI prefixing
print('Element <svg>:')
for k, v in e.items():
print(' %s: %s' % (k, v))
# ...but alas: the 'xmlns:' attributes have been deleted by the parser

xml = '''<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<!-- Created with Inkscape (http://www.inkscape.org/) -->

<svg
width="210mm"
height="297mm"
viewBox="0 0 210 297"
version="1.1"
id="svg285"
inkscape:version="1.2.1 (9c6d41e410, 2022-07-14)"
sodipodi:docname="test.svg"
xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape"
xmlns:sodipodi="http://sodipodi.sourceforge.net/DTD/sodipodi-0.dtd"
xmlns="http://www.w3.org/2000/svg"
xmlns:svg="http://www.w3.org/2000/svg">
<sodipodi:namedview
id="namedview287"
pagecolor="#ffffff"
bordercolor="#000000"
borderopacity="0.25"
inkscape:showpageshadow="2"
inkscape:pageopacity="0.0"
inkscape:pagecheckerboard="0"
inkscape:deskcolor="#d1d1d1"
inkscape:document-units="mm"
showgrid="false"
inkscape:zoom="0.2102413"
inkscape:cx="394.78447"
inkscape:cy="561.25984"
inkscape:window-width="1827"
inkscape:window-height="1177"
inkscape:window-x="85"
inkscape:window-y="-8"
inkscape:window-maximized="1"
inkscape:current-layer="layer1" />
<defs
id="defs282" />
<g
inkscape:label="Ebene 1"
inkscape:groupmode="layer"
id="layer1">
<rect
style="fill:#aaccff;stroke-width:0.264583"
id="rect289"
width="61.665253"
height="54.114403"
x="33.978813"
y="94.38559" />
</g>
</svg>
'''

if __name__ == '__main__':
test_svg(xml)

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

I have no idea why, I used to remove namespaces, following the advice
from stackoverflow:

https://stackoverflow.com/questions/4255277/lxml-etree-xmlparser-remove-unwanted-namespace

_ns_removal_xslt_transform = etree.XSLT(etree.fromstring('''
    <xsl:stylesheet version="1.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
    <xsl:output method="xml" indent="no"/>

    <xsl:template match="/|comment()|processing-instruction()">
        <xsl:copy>
          <xsl:apply-templates/>
        </xsl:copy>
    </xsl:template>

    <xsl:template match="*">
        <xsl:element name="{local-name()}">
          <xsl:apply-templates select="@*|node()"/>
        </xsl:element>
    </xsl:template>

    <xsl:template match="@*">
        <xsl:attribute name="{local-name()}">
          <xsl:value-of select="."/>
        </xsl:attribute>
    </xsl:template>
    </xsl:stylesheet>
'''))

xml_doc = _ns_removal_xslt_transform(

    etree.fromstring(my_xml_data)

)


Later on, when I worked with SVG, I used BeautifulSoup.

Axy.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)

Jon Ribbens wrote:

That's because you *always* need to know the URI of the namespace,
because that's its only meaningful identifier. If you assume that a particular namespace always uses the same prefix then your code will be completely broken. The following two pieces of XML should be understood identically:

<svg xmlns:inkscape="http://www.inkscape.org/namespaces/inkscape">
<g inkscape:label="Ebene 1" inkscape:groupmode="layer" id="layer1">

and:

<svg xmlns:epacskni="http://www.inkscape.org/namespaces/inkscape">
<g epacskni:label="Ebene 1" epacskni:groupmode="layer" id="layer1">

So you can see why e.get('inkscape:label') cannot possibly work, and why e.get('{http://www.inkscape.org/namespaces/inkscape}label') makes sense.

I get it. It does.

The xml.etree author obviously knew that this was cumbersome, and
hence you can do something like:

namespaces = {'inkspace': 'http://www.inkscape.org/namespaces/inkscape'}
element = root.find('inkspace:foo', namespaces)

which will work for both of the above pieces of XML.

Makes sense. It forces me to make up my own prefixes which I can then safely use in my code rather than relying on the xml's generator to not change their prefixes.

BTW, I only now thought to look at what actually is at Inkscape's namespace URI, and it turns out to be quite a nice explanation of what a namespace is and why it looks like a URL.

--- SoupGate-Win32 v1.05
* Origin: fsxNet Usenet Gateway (21:1/5)