We Keep Coding

How to manipulate xml based on the specific tags?

There's an XML something like this
<OUTER>
<TYPE>FIRST</TYPE>
<FIELD1>1</FIELD1>
<ID>55056</ID>
<TASK>
<FILE>
<OPTIONS>1</OPTIONS>
</FILE>
</TASK>
</OUTER>
<OUTER>
<TYPE>SECOND</TYPE>
<FIELD1>2</FIELD1>
<ID>58640</ID>
<TASK>
<FILE>
<OPTIONS>1</OPTIONS>
</FILE>
</TASK>
</OUTER>
The text in the tag ID needs to be updated with a new value, it's present in this variable NEW_ID1.The comparison should happen with the type tag, i.e only if the text == FIRST, we need to replace the ID with new ID, and write it back to XML similarly if type = SECOND, update ID with NEW_ID2 and so on,how to do so? I tried the following way,
tree = ET.parse("sample.xml")
root = tree.getroot()
det = tree.findall(".//OUTER[TYPE='FIRST']")
.
.
ID = NEW_ID1
tree.write("sample.xml")
but not able to manipulate it further

You are close, except TYPE isn't an attribute, it is a tag/element, so [TYPE='FIRST'] will not work.
Instead what you can do is iterate through all of the OUTER tags/elements, and test to see if they contain a TYPE with the value "FIRST" as text value. Then you can grab the OUTER tags ID decendant, and change it's text value.
For example:
tree = ET.parse("sample.xml")
root = tree.getroot()
for outer in tree.findall(".//OUTER"):
elem = outer.find(".//FIRST")
if elem.text == "FIRST":
id_elem = outer.find(".//ID")
id_elem.text = "NEWID1"
tree.write("sample.xml")
Note: I am assuming that your xml file doesn't only contain the markup that is in your question. There should only be one root element in an xml file.

How to get the xml element as a string with namespace using ElementTree in python?

I need to get the elements from xml as a string. I am trying with below xml format.
<xml>
<prot:data xmlns:prot="prot">
<product-id-template>
<prot:ProductId>PRODUCT_ID</prot:ProductId>
</product-id-template>
<product-name-template>
<prot:ProductName>PRODUCT_NAME</prot:ProductName>
</product-name-template>
<dealer-template>
<xsi:Dealer xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">DEALER</xsi:Dealer>
</dealer-template>
</prot:data>
</xml>
And I tried with below code:
from xml.etree import ElementTree as ET
def get_template(xpath, namespaces):
tree = ET.parse('cdata.xml')
elements = tree.getroot()
for element in elements.findall(xpath, namespaces=namespaces):
return element
namespace = {"prot" : "prot"}
aa = get_template(".//prot:ProductId", namespace)
print(ET.tostring(aa).decode())
Actual output:
<ns0:ProductId xmlns:ns0="prot">PRODUCT_ID</ns0:ProductId>
Expected output:
<prot:ProductId>PRODUCT_ID</prot:ProductId>
I should not remove the xmlns from the document where it presents in the document. And It has to be removed where it not presents. Example product-id-template is not containing the xmlns so it needs to be retrieved without xmlns. And dealer-template contains the xmlns so it needs to be retrieved with xmlns.
How to achieve this?

You can remove xmlns with regex.
import re
# ...
with_ns = ET.tostring(aa).decode()
no_ns = re.sub(' xmlns(:\w+)?="[^"]+"', '', with_ns)
print(no_ns)
UPDATE: You can do a very wild thing. Although I can't recommend it, because I'm not a Python expert.
I just checked the source code and found that I can do this hack:
def my_serialize_xml(write, elem, qnames, namespaces,
short_empty_elements, **kwargs):
ET._serialize_xml(write, elem, qnames,
None, short_empty_elements, **kwargs)
ET._serialize["xml"] = my_serialize_xml
I just defined my_serialize_xml, which calls ElementTree._serialize_xml with namespaces=None. And then, in dictionary ElementTree._serialize, I changed value for key "xml" to my_serialize_xml. So when you call ElementTree.tostring, it will use my_serialize_xml.
If you want to try it, just place the code(above) after from xml.etree import ElementTree as ET (but before using the ET).

How to parse the xml with xmlns attribute using python

<?xml version="1.0" ?>
<school xmlns="loyo:22:2.2">
<profile>
<student xmlns="loyo:5:542">
<marks>
<mark java="java:/lo">
<ca1>200</ca1>
</mark>
</marks>
</student>
</profile>
</school>
I trying to access the ca1 text. I am using etree but I cannot access it. I'm using below code.
import xml.etree.ElementTree as ET
tree = ET.parse('mca.xml')
root = tree.getroot()
def getElementsData(xpath):
elements = list()
if root.findall(xpath):
for elem in root.findall(xpath):
elements.append(elem.text)
return elements
else:
raise SystemExit("Invalid xpath provided")
t = getElementsData('.//ca1')
for i in t:
print(i)
I tried in different way to access it I don't know the exact problem. Is it recording file type issue?

Your document has namespaces on nodes school and student, you need to incorporate the namespaces in your search. Since you are looking for ca1, which is under student, you will need to specify the namespace that student node has:
import xml.etree.ElementTree as ET
tree = ET.parse('mca.xml')
root = tree.getroot()
def getElementsData(xpath, namespaces):
elements = root.findall(xpath, namespaces)
if elements == []:
raise SystemExit("Invalid xpath provided")
return elements
namespaces = {'ns_school': 'loyo:22:2.2', 'ns_student': 'loyo:5:542'}
elements = getElementsData('.//ns_student:ca1', namespaces)
for element in elements:
print(element)
Notes
Since your namespaces have no names, I gave them such names as ns_school, ns_student, but these name can be anything (e.g. ns1, mystudent, ...)
In a more complex system, I recommend raising some other kinds of errors and let the caller decide whether or not to exit.

How about traversing like this
import xml.etree.ElementTree
e = xml.etree.ElementTree.parse('test.xml').getroot()
data = e.getchildren()[0].getchildren()[0].getchildren()[0].getchildren()[0].getchildren()[0].text
print(data)

Try the following xpath
tree.xpath('//ca1//text()')[0].strip()

parse a xml file in python which nested tags are there

i just want to parse one xml file which is like as
<?xml version="1.0" encoding="UTF-8"?><Significant Major="3" Minor="0" Revision="1" xmlns="urn:reuterscompanycontent:significantdevelopments03"><RepNo>0091N</RepNo><CompanyName Type="Primary">XYZ</CompanyName><Production Date="2017-02-23T18:10:39" /><Developments><Development ID="3534388"><Dates><Source>2017-02-23T18:18:32</Source><Initiation>2017-02-23T18:18:32</Initiation><LastUpdate>2017-02-23T18:23:26</LastUpdate></Dates><Flags><FrontPage>0</FrontPage><Significance>1</Significance></Flags><Topics><Topic1 Code="254">Regulatory / Company Investigation</Topic1></Topics><Headline>FTC approves final order settling charges for Abbott's deal with St. Jude Medical</Headline></Development></Developments></Significant>
I just want to parse the Development tag and parse its every nested tag
i have below code:
import xml.etree.cElementTree as ET
tree = ET.ElementTree(file='../rawdata/SigDev_0091N.xml')
#get the root element
root = tree.getroot()
#print root.tag, root.attrib
for child in root:
#print child.tag, child.attrib
name = child.tag
print name
print 'at line 13'
if name is 'Developments':
print 'at line 15'
for devChild in name['Developments']:
print devChild.tag,devChild.attrib
it is not going inside the if block, i dont know why?

Checking name is 'Developments' always return false as child.tag is returning the value in {xmlns}tagname format.
For your case:
name = {urn:reuterscompanycontent:significantdevelopments03}Developments
You may refer to the answer of this question.
Simple string methods strip(), find(), split() or re can help you to skip the namespace for comparison.
Python documentation related: https://docs.python.org/2/library/xml.etree.elementtree.html#parsing-xml-with-namespaces

How can I strip namespaces out of an lxml tree?

Following on from Removing child elements in XML using python ...
Thanks to #Tichodroma, I have this code:
If you can use lxml, try this:
import lxml.etree
tree = lxml.etree.parse("leg.xml")
for dog in tree.xpath("//Leg1:Dog",
namespaces={"Leg1": "http://what.not"}):
parent = dog.xpath("..")[0]
parent.remove(dog)
parent.text = None
tree.write("leg.out.xml")
Now leg.out.xml looks like this:
<?xml version="1.0"?>
<Leg1:MOR xmlns:Leg1="http://what.not" oCount="7">
<Leg1:Order>
<Leg1:CTemp id="FO">
<Leg1:Group bNum="001" cCount="4"/>
<Leg1:Group bNum="002" cCount="4"/>
</Leg1:CTemp>
<Leg1:CTemp id="GO">
<Leg1:Group bNum="001" cCount="4"/>
<Leg1:Group bNum="002" cCount="4"/>
</Leg1:CTemp>
</Leg1:Order>
</Leg1:MOR>
How do I modify my code to remove the Leg1: namespace prefix from all of the elements' tag names?

One possible way to remove namespace prefix from each element :
def strip_ns_prefix(tree):
#iterate through only element nodes (skip comment node, text node, etc) :
for element in tree.xpath('descendant-or-self::*'):
#if element has prefix...
if element.prefix:
#replace element name with its local name
element.tag = etree.QName(element).localname
return tree
Another version which has namespace checking in the xpath instead of using if statement :
def strip_ns_prefix(tree):
#xpath query for selecting all element nodes in namespace
query = "descendant-or-self::*[namespace-uri()!='']"
#for each element returned by the above xpath query...
for element in tree.xpath(query):
#replace element name with its local name
element.tag = etree.QName(element).localname
return tree