I am trying to create a Jenkins job using Jenkins module in python. I am able to successfully connect with jenkins and perfrom get job_count as well as create_job() method.
In create_job() method i can perfrom this operation only with "jenkins.EMPTY_CONFIG_XML" parameter. How do i pass my own xml config file? below is my code, I have config saved on local, how to pass it by replacing EMPTY_CONFIG_XML. I tried few things, didn't work. Couldn't find it online. My below code is working. It's creating TestJob with EMPTY_CONFIG_XML. can someone please help how to pass customized XML file? Thank you for help!
import jenkins
import xml.etree.ElementTree as ET
server = jenkins.Jenkins("http://x.x.x.x:8080", username="foo", password="baar")
#print server.get_whoami()
server.create_job("TestJob",jenkins.EMPTY_CONFIG_XML)
Looking through the documentation for create_job, the config_xml parameter should be passed as a string representation of the xml.
I used a xml.etree.ElementTree to parse the XML file and convert it into a string:
import xml.etree.ElementTree as ET
def convert_xml_file_to_str():
tree = ET.parse(path_to_config_file)
root = tree.getroot()
return ET.tostring(root, encoding='utf8', method='xml').decode()
def main():
target_server = jenkins.Jenkins(url, username=username, password=password)
config = convert_xml_file_to_str()
target_server.create_job(job_name, config)
main()
I found this thread very helpful in understanding how to parse XML files, it also has a nice explanation about differences between Python2/3.
Related
I am trying to export a tableau view as an image/csv (doesn't matter) using Python. I googled and found that REST API would help here, so I created a Personal Access Token and wrote the following command to connect: -
import tableauserverclient as TSC
from tableau_api_lib import TableauServerConnection
from tableau_api_lib.utils.querying import get_views_dataframe, get_view_data_dataframe
server_url = 'https://tableau.mariadb.com'
site = ''
mytoken_name = 'Marine'
mytoken_secret = '$32mcyTOkmjSFqKBeVKEZYpMUexseV197l2MuvRlwHghMacCOa'
server = TSC.Server(server_url, use_server_version=True)
tableau_auth = TSC.PersonalAccessTokenAuth(token_name=mytoken_name, personal_access_token=mytoken_secret, site_id=site)
with server.auth.sign_in_with_personal_access_token(tableau_auth):
print('[Logged in successfully to {}]'.format(server_url))
It entered successfully and gave the message: -
[Logged in successfully to https://tableau.mariadb.com]
However, Iam at a loss now on how to access the tableau workbooks using Python. I searched here:-
https://help.tableau.com/current/api/rest_api/en-us/REST/rest_api_ref_workbooks_and_views.htm
but was unable to write these commands like GET or others in python.
Can anyone help?
I'm assuming you don't know the view_id of the view you're looking for
Adding this after the print in the with block will query all the views available on your site;
all_views, pagination_item = server.views.get()
print([view.name for view in all_views])
Then find the view you're looking for in the printed output and note the view_id for use like this;
view_item = server.view.get_by_id('d79634e1-6063-4ec9-95ff-50acbf609ff5')
From there, you can get the image like this;
server.views.populate_image(view_item)
with open('./view_image.png', 'wb') as f:
f.write(view_item.image)
The tableauserverclient-python docs should help you out a ton as well
https://tableau.github.io/server-client-python/docs/api-ref#views
I have an account in https://es.besoccer.com/ and they have an api for getting data in a xml.
I have this code in python for print the values of the xml I need:
from xml.dom import minidom
doc = minidom.parse("datos.xml")
partidos = doc.getElementsByTagName("matches")
for partido in partidos:
local = partido.getElementsByTagName("local")[0]
visitante = partido.getElementsByTagName("visitor")[0]
print("local:%s" % local.firstChild.data)
print("visitante:%s" % visitante.firstChild.data)
canales=partido.getElementsByTagName("channels")
for canal in canales:
nombre=canal.getElementsByTagName("name")[0]
print("canal:%s" % nombre.firstChild.data)
The problem is thatthe XML of this site is a url so I don´t know how to read the xml directly form the url. Other problem is that the xml contains some tags that are a link, and python throughs a error with that tags that contains a url.
Read the API docs here: https://www.besoccer.com/api/documentacion
After you understand which API call you need to use, prepare the URL and the query arguments and use a library like requests in order to read the data.
Once you have the reply (assuming it is XML based) - you can use your code and parse it.
I have the following code in Python 3. I am using the import xml.etree.ElementTree as ET for XML parsing. the webScraper grab the text from an webside but on that website there is text between the <link></link> tag, but the program returns None. I can se that the program finds all tags but where the tag result should be printed it only says None.
result = webScrapper.scrappPart("http://www.dn.se/rss/senaste-nytt/", "body")
root = ET.fromstring(result)
for items in root.findall('.//item'):
link = items.find('link')
print(link.text)
Does anyone know how to fix this?
Since your URL is actually an RSS feed, you'd be much better off using an RSS feed parser on it, instead of trying to roll your own. Fortunately, this is why feedparser exists. Check this out:
import feedparser as fp
feed = fp.parse("http://www.dn.se/rss/senaste-nytt/")
for entry in feed["entries"]:
print(entry["link"])
This returns
http://www.dn.se/sport/fotboll/cavani-het-i-svalt-psg/
http://www.dn.se/sport/fotbolls-em/kompany-missar-em/
http://www.dn.se/nyheter/sverige/livvaktens-slakting-fick-praktik-hos-sahlin-trots-myndighetens-avslag/
http://www.dn.se/sport/st-louis-andraperiod-avgjorde/
http://www.dn.se/nyheter/varlden/syrien-spanska-journalister-fria/
http://www.dn.se/sport/dansk-dynamit-ska-stoppa-tre-kronor/
http://www.dn.se/nyheter/sverige/mordmisstankt-slappt-ur-haktet-1/
http://www.dn.se/nyheter/varlden/ekonomiprofessor-loste-ekvation-togs-for-terrorist/
http://www.dn.se/sport/fotboll/leicester-firade-med-storseger/
http://www.dn.se/ekonomi/protester-mot-ny-granskontroll-urartade/
http://www.dn.se/sport/ishockey-vm/jimmie-ericsson-jag-ar-beredd-gora-allt-for-att-vinna/
http://www.dn.se/sport/ishockey-vm/schweiz-straffat-av-kazakstan/
http://www.dn.se/nyheter/varlden/natosoldater-dodade-i-afghanistan-2/
http://www.dn.se/sport/forsta-matchen-till-eslov/
http://www.dn.se/nyheter/sverige/drunknad-man-hittad-av-dykare/
http://www.dn.se/ekonomi/tagstopp-efter-olycka/
http://www.dn.se/sport/kristianstad-till-sm-final/
http://www.dn.se/sthlm/en-person-attackerad-med-kniv-i-centrala-stockholm/
http://www.dn.se/nyheter/sverige/inga-spar-efter-forsvunnen-22-arig-student/
http://www.dn.se/sport/fotboll/forlust-for-rydstrom-i-tranardebuten/
http://www.dn.se/nyheter/sverige/manga-grasbrander-runt-om-i-landet/
http://www.dn.se/nyheter/sverige/tre-gripna-efter-skottlossning-i-malmo/
http://www.dn.se/sport/fotboll/elfsborg-ar-med-i-toppen-igen/
http://www.dn.se/sport/em-silver-till-rissveds/
which I assume is what you're looking for.
You can use ElementTree just fine, you just need to pass the source and use the xpath:
from xml.etree import ElementTree as et
import requests
tree = et.fromstring(requests.get("http://www.dn.se/rss/senaste-nytt/").content)
print([x.text for x in tree.findall(".//item//link")])
Output:
['http://www.dn.se/nyheter/varlden/andlig-ledare-ihjalhackad-i-bangladesh/', 'http://www.dn.se/nyheter/sverige/tillstandet-battre-for-pakord-ettaring/', 'http://www.dn.se/ekonomi/maria-crofts-dags-att-gora-nagot-at-orattvisa-pensioner/', 'http://www.dn.se/nyheter/varlden/turkisk-militar-dodade-55-is-krigare/', 'http://www.dn.se/nyheter/varlden/massiv-fiskdod-i-sjo/', 'http://www.dn.se/nyheter/varlden/kanadabranden-i-bilder/', 'http://www.dn.se/nyheter/sverige/manga-saknas-efter-jordskred-i-kina/', 'http://www.dn.se/nyheter/sverige/fortsatt-sokande-efter-student/', 'http://www.dn.se/nyheter/sverige/en-dod-i-villabrand-8/', 'http://www.dn.se/nyheter/politik/v-vill-ta-bort-terrorstampel-pa-pkk/', 'http://www.dn.se/ekonomi/raknehjalp-pa-natet-ger-ratt-underhall/', 'http://www.dn.se/nyheter/varlden/kanadabranden-fullstandigt-okontrollerad/', 'http://www.dn.se/nyheter/varlden/attentat-mot-journalister-besvarande-for-erdogan/', 'http://www.dn.se/nyheter/varlden/superlobbyist-ska-gora-trump-serios/', 'http://www.dn.se/nyheter/vetenskap/karin-bojs-en-typisk-foralder-ar-28-ar-gammal/', 'http://www.dn.se/sport/nervos-vantan-pa-em-biljetter/', 'http://www.dn.se/ekonomi/ovantat-stort-exportfall-i-kina/', 'http://www.dn.se/ekonomi/lott-gav-35-miljarder-i-vinst-i-usa/', 'http://www.dn.se/nyheter/vetenskap/fabels-kansliga-nos-ska-ge-svar-om-massmordet/', 'http://www.dn.se/sport/johan-esk-nu-borde-idrotten-lara-ledarskap-av-naringslivet-1/', 'http://www.dn.se/sport/melker-karlsson-malskytt-for-san-jose/', 'http://www.dn.se/sport/backstrom-visade-vagen-till-washingtons-viktiga-vinst/', 'http://www.dn.se/nyheter/varlden/15-miljoner-signaturer-bekraftade/', 'http://www.dn.se/nyheter/varlden/medan-du-sov-varlden-i-korthet-8-maj-1/', 'http://www.dn.se/nyheter/varlden/karnvapen-bara-om-landet-hotas/', 'http://www.dn.se/nyheter/varlden/protester-mot-att-avskaffa-senat/', 'http://www.dn.se/nyheter/sverige/industri-brann-i-uppsala/', 'http://www.dn.se/nyheter/varlden/atta-poliser-dodade-i-attack/', 'http://www.dn.se/nyheter/varlden/tva-fast-vid-myr-utanfor-kiruna/', 'http://www.dn.se/sport/hockeyhasten-nyquist-vann-kentucky-derby/', 'http://www.dn.se/nyheter/varlden/15-miljoner-signaturer-bekraftade-i-venezuela/']
Or using lxml which can also get the source for you:
from lxml import etree
result = etree.parse("http://www.dn.se/rss/senaste-nytt/")
print(result.xpath("//item//link//text()"))
Which gives you the exact same output.
I am a newbie and having 1 week experience writing python scripts.
I am trying to write a generic parser (Library for all my future jobs) which parses any input XML without any prior knowledge of tags.
Parse input XML.
Get the values from the XML and Set the values basing on the tags.
Use these values in the rest of the job.
I am using the "xml.etree.ElementTree" library and i am able to parse the XML in the below mentioned way.
#!/usr/bin/python
import os
import xml.etree.ElementTree as etree
import logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
logger.info('start reading XML property file')
filename = "mood_ib_history_parameters_DEV.xml"
logger.info('getting the current location')
__currentlocation__ = os.getcwd()
__fullpath__ = os.path.join(__currentlocation__,filename)
logger.info('start parsing the XML property file')
tree = etree.parse(__fullpath__)
root = tree.getroot()
hive_db = root.find("hive_db").text
EDGE_HIVE_CONN = root.find("EDGE_HIVE_CONN").text
target_dir = root.find("target_dir").text
to_email_alias = root.find("to_email_alias").text
to_email_cc = root.find("to_email_cc").text
from_email_alias = root.find("from_email_alias").text
dburl = root.find("dburl").text
SQOOP_EDGE_CONN = root.find("SQOOP_EDGE_CONN").text
user_name = root.find("user_name").text
password = root.find("password").text
IB_log_table = root.find("IB_log_table").text
SR_DG_master_table = root.find("SR_DG_master_table").text
SR_DG_table = root.find("SR_DG_table").text
logger.info('Hive DB %s', hive_db)
logger.info('Hive DB %s', hive_db)
logger.info('Edge Hive Connection %s', EDGE_HIVE_CONN)
logger.info('Target Directory %s', target_dir)
logger.info('To Email address %s', to_email_alias)
logger.info('CC Email address %s', to_email_cc)
logger.info('From Email address %s', from_email_alias)
logger.info('DB URL %s',dburl)
logger.info('Sqoop Edge node connection %s',SQOOP_EDGE_CONN)
logger.info('Log table name %s',IB_log_table)
logger.info('Master table name %s',SR_DG_master_table)
logger.info('Data governance table name %s',SR_DG_table)
Now the question is if i want to parse an XML without any knowledge of the tags and elements and use the values how do i do it. I have gone through multiple tutorials but all of them help me with parsing the XML by using the tags like below
SQOOP_EDGE_CONN = root.find("SQOOP_EDGE_CONN").text
Can anybody point me to a right tutorial or library or a code snippet to parse the XML dynamically.
I think official documentation is pretty clear and contains some examples: https://docs.python.org/3/library/xml.etree.elementtree.html
The main part you need to implement is loop over the child nodes (potentially recursively):
for child in root:
# child.tag contains the tag name, child.attrib contains the attributes
print(child.tag, child.attrib)
Well parsing is easy as that - etree.parse(path)
Once you've got the root in hand using tree.getroot() you can just iterate over the tree using Python's "in":
for child_node in tree.getroot():
print child_node.text
Then, to see tags these child_nodes have, you do the same trick.
This lets you go over all tags in the XML without having to know the tag names at all.
in Python/Django, I need to parse and objectify a file .xml according to a given XMLSchema made of three .xsd files referring each other in such a way:
schema3.xsd (referring schema1.xsd)
schema2.xsd (referring schema1.xsd)
schema1.xsd (referring schema2.xsd)
xml schemas import
For this I'm using the following piece of code which I've already tested being succesfull when used with a couple of xml/xsd files (where .xsd is "standalone" without refering others .xsd):
import lxml
import os.path
from lxml import etree, objectify
from lxml.etree import XMLSyntaxError
def xml_validator(request):
# define path of files
path_file_xml = '../myxmlfile.xml'
path_file_xsd = '../schema3.xsd'
# get file XML
xml_file = open(path_file_xml, 'r')
xml_string = xml_file.read()
xml_file.close()
# get XML Schema
doc = etree.parse(path_file_xsd)
schema = etree.XMLSchema(doc)
#define parser
parser = objectify.makeparser(schema=schema)
# trasform XML file
root = objectify.fromstring(xml_string, parser)
test1 = root.tag
return render(request, 'risultati.html', {'risultato': test1})
Unfortunately, I'm stucked with the following error that i got with the multiple .xsd described above:
complex type 'ObjectType': The content model is not determinist.
Request Method: GET Request URL: http://127.0.0.1:8000/xml_validator
Django Version: 1.9.1 Exception Type: XMLSchemaParseError Exception
Value: complex type 'ObjectType': The content model is not
determinist., line 80
Any idea about that ?
Thanks a lot in advance for any suggestion or useful tips to approach this problem...
cheers
Update 23/03/2016
Here (and in the following answers to the post, because it actually exceed the max number of characters for a post), a sample of the files to figure out the problem...
sample files on GitHub
My best guess would be that your XSD model does not obey the Unique Particle Attribution rule. I would rule that out before looking at anything else.