How can I run Python code using Informatica command tasks - python

I'm attempting to run a Python script using an Informatica command task to scrape data from the web. Each time I start the workflow I get the error:
Command task instance [c_Run_Code]: execution of command [Test_Scrape] did not complete successfully with exit code [1]
I'm using the hash-bang: #!C\Python27\python.exe as the first line in the code, and the command I'm issuing is:
C\Python27\python.exe C\Documents\Python\Test_Scrape.py
Why do I continuously get this error?

I have found and fixed the problem. For those wishing to do something similar: either wrap the command in a batch or powershell file, or ensure that the python version installed has been added to the PATH.

Related

Execute Process Task in SSIS for Python Script

I'm trying to execute a python script by SSIS Execute Process Task. The Python script is correct. This script read data from yfinance and do some calculation, then export the calculation data into database. I need to execute this script every day. I make a batch file for execution of script file:
I used SSIS Execute SQL Task and configure properties follow:
But I received this error:
[Execute Process Task] Error: In Executing "D:....\batch_ssis.bat" "" at "", The process exit code was "9009" while the expected was "0".
That exit code 9009 comes from windows, it tries to execute batch_ssis.bat and can't resolve the information contained in the batch file
Make sure you do not have spaces in your path D:.....\ , if you do add quotations to your path so it looks like this "D:......"
Also ensure that python is installed and configured in your PATH windows environment variable so that you can call python from any directory within windows
Set FailTaskIfReturnCodeIsNotSuccessValue to False.
This worked for me today. It ignores whatever exit code the batch script returns.
In my case the expected job runs well but SSIS package failed because of the 9009 exit code returned. I still haven't figured out why.

Can I control a Powershell session from Python?

I want to initialise a powershell subprocess and be able to send commands to it and receive its output using Python on Windows.
Please can someone help with simple examples? I have gone through almost every other post similar to this on here but haven't had any luck with my specific problem...
I am to run this code to automate a current manual process conducted on Powershell.
This is a high level look at what I want to achieve using Python
I want to launch a powershell session using Python
Then CD into a directory
Run a powershell script in that directory
Display/retrieve all outputs
And pass new commands in the same session, again, whilst capturing outputs
So far all I have is the following code which launches powershell and runs a .ps1 file by giving directory. I'm not sure how to proceed from here or if this is even the best approach given the steps I need to undertake shown above.
p = subprocess.Popen(["powershell.exe", Powershell.exe -executionpolicy remotesigned -File 'G:\My Documents\helloworld.ps1';"]. stdout=sys.stdout)
p.communicate()
After running the above code, python terminal displays the message 'Test Message' correctly

Running python script from SAS Enterprise Guide

I need some help running a python script from inside SAS. I want to take advantage of the possibility of scheduling a SAS code to run every day, I already have a script in python that manipulates everything I need to do. Is there a way to host the python script into SAS and make it run every day without the need of my pc to be turned on?
You need to verify if you have the XCMD option enabled. If it's enabled, your log will show XCMD and then you can use an X statement to run your python scripts.
proc options option=xcmd;
run;
Next, find the command line that you'll need to run the scripts from the command line. Then place that after an X in your code or use CALL SYSTEM() or X to execute the command. Hardest part is usually making sure the quotes are correctly resolved.
x 'command line command goes here';

Import error when attempting to run python script from command line

I can run my python script within jupyter, but I want to be able to schedule it to run every morning by using windows task scheduler. In order to do this, I first need to make sure I can run it through the command line.
However, when I attempt to run the script I get an import error:
"ImportError: cannot import name 'ssl' from 'urllib3.util.ssl_' "
I do not understand why this happens and can I do to sort it out.
I have seen there similar questions where about running scripts from the command line but I do not see how can they answer this.

Cron job can't run bash script (that shells python script requiring xsession) that runs perfectly well from console

I am using dryscrape in a python script. The python script is called in a bash script, which is run by cron. For those who may not be aware, dryscrape is a headless browser (use QtWebkit in the background - so requires an xsession).
Here are the main points concerning the issue I'm having
When I run the python script from the command line, it works
When I run the bash script from the command line, it works too
I figured out that this may have something to do with different environments between my command prompt and when the cron job is running, so I modified my bash script to source my .profile as follows:
#/bin/bash
. /full/path/to/my/home/directory/.profile
python script_to_run.py
This is what my cronjob crontab entry looks like:
0,55 14-22 * * 1-5 /path/to/script.sh >> $(date "+/path/to/logs/\%Y\%m\%d.mydownload.log" )
By the way, I know that the job is being run (I can see entries in /var/log/syslog, and the script also writes to a log file - which is where I get the error message below):
In all cases, I got the following error message:
Could not connect to X server. Try calling dryscrape.start_xvfb()
before creating a session
I have installed the prerequisites, on my machine (obviously - since it runs at the command line). At the moment, I have run out of ideas.
What is causing the script to run fine at the console, and then fail when run by cron?
[[Relevant Details]]
OS: Linux 16.0.4 LTS
bash: version 4.3.46(1)
cron user: myself (i.e. same user at the command prompt)
dryscrape: version 1.0.1
The solution to this was to call the dryscrape.start_xvfb() method before starting the dryscrape session.
Cron user does not have display, so you cannot run any command which requires a display.
You need to modify the python script to do not use any type of display (check carefully, because some python commands, even though they do not open any display , they internally check for this variable).
The best way to test is to ssh into the machine without Display, and check if you can run it from there without erros.

Categories