Error mark in eclipse/pydev when running ipython -i - python

I work interactively on data in a ipython console. I have a small script which I can use to Import new data, but normally I want to keep the old data in RAM too. So what I did is to add the following piece of code to the Import script:
try:
data_list.append(data)
except NameError:
data_list = [data]
I execute this script from the ipython console with run -i scriptname.py. This adds the new data to the data_list, or creates a new list if it doesn't exist. This works basically fine.
Now the question: in eclipse/pydev, I get an error sign, because the Parser recognizes that data_list has not be assigned before. Is there any way to make the error sign disappear?

Related

Is there any way to run another cell of R code which depends on current python flask app cell's input by user in google-colaboratory?

I am running a flask app in google colab using flask_ngrok and in that I am asking user to upload a .pssm file.
Now I want to run a R package at same time as it depends on file uploaded by user, along with app cell running (basically I want to jump back and forth to cells without affecting current cell which is running my app).
I'm using %load_ext rpy2.ipython to use R and Python in one notebook.
I have seen some multithreading packages like parsl but problem is that I want R cell not Python cell.
Or Is there any other way to this problem like making request to another colab notebook and getting response in current notebook (getting a Response : csv file, Request with a: input file).
Is it possible technically?
I solved my problem like this:
filenm=""
%%R -i filenm -o fun1
f1<-function(filenm)
{
d1<-NULL
d1<-rbind(d1,some_function(filenm))
write.csv(d1,file="/content/t1.csv")
}
Here -i is used for passing input variable from python to R in our case "filenm" and -o is used to pass the output variable from R to python in our case we get fun1 as output in python
Now that R function is converted into python so that we can use it like this:
fun1(filenm) #in Python shell
Hope this helps 😊

How to fix: No file output from remote wmic called via Python WMI module through win32_Process.Create

I'm attempting to execute WMIC on a remote host through the WMI module via Win32_Process.Create() function and save the output on said remote host via /output, more or > but neither of these is generating the expected output. Instead, I am receiving empty files.
I have tried using /output, more and > to achieve proper output but the resultant files are always blank. I know that I am creating processes with expected output otherwise because I have tried using 'ipconfig >' into a file on the remote host and it gives the expected output with network configuration data, it is only WMIC that is giving me trouble. I have also tried removing the 'cmd.exe /c' portion and just running directly with wmic.exe with no luck as well as specifying the exact executable location.
I've also tried using /Output:FILENAME alone and /Output:STDOUT with the redirector (>) to a file. None seem to work properly.
import wmi
session = wmi.WMI(computer="192.168.8.132", user="testing", password="testing")
process_startup = session.Win32_ProcessStartup.new()
command = r"wmic.exe /output:C:\Users\testing\WMIUsers0.txt /namespace:\\root\cimv2 UserAccount get * /value"
process_id, return_value = session.Win32_Process.Create(CommandLine=command, ProcessStartupInformation=process_startup)
print("Process Executed: "+command)
print("Process ID: "+str(process_id)+" , Return Value (0 = Success): "+str(return_value)+"\n")
The above code generates an empty file on the host at the expected location rather than the actual WMI returned data. I copy pasted the 'command' given above into the remote host and it works as expected for stdout as well as generating the data within the file.
This is only an approximation of my code but represents the issue well - what do I have to modify to have actual results populate in the destination file rather than a blank file when running remotely over the WMI module?
edit: Using ProcessHacker on the target, I can see my process started with the parameters 'cmd.exe /c wmic /output:"C:\Users\testing\WMIUsers0.txt" /namespace:\root\cimv2 UserAccount get * /value' - If I copy paste this into a separate cmd window, it works as expected and generates the file but when I run it remotely, even though it has the proper parameters, it generates an empty file. I am going insane trying to figure out why this isn't working..
Edit2: I've tried to many different variations utilizing subprocess.call, check_output and other things and am having no luck trying to do this in Python. Today is not my day.
Edit3: Decided to just use subprocess.getoutput and store the data locally because getting it to execute remotely was driving me insane - No idea why it won't work after inspecting what was executing - same exact cmdline window parameters but one generates blank file, other was dataful. Crazy, has to be some functionality of the WMI module preventing proper transfer..or something...no idea.

Can't get working command line on prompt to work on subprocess

I need to extract text from a PDF. I tried the PyPDF2, but the textExtract method returned an encrypted text, even though the pdf is not encrypted acoording to the isEncrypted method.
So I moved on to trying accessing a program that does the job from the command prompt, so I could call it from python with the subprocess module. I found this program called textExtract, which did the job I wanted with the following command line on cmd:
"textextract.exe" "download.pdf" /to "download.txt"
However, when I tried running it with subprocess I couldn't get a 0 return code.
Here is the code I tried:
textextract = shlex.split(r'"textextract.exe" "download.pdf" /to "download.txt"')
subprocess.run(textextract)
I already tried it with shell=True, but it didn't work.
Can anyone help me?
I was able to get the following script to work from the command line after installing the PDF2Text Pilot application you're trying to use:
import shlex
import subprocess
args = shlex.split(r'"textextract.exe" "download.pdf" /to "download.txt"')
print('args:', args)
subprocess.run(args)
Sample screen output of running it from a command line session:
> C:\Python3\python run-textextract.py
args: ['textextract.exe', 'download.pdf', '/to', 'download.txt']
Progress:
Text from "download.pdf" has been successfully extracted...
Text extraction has been completed!
The above output was generated using Python 3.7.0.
I don't know if your use of spyder on anaconda affects things or not since I'm not familiar with it/them. If you continue to have problems with this, then, if it's possible, I suggest you see if you can get things working directly—i.e. running the the Python interpreter on the script manually from the command line similar to what's shown above. If that works, but using spyder doesn't, then you'll at least know the cause of the problem.
There's no need to build a string of quoted strings and then parse that back out to a list of strings. Just create a list and pass that:
command=["textextract.exe", "download.pdf", "/to", "download.txt"]
subprocess.run(command)
All that shlex.split is doing is creating a list by removing all of the quotes you had to add when creating the string in the first place. That's an extra step that provides no value over just creating the list yourself.

Python Syntax Error on Data Cast in R Script Issue

I'm attempting to run an R script from python. I can see it's executing, but python is erroring out on a syntax error for the R script. The R script runs fine in RStudio, so I'm not sure what the issue might be. Any suggestions would be helpful.
This is my python code:
import sys
import subprocess
# other stuff going on
# execute R script which returns a document in a directory
subprocess.call([sys.executable, "C:/Users/path/sentiment_extraction.r"])
sentiment = pd.read_csv("new_sentiment_data.csv")
And here is my R script:
#! /usr/bin/Rscript
require(syuzhet)
today_news <- read.csv("C:/Users/path/Today_News.csv") # created from above python script
#Sentiments words table:
records <- as.character(today_news$News)
sentiment <- get_nrc_sentiment(records)
# other stuff that makes a file
Then this is the Python error I get:
File "C:/Users/path/sentiment_extraction.r", line 7
records <- as.character(today_news$News)
^
SyntaxError: invalid syntax
The data I use syuzhet on needs to be a character vector so I cast the column.
I figured it out. My parameters to run the R file were incorrect. I had to pass the path to Rscript.exe:
subprocess.call(["C:/Users/seeme/Documents/R/R-3.4.0/bin/Rscript",
"C:/Users/path/sentiment_extraction.r"])

Call a cmd.exe bat.-like command with python

i need some help with this...
I have a program installed on my computer that i want to call to calculate some things and give me an output-file...
in Matlab the command "dos()" does the job giving me also the cmd screen output in matlab.
I need this to work in python but i am making something wrong.
data='file.csv -v'
db=' -d D:\directory\bla\something.db'
anw='"D:\Program Files\bla\path\to\anw.exe"' + db + ' -i' + data
"anw" output is this one:
>>> anw
'"D:\\Program Files\\bla\\path\\to\\anw.exe" -d D:\\directory\\bla\\something.db -i file.csv -v'
## without the "" it does not work either
import subprocess as sb
p= sb.Popen('cmd','/K', anw) ## '/C' does not work either
i get the following error message from cmd.exe inside the python shell
Windows cannot find "\"D:\Program Files\bla\path\to\anw.exe"" Make sure you typed the name correctly, and then try again.
this line runs when i make a bat. file out of it.
it runs in matlab via "dos(anw)" so what is wrong here?
ps: i have blanks in my command... could this be the problem? i do not know where the first "\" comes from in the cmd. exe error message
for now i created a bat. file with all the stuff cmx.de should do in the specific directory where the input file lies...
i just had to tell python to change directory with
import os
os.chdir("D:\working\directory")
os.system(r'D:\working\directory\commands.bat')
it works good and gives me the output of cmd directly in the python shell

Categories