I want to create a python file that uses a code stored in database
I have a table called CodeTable that has These data
ID Code
-----------
1 import pymssql import pandas as pd import matplotlib.pyplot as plt import seaborn as sns import numpy as np df = pd.read_csv(r'C:\Projects\G.csv') plt.figure(figsize=(12, 9))
2 X = 1 + MasterKey
and in my code I have this
MasterKey = 7
#Some code to call Record with ID = 2 from DB
# a function to execute Python dynamically <-------- I need this?!!
print(MasterKey) #<------------ Should return 8
Thanks
You can use the exec builtin function. For example, exec("print('Hello World!')")
Exec Documentation:
This function supports dynamic execution of Python code. object must be either a string or a code object. If it is a string, the string is parsed as a suite of Python statements which is then executed (unless a syntax error occurs).
I think this is pandas dataframe so we can using eval
pd.eval(df.loc[df.ID==2,'Code'].str.split('=').str[-1])[0]
8
Related
I am working Jupyter notebook on AWS Sagemaker instance. For convenience I wrote a .py file with couple of functions as defined;
#function to gather the percent of acts in each label feature combo
def compute_pct_accts(data, label_cnt):
"""
data is the output from aggregate_count
labe_cnt gives the breakdown of data for each target value
"""
label_data_combined = pd.merge(data, label_cnt, how='inner', left_on= 'label', right_on = 'label')
label_data_combined['Act_percent'] = np.round((label_data_combined['ACT_CNT']/label_data_combined['Total_Cnt'])*100,2)
return label_data_combined
#write a function to perform aggregation for target and feature column
def aggregate_count(df, var, target):
"""
df is the dataframe,
var is the feature name
target is the label varaible(0 or 1)
"""
label_var_cnt = df.groupby([var,target],observed=True)['ID'].count()
label_var_cnt = label_var_cnt.reset_index()
label_var_cnt.rename(columns={'ID':'ACT_CNT'},inplace=True)
return label_var_cnt
Both these functions are stored in a .py file called file1.py. Then to retrieve them in my notebook I typed;
from file1 import *
import pandas as pd
This command did import both functions. But when I tried to run the function;
compute_pct_accts(GIACT_Match_label_cnt, label_cnt)
I am getting a Name error;
pd not found
Please note that I have imported pandas as pd in my jupyter notebook. I am aware of using the option
%run -i compute_pct_accts_new.py
but that forces me to write a new python file with that function. My question is, can we have one python file with all functions defined in it, so that we can import all of them at once and use interactively in notebook.
Help is appreciated.
Try importing pandas in the .py file containing the function you want to import.
I am working on a small library and I need to know can I import modules like numpy, sklearn and etc. Using functions. For example:
def ml():
import numpy as np
import pandas as pd
x = np.array([1,2,647,345,3,7,3,8,36,64])
Is this possible ?
Simply can I import a module using a function and then use that later outside the function
The main idea is when the user calls the function ml he has all the modules related to machine learning imported and then he can use them. X = np.array was just kind of an example.
UPDATED
This should work
import importlib
def importmd(modulex):
return importlib.import_module(modulex) #Returning the module
np = importmd("numpy") #Same as import numpy as np
I have a python file which has many 'def' values in it. When I try to integrate the python file with android studio, I get a type error. The input is a image file and I want the lowerRange and upperRange to be based on that image, so I cannot define a value to them, since image size can vary everytime.
import numpy as np
import cv2
import os
import matplotlib.pyplot as plt
from PIL import Image
def host(croppedImage,lowerRange, upperRange):
mask_yellow = cv2.inRange(croppedImage,lowerRange,upperRange)
dilatation_type = cv2.MORPH_RECT
dilatation_size = 1
element = cv2.getStructuringElement(dilatation_type, (dilatation_size + 2, dilatation_size+2), (dilatation_size, dilatation_size))
dilated_mask_image = cv2.dilate(mask_yellow, element)
return dilated_mask_image
def DrawContourRect(contour):
rect = cv2.minAreaRect(contour)
return cv2.boxPoints(rect)
-----------------------------This is just a part of code---------------------------------
And this is the xml code for python object:
PyObject pyo = py.getModule("file");
PyObject obj = pyo.callAttr("host", imageString);
And the error is this:
com.chaquo.python.PyException: TypeError: detect() missing 2 required positional arguments: 'lowerRange' and 'upperRange'
at <python>.chaquopy_java.call(chaquopy_java.pyx:285)
at <python>.chaquopy_java.Java_com_chaquo_python_PyObject_callAttrThrows(chaquopy_java.pyx:257)
Is there any way to solve this problem and how can the chaquopy read each and every 'def' value (host and DrawContourRect.
The error message and code don't seem to match, but I assume that host and detect are either the same function, or that they have the same signature.
If that's correct, then the problem is simply that you're passing 1 argument to a function that requires 3. If you want the lowerRange and upperRange to be based on the image, then you'll have to either:
Calculate them on the Java side and pass them to Python using the 2 extra arguments; OR
Remove the 2 extra arguments from the function, and calculate the range on the Python side.
I'm trying to use the multiprocessing package to compute a function on a very large Pandas dataframe. However I ran into a problem with the following error:
OverflowError: cannot serialize a bytes objects larger than 4GiB
After applying the solution to this question and using protocol 4 for pickling, I ran into the following error instead, which is also quoted by the solution itself:
error: 'i' format requires -2147483648 <= number <= 2147483647
The answer to this question then suggests to use the dataframe as a global variable.
But ideally I would like the dataframe to still be an input of the function, without having the multiprocessing library copying and pickling it multiple times in the background.
Is there some other way I can design the code to not run into the issue?
I was able to replicate the problem with this example:
import multiprocessing as mp
import pandas as pd
import numpy as np
import time
import functools
print('Total memory usage for the dataframe: {} GB'.format(df.memory_usage().sum() / 1e9))
def slow_function(some_parameter, df):
time.sleep(1)
return some_parameter
parameters = list(range(100))
with mp.Pool(20) as pool:
function = functools.partial(slow_function, df=df)
results = pool.map(function, parameters)
try Dask
import dask.dataframe as dd
df = dd.read_csv('data.csv')
docs : https://docs.dask.org/en/latest/dataframe-api.html
I have a quick question regarding my python code. I am using a tool called OVITO, which works quite nicely when I run in the GUI. I am trying to run it using the python scripting interface, but I am not wanting to get the results that I want.
The code should create a list of files with a single number. However, the number for each file should change. I am calculating a specific property for my material (vacancies), and I know vacancies are created, but my code says 0 for all my files. Can someone tell me if I am overwriting somehow or if something is obviously wrong? Again, I do not expect anyone here to know OVITO, but just with the python part, curious if something is clearly wrong.
Import OVITO modules.
from ovito.io import *
from ovito.modifiers import *
# Import NumPy module.
import numpy
import sys
node = import_file("../cascade.dump",multiple_frames = True)
for i in range(node.source.num_frames):
with open("{}.out".format(i),'w') as f:
mod = WignerSeitzAnalysisModifier(per_type_occupancies = True)
mod.reference.load("../../../../../../STP/position_perfect_300.dump")
node.modifiers.append(mod)
node.compute()
node.modifiers.append(SelectExpressionModifier(expression = 'Occupancy.1==0&&Occupancy.2==0 && ParticleType==1'))
node.compute()
f.write("%i\n" % numpy.count_nonzero(node.output.particle_properties['Selection']))
f.close()
It's a very old post, but to whom it can help the solution is just to give the frame to compute in node.compute().
Import OVITO modules.
from ovito.io import *
from ovito.modifiers import *
# Import NumPy module.
import numpy
import sys
node = import_file("../cascade.dump",multiple_frames = True)
for i in range(node.source.num_frames):
with open("{}.out".format(i),'w') as f:
mod = WignerSeitzAnalysisModifier(per_type_occupancies = True)
mod.reference.load("../../../../../../STP/position_perfect_300.dump")
node.modifiers.append(mod)
node.compute(i)
node.modifiers.append(SelectExpressionModifier(expression = 'Occupancy.1==0&&Occupancy.2==0 && ParticleType==1'))
node.compute(i)
f.write("%i\n" % numpy.count_nonzero(node.output.particle_properties['Selection']))
f.close()