createDataFrame not working in Spark 2.0.0 - python

I am trying to work through some of the examples in the new Spark 2.0 documentation. I am working in Jupyter Notebooks and command line. I can create a SparkSession with no problem. However when I try to create a dataframe I get the error of:
AttributeError: 'function' object has no attribute 'createDataFrame'
spark = SparkSession.builder.master("local").appName("Search").config(conf=SparkConf()).getOrCreate
d = [{'name': 'Alice', 'age': 1}]
spark.createDataFrame(d).collect()
Can someone please explain what I need to do to fix this error ? I have searched through the official documentation and not found anything on this particular error. Thank you.


getOrCreate is a method on SparkSession.Builder. You need to invoke it by adding the parentheses after:
spark = SparkSession.builder.master("local").appName("Search").config(conf=SparkConf()).getOrCreate()
See for more information: https://spark.apache.org/docs/2.0.1/api/java/org/apache/spark/sql/SparkSession.html
In general, the 'function' object has no attribute error is very common when you are accidentally referencing a function rather than invoking it.

Related

I am running the code below in python and getting the error 'AttributeError: 'QgridWidget' object has no attribute 'to_excel''

I am trying to use to excel function using the code below. I am getting the error df.to_excel('dataset1.xlsx')
Below is the code.
df=qgrid.show_grid(data)
df.to_excel('dataset1.xlsx')

qgrid.show_grid() returns a QgridWidget instance. You can access the underlying dataframe using the df attribute:
grid = qgrid.show_grid(data)
grid.df.to_excel('dataset1.xlsx')

How to implement refining conflict in constraint programming

I use Docplex with python 3.7 to implement constraints programming. when it was infeasible, how can i proceed to list constraints those was to source of the conflict?
mdl.export_as_cpo(out="/home/..../MCP3.lp")
msol = mdl.solve(FailLimit=700000, TimeLimit=1600)
DInfos= msol.get_solver_infos()
mconflict=msol.CpoRefineConflictResult()
mconflict.get_all_member_constraints()
Error message:
mconflict=msol.CpoRefineConflictResult()
AttributeError: 'CpoSolveResult' object has no attribute 'CpoRefineConflictResult'

solve returns a SolveResult, and CpoRefineConflictResult is a class in docplex.cp.solution. So, the error message is correct: a SolveResult does not have an attribute CpoRefineConflictResult. You'd expect the CpoRefineConflictResult as the result of the conflict refiner.
You should probably read through the documentation a bit more http://ibmdecisionoptimization.github.io/docplex-doc/cp/docplex.cp.solution.py.html
You can call the .refine_conflict() method on the CpoSolver object to obtain a CpoRefineConflictResult, as documented here http://ibmdecisionoptimization.github.io/docplex-doc/cp/docplex.cp.solver.solver.py.html#detailed-description
Perhaps you can provide a minimal, reproducible example, if you need a more specific solution to your problem. https://stackoverflow.com/help/minimal-reproducible-example

I have add:
from docplex.cp.solver.solver import CpoSolver
After, i have add those lines if the model is infeasible:
mconfl= CpoSolver(model)
mconf = mconfl.refine_conflict()

mininet-wifi error:'Mininet' object has no attribute 'addBaseStation'

When I execute my Python script using mininet-wifi, I am getting the following error, and I don't know why?
error:'Mininet' object has no attribute 'addBaseStation'
Should I change addBaseStation into addAcessPoint? If so, what is the difference between these?

Try out addAccessPoint() instead.

AttributeError: 'ParentedTree' object has no attribute 'label'

I am basically working on parsed tree and trying to annotate tree nodes dominating empty categories(Empty node annotation).
I have defined a recurvsive function as below but the error that I am getting is "AttributeError: 'ParentedTree' object has no attribute 'label'".
def annotateTraceNodes(node):
numChildren = len(node);
numNone=0;
for child in node:
if isinstance(child,Tree):
annotateTraceNodes(child);
if(numChildren==0 or child.label().endswith("-NONE-")):
numNone+=1;
if(numChildren==numNone):
print "setting the label";
node.set_label(node.label()+"-NONE-");

Google suggests you're using NLTK, which I'm going to assume is the case.
A ParentedTree does not have a method called .label(). So when you write things like this:
child.label().endswith("-NONE-")
Python doesn't know what to do.
A Tree, on the other hand, does have a .label() method. Did you perhaps use a ParentedTree instead of a Tree somewhere?

Could also be a versioning issue.
Worked for me with version 3.3
pip install nltk==3.3

geom_dotplot in rpy2?

When I try to use gplot2.geom_dotplot I get the error:
AttributeError: 'module' object has no attribute 'geom_dotplot'
Does this function have a different name in Rpy2? thanks.

The mapping of that function is just missing. This is a bug with rpy2. The fix will in the repository shortly (will be released with version 2.3.4).
In the meantime a workaround can be to add the following to your code.:
from rpy2.robjects.lib import ggplot2
class GeomDotplot(ggplot2.Geom):
_constructor = ggplot2.ggplot2_env['geom_dotplot']
ggplot2.geom_dotplot = GeomDotplot.new

Categories