I'm trying to download an ipython notebook file as a .py file. It's working fairly well, except the .py file is interspersed with "In: []" at cell boundaries. I could live with them being there, but I'd rather they weren't.
Any simple fix?
Example output (what I see in the .py file):
# In[4]:
# Get names of all files
text_files = glob.glob('hw3data/*')
#print text_files
# In[5]:
def file_contents(file_name):
with open(file_name) as f:
return f.read()
Edit: Essentially, I'm wondering if it's possible to make notebook itself not output #In[ ]. Is there a commandline option, or a utility, or some kind of %magic?
Edit: Going by https://github.com/ipython/ipython/issues/5780, it looks like the suggested solution is just to use a custom template. From minrk:
It's just a marker that indicates where the cells were. It is by design, but it has no effect on the Python, since it is a comment. If you want to remove them, you can use a custom exporter template that doesn't add these comments.
try
def rc(in_, out_):
in_ = open(in_, 'r')
out_ = open(out_,'w')
out_.write(in_.readline())
for line in in_:
if not line.lstrip().startswith('# In['):
out_.write(line)
in_.close()
out_.close()
rc('~/in_file.py', '~/out_file.py')
As a temporary hack, I found that removing lines seven through nine of the python export template of nbconvert (ipython/IPython/nbconvert/templates/python.tpl in site-packages) stops the input prompt from being output. This still isn't ideal, as I'm modifying site packages.
Affected lines:
{% block in_prompt %}
# In[{{ cell.prompt_number if cell.prompt_number else ' ' }}]:
{% endblock in_prompt %}
Related
I am trying to run the below command in my jupyter notebook, it's working fine cuz no error showed up but also no output. But if am trying the same command in an IDE it's working fine. What can be the problem??
import csv
class Item:
#classmethod
def instantiate_from_csv(cls):
with open('item.csv', 'r') as f:
reader = csv.DictReader(f) #this method will read the content from csv file as a dictionary
items = list(reader)
for item in items:
print(item)
Item.instantiate_from_csv()
I tried in jupyter, I didn't get the expected output, a list of all the items mentioned in the csv in the form of dictionary so then I tried in IDLE it worked fine and I got the expected output.
When you execute
with open('item.csv') as f:
the CWD, current working directory, is pretty important.
Use os.getcwd() to verify it is what you expect.
If it isn't, consider switching from
a relative to an absolute pathname,
that looks like /some/where/item.csv.
Your previous efforts might have accidentally
created a zero-length data file. If you notice
such a .CSV file, just delete it, so that
future attempts to read it will produce a
more informative diagnostic error message.
so first and foremost i wanted to connect multiple lines into one and add ","
so in example
line1
line2
line2
to
line1,line2,line3
i managed to make it work with this script right here
filelink = input("Enter link here ")
fix = file = open(filelink, "r")
data=open(filelink).readlines()
for n,line in enumerate(data):
if line.startswith("line"):
data[n] = "\n"+line.rstrip()
else:
data[n]=line.rstrip()
print(','.join(data))
HOWEVER in the terminal itself it shows it executed perfectly but in the text file itself it's still remains the same no connected lines and no commas
side note. i would love some explanations how does the loop work and what "enumerate" stands for and why specifically this code i tried googling each one separately and understand the code but i didn't manage to find what i was looking for if anyone keen to explain the code line by line shortly i would be very appreciative
Thanks in advance <3
This is somewhat superfluous:
fix = file = open(filelink, "r")
That's assigning two names to the same file object, and you don't even use fix, so at least drop that part.
For handling files, you would be better using a context manager. That means that you can open a resource and they will automatically get closed for you once you're done (usually).
In any case, you opened in read mode with open(filelink, "r") so you'll never change the file contents. print(','.join(data)) will probably show you what you expect, but print() writes to stdout and the change will only be in your terminal. You will not modify the base file with this. But, I think you're sufficiently close that I'll try close the missing connection.
In this case, you need to:
open the file first in read mode to pull the data out.
Do the transform in python to the data
Open the file again in write mode (which wipes the existing contents)
Write the transformed data
So, like this:
filelink = input("Enter link here ")
with open(filelink) as infile: # context manager, by default in "r" mode
data = [item.strip() for item in infile.readlines()]
data = ','.join(data)
# Now write it back out
with open(filelink, "w") as outfile:
outfile.write(data)
I am using the mako template library to generate a test.txt file, but the generated file contains additional empty lines in between each row.
I have found a similar question about this issue here and the solution proposed suggests the use of markupsafe, but I am not convinced that this is also suitable in my case as it considers passing the text to be formatted as an argument when rendering the template, which is not what I want to do (I don't want to change most of the text in the template, just a few variables I am inputting).
I think it's also worth mentioning that if I print the rendered template in Python, it prints with the correct formatting; the extra lines only appear in the file I write the template data to (test.txt) using Python's file write().
Piggy-backing off of this answer, the solution is to open the file for write-binary instead of just write. Then you need to convert your string to bytes and write that to your file. The following worked for me (tl;dr last two lines):
templates = TemplateLookup(directories=[input_dir,], module_directory=mako_module_dir)
try:
rendered_output = templates.get_template(target_template).render_unicode(**data)
except:
print(mako_exceptions.text_error_template().render())
return
with open(f'{output_dir}{os.path.sep}{os.path.basename(output_filename)}', 'wb') as outfile:
outfile.write(rendered_output.encode())
Background
I'm writing a html book about doing things in python. It contains a ton of text and code interspersed with output. I want to be able to modify my python code at any time, and have one script to batch update all the HTML files with the changes.
Since I'll have tons of html files and tons of python snippets, I can't copy and paste from python into HTML manually every time I change something. that will be a nightmare.
Edit:
So I have two complimentary files: a python file that contains the logic, and an HTML file that is the tutorial for that python code. I'd like to be able to edit my python files at will, and tag them so that my HTML file can be updated too.
At the moment, when I update my python code, I just run a separate python script that looks in both files for matching tags, and copies the python code between its tags over to the HTML between its matching tags. I have many such tags throughout the files. I also have many such pairs of files.
the tags are ^tagname and end with ^/tagname
However, this only works for the code itself, not the code's output. I'd like to also be able to (if desired) copy the output from the tagged python over to to the html as well, and have it appear inside a slightly modified tag.
I was thinking for output &tagname and &/tagname.
End Edit
But getting the output from the python files to do the same thing is proving to be quite difficult.
My python code is: (testCode.py)
#%%
# ^test
x=1
print(x)
# ^/test
#%%
print("this output I don't care about")
#%%
# ^test2
y=2
print(y)
# ^/test2
So, the test and test2 tags are how I want to split the code up.
My html looks like this:
<p> Some text explaining the purpose of the python code in test1 </p>
<!--^test-->
<!--^/test-->
<p> some more text explaining about test2</p>
<!--^test2-->
<!--^/test2-->
The code between test1 tags in python file get copied between the comments above. And the same for test 2.
Here's a screenshot of what I have so far in my actual html document. It looks great, but it's missing the output.
The problem is I can't figure out to split up the output based on tags inside comments in the code. I need to use these tags to split the output into chucks associated with each tag.
Desired output
My desired output is a string such as the following:
# ^test
1
# ^/test
this is output I don't care about
# ^test2
2
# ^/test2
Attempt
I've successfully captured the output of the file into a string using:
python_file = 'testCode.py'
process_result = subprocess.run(['python', './' + python_file], capture_output=True, universal_newlines=True)
output = str(process_result.stdout)
but obviously that just prints:
1
this is output I don't care about
2
I am using subprocess because this function will eventually be called in a loop with a list of python files to update the output from.
I'm completely stumped about to get access to the appropriate tags and intersperse them in.
Perhaps I'm approaching this wrong.
Note: This solution works, but is messy.
Any suggestions for improvement would be appreciated!
Based on #flakes comment, I've come up with a working solution, but it's not ideal. I'm positive there's a better, more elegant way to do this.
I've decided to reduce the automation a bit to also reduce needed code and complexity.
For each article I now have a folder containing
ArticleName.py
ArticleName.html
ArticleName_helper.py
OutputFiles [Folder]
Inside ArticleName.py, I have tagged code. If I want to save the output of the code inside the tag, I also create an OutputTag object (described below).
#%%
out = OutputTag('test', __file__)
# ^test
x=1
print(x)
print("test")
# ^/test
out.done()
OuputTag object and an object to replace sdtout (Stored in a different folder)
import os
import sys
#%%
class CustomStdout(object):
def __init__(self, *files):
self.files = files
def write(self, obj):
for f in self.files:
f.write(obj)
f.flush() # If you want the output to be visible immediately
def flush(self) :
for f in self.files:
f.flush()
#%%
class OutputTag:
def __init__(self, tag, origin_file):
origin_dir = os.path.dirname(origin_file)
if not os.path.exists(origin_dir) :
raise FileNotFoundError("Origin file does not exist!")
dir = origin_dir + "/OutputFiles"
if not os.path.exists(dir):
os.makedirs(dir)
file_name = os.path.join(dir, tag + '.txt')
self.original = sys.stdout
self.f = open(file_name, 'w')
# This will go to stdout and the file out.txt
sys.stdout = CustomStdout(sys.stdout, self.f)
def done(self):
self.f.close();
sys.stdout = self.original
Ok so that populates the OutputFiles folder with a bunch of .txt files named after the tags.
Then I run the ArticleName_helper.py script.
It has a reference to the py and html files.
The helper script looks for code-tags inside ArticleName.html.
It searches inside ArticleName.py for matching tags, and copies any code between them.
This copied code then replaces any existing text between the tags in the HTML.
Similarly, it searches for output tags (&Tag instead of ^Tag) in html.
It looks for files in OutputFolder that match the tag name.
Then loads the text inside, replacing any text in the html file between the output tags.
In my IDE I have auto code completion. In python, when I type html and press tab it auto generates the following with a multi-cursor ready to overwrite TAG in all 3 spots:
out = OutputTag('TAG', __file__)
# ^TAG
# ^/TAG
out.done()
similarly in html editing, I have it autocomplete the following when I type py and hit tab:
<pre><code class="language-python">
<!--^TAG-->
<!--^/TAG-->
</code></pre>
And a similar one for output with the ^ replaced by &
So the current workflow is:
Write python file
Add tags if wanted
Write HTML tutorial, adding comment tags where needed for both input and associated output.
Run Python file to update output
Run Helper Script
HTML file auto populates with code and output where tags were added.
After making edits, just do steps 4-6.
The result!
I am using the application Zim Wiki(cross-platform, FOSS), which I am using to keep a personal wiki with lots of data coming from tables, copy and pasting, my own writing, and downloading and attaching .png and .html files for viewing and offline access.
The data that is not written or pasted can be stored in tables in the form of names, url addresses, and the names and locations of images and other attachments.
To insert into zim, I can use the front end with WSIWYG, or to make the skeleton of each entry, I could modify a template text entry. If I do this, nothing matters except for the location and identity of each character in each line.
By supplying the text in this image:
DandelionDemo source text,
--I can make this entry for Dandelion:
DandelionDemo Wiki.
So, I can generate and name the Wiki entry in Zim, which creates the .txt file for me, and inserts the time stamp and title, so, the template for this type of entry without the pasted fields would be:
**Full Scientific Name: **[[|]]**[syn]**
**Common Name(s): **
===== =====
**USDA PLANTS entry for Code:** [[https://plants.usda.gov/core/profile?symbol=|]] **- CalPhotos available images for:** [[https://calphotos.berkeley.edu/cgi/img_query?query_src=photos_index&where-taxon=|]]
**---**
**From - Wikipedia **[[wp?]] **- **[[/Docs/Plants/]]
{{/Docs/Plants/?height=364}}{{/Docs/Plants/?height=364}}
**()** //,// [[|(source)]]
**()** //// [[|(source)]]
**Wikipedia Intro: **////
---
So the first line with content, after the 31st character(which is a tab), you paste "http... {etc}. Then the procedure would insert "Taraxacum officinale... {etc}" after the "|", or what was the 32nd character, and so on. This data could be from "table1" and "table2", or combining the tables to make an un-normalized "table1table2", where each row could be converted to text or a .csv or I don't know, what do you think?
Is there a way, in LibreOffice to do this? I have used LibreOffice Base to generate a "book" form that populated fields, but it was much less complex data, without wiki liking and drag-and-drop pasting of images and attachments. So maybe the answer is to go simpler? The tables are not currently part of a registered database, but I could do that, once I have decided on the method of doing this.
I am ultimately looking for a "way", hopefully an "easy" way. However, that may not be in LibreOffice. If not, I know that I could do this in Python, but I haven't learned much about Python yet. If it involves a language, that is the first and only one I don't know that I will invest in learning for this project. If you know a "way" to do this in Python, let me know, and my first project and way of framing my study process will be in learning the methods that you share.
If you know of some other Linux GUI, I am definitely interested, but only in active free and open source builds that involve minimal/no compiling. I know the basics of SQL and DBMS's. In the past, have gotten Microsoft SQL server lite to work, but not DBeaver, yet. If you know of a CLI way also let me know, but I am a self-taught outdoors-loving Linux newb and mostly know about how to tweak little settings in programs, how to use moderately easy programs like ImageMagick, and I have built a few Lamp stacks for Drupal and Wordpress (no BASH etc...).
Thank you very much!
Ok, since you want to learn some python, let me propose you a way to do it this. First you need a template engine -like jinja2 (there are many others)-, a data source in our example a .csv file, -could be other like a db- and finally some code that reads the csv line by line and mix the content with the template.
Sample CSV file:
1;sample
2;dandelion
3;just for fun
Sample template:
**Full Scientific Name: **[[|]]**[syn]**
**Common Name(s): *{{name}}*
===== =====
USDA PLANTS entry for Code: *{{symbol}}*
---
Sample code:
#!/usr/bin/env/python
#
# Using the file system load
#
# We now assume we have a file in the same dir as this one called
# test_template.ziim
#
from jinja2 import Environment, FileSystemLoader
import os
import csv
# Capture our current directory
THIS_DIR = os.path.dirname(os.path.abspath(__file__))
def print_zim_doc():
# Create the jinja2 environment.
# Notice the use of trim_blocks, which greatly helps control whitespace.
j2_env = Environment(loader=FileSystemLoader(THIS_DIR),
trim_blocks=True)
template = j2_env.get_template('test_template.zim')
with open('example.csv') as File:
reader = csv.reader(File, delimiter=';')
for row in reader:
result = template.render(
symbol=row[0]
, name=row[1]
)
# to save the results
with open(row[0]+".txt", "wt") as fh:
fh.write(result)
fh.close()
if __name__ == '__main__':
print_zim_doc()
The code is pretty simple, reads the template located in the same folder as the python code, opens the csv file (also located in the same place), iterates over each line of the csv and renders the template using the values of the csv columns to fill the {{var_name}} in the template, finally saves the rendered result in a new file named as one of the csv column values. This sample will generate 3 files (1.txt, 2.txt, 3.txt). From here you can extend and improve the code to get your desired results.