This question already has answers here:
Python, want logging with log rotation and compression
(11 answers)
Closed 4 years ago.
I am currently using logging package in Python to log statements to a single file (log.log)
The problem I am facing is if the log file gets too large, it takes a lot of time to open the log file.
I thought after a pre-defined file size or line numbers, I can create a log file called log.log.1, and continue running with a fresh log.log.
Also, after log.log.5, the old logs should get deleted automatically.
How can I achieve this?
Depending on how you want to rotate the logs, you can use either the TimedRotatingFileHandler to rotate files at a given time interval (for example, create a new file ever day, or every hour); or you can use the RotatingFileHandler to create new files after they reach a certain size.
The logging cookbook has examples to help you get started.
Related
This question already has answers here:
Python trace module - Trace lines as they are executed, but save to file, rather than stdout
(5 answers)
Closed 1 year ago.
Here is the situation: I have a python program that crashes when i do something that shouldn't crash. I'd like to debug it myself so i am not asking for help on why it crashes but how i debug it like i want to.
I want to compare what lines execute in what order when it works and when i do the thing that doesn't work.
I found the trace module which you can use like this:
python -m trace -t program.py
And it will "Display lines as they are executed" (print them). However, the program i am making is an open world game in curses and therefore the output gets really weird since the terminal is already used by my curses game and i basically cannot use it.
So i have to write the output to a file somehow if i want to use it.
How do i make it write to a file, or are there any other methods?
I suggest using the python logging library. This will give you more control since you can add logging wherever you want including values of variables. You can also direct the output to stderr (along with command line redirection) or to a file in order to separate it from the curses ui of your actual program.
This question already has answers here:
How do I watch a file for changes?
(28 answers)
Closed 1 year ago.
I'm writing a program in which I need the program to verify if there is new content in a file. If the file HAS content, but the content has already been viewed previously by the user, it doesn't count as a notification, but if it is new, it is shown to the user.
How do I do this?
Did you try using Watchdog?
Python API library and shell utilities to monitor file system events.
Directory monitoring made easy with:
A cross-platform API.
A shell tool to run commands in response to directory changes.
Get started quickly with a simple example in Quickstart...
This question already has answers here:
What is the best way to call a script from another script? [closed]
(16 answers)
Closed 3 years ago.
I am a beginner to Python, and I would like a way to run a Python file from Python.
For example, I have a Python file named checker.py. In that file, I would like to iterate over a folder that contains inputs and outputs, and I would like to, using the Python script, give input to a different Python file, and check if it matches the expected output (in a different file). Is there any way to do this in Python?
Here is the GitHub link for the problems I have completed and need to check so far: https://github.com/vishnupsatish/CCC-practice
Try to split your big problem into several smaller ones.
First, try to find all files.
Then execute the Python files ( https://docs.python.org/3/library/subprocess.html )
Then read the output files and compare the results.
As this is a practice for you, I won't deliver all the code. Just try to do one small task after the other. And then ask when you encounter a problem.
This question already has answers here:
How do I watch a file for changes?
(28 answers)
Closed 5 years ago.
I am relatively new to python, but I am trying to create an automated process where my code will listen for new file entries in a directory. For example, someone can manually copy a zip file into a particular folder, and I want my code to recognize the file once it has completely been copied into the folder. The code can then do some manipulations, but that is irrelevant. I currently have my code just checking for a new file every 5 seconds, but this seems inefficient to me. Can someone suggest something that is more asynchronous?
Checking for new file every several seconds is actually not that bad an approach — in fact, it is the only really portable way to monitor for filesystem changes. If you're running on Linux, answers in the question linked by #sdolan will help you check for new files more efficiently, but they won't help you with the other part of your question.
Detecting that the file has been copied completely is much more difficult than it first appears. Your best best is, when a new file is detected, wait until it hasn't been touched for a while before processing it. The length of the wait period is best determined experimentally. It's a balancing act: make the interval too short, and you're risking operating on incomplete files; make it too long, and the user will notice a delay between the copy operation finishing and your code processing it.
This question already has answers here:
Python FTP get the most recent file by date
(5 answers)
Closed 4 years ago.
I am using Python to connect to an FTP server that contains a new list of data once every hour. I am only connecting once a day, and I only want to download the newest file in the directory. Is there a way to do this?
Seems like any system that is automatically generating a file once an hour is likely to be using an automated naming scheme. Are you over thinking the problem by asking the server for the newest file instead of more easily parsing the file names?
This wouldn't work in all cases, and if the directory got large it might become time consuming to get the file listing. But it seems likely to work in most cases.
Look at ftplib in your current version of python. You can see a function to handle the result of the LIST command that you would issue to do a dir, if you know a last time that you run a successful script then you can parse the result from the LIST and act on the new files on the directory. See the ftplib for more info on how to do it. The retrlines function is what I would expect to use.