This question already has answers here:
Python FTP get the most recent file by date
(5 answers)
Closed 4 years ago.
I am using Python to connect to an FTP server that contains a new list of data once every hour. I am only connecting once a day, and I only want to download the newest file in the directory. Is there a way to do this?
Seems like any system that is automatically generating a file once an hour is likely to be using an automated naming scheme. Are you over thinking the problem by asking the server for the newest file instead of more easily parsing the file names?
This wouldn't work in all cases, and if the directory got large it might become time consuming to get the file listing. But it seems likely to work in most cases.
Look at ftplib in your current version of python. You can see a function to handle the result of the LIST command that you would issue to do a dir, if you know a last time that you run a successful script then you can parse the result from the LIST and act on the new files on the directory. See the ftplib for more info on how to do it. The retrlines function is what I would expect to use.
Related
This question already has answers here:
How do I watch a file for changes?
(28 answers)
Closed 1 year ago.
I'm writing a program in which I need the program to verify if there is new content in a file. If the file HAS content, but the content has already been viewed previously by the user, it doesn't count as a notification, but if it is new, it is shown to the user.
How do I do this?
Did you try using Watchdog?
Python API library and shell utilities to monitor file system events.
Directory monitoring made easy with:
A cross-platform API.
A shell tool to run commands in response to directory changes.
Get started quickly with a simple example in Quickstart...
This question already has answers here:
What is the best way to call a script from another script? [closed]
(16 answers)
Closed 3 years ago.
I am a beginner to Python, and I would like a way to run a Python file from Python.
For example, I have a Python file named checker.py. In that file, I would like to iterate over a folder that contains inputs and outputs, and I would like to, using the Python script, give input to a different Python file, and check if it matches the expected output (in a different file). Is there any way to do this in Python?
Here is the GitHub link for the problems I have completed and need to check so far: https://github.com/vishnupsatish/CCC-practice
Try to split your big problem into several smaller ones.
First, try to find all files.
Then execute the Python files ( https://docs.python.org/3/library/subprocess.html )
Then read the output files and compare the results.
As this is a practice for you, I won't deliver all the code. Just try to do one small task after the other. And then ask when you encounter a problem.
This question already has answers here:
Download large file in python with requests
(8 answers)
Closed 3 years ago.
I am looking for a python download method where I can download the file directly to disk using a specified destination and file name, and if the download takes too long, time out.
From here it looks like there's three main python download options
https://stackabuse.com/download-files-with-python/
urllib.request.urlretrieve
does not have a timeout option.
requests.get(url)
Has timeout, but requires the file be opened before saving it. It also looks like it gets the file name from the url.
wget.download
also doesn't seem to have a timeout option.
Are there any python download methods that satisfy all three of my requirements?
I don't think you can write to a file without opening it.
urllib opens it too.
This question already has answers here:
Python, want logging with log rotation and compression
(11 answers)
Closed 4 years ago.
I am currently using logging package in Python to log statements to a single file (log.log)
The problem I am facing is if the log file gets too large, it takes a lot of time to open the log file.
I thought after a pre-defined file size or line numbers, I can create a log file called log.log.1, and continue running with a fresh log.log.
Also, after log.log.5, the old logs should get deleted automatically.
How can I achieve this?
Depending on how you want to rotate the logs, you can use either the TimedRotatingFileHandler to rotate files at a given time interval (for example, create a new file ever day, or every hour); or you can use the RotatingFileHandler to create new files after they reach a certain size.
The logging cookbook has examples to help you get started.
This question already has answers here:
How do I watch a file for changes?
(28 answers)
Closed 5 years ago.
I am relatively new to python, but I am trying to create an automated process where my code will listen for new file entries in a directory. For example, someone can manually copy a zip file into a particular folder, and I want my code to recognize the file once it has completely been copied into the folder. The code can then do some manipulations, but that is irrelevant. I currently have my code just checking for a new file every 5 seconds, but this seems inefficient to me. Can someone suggest something that is more asynchronous?
Checking for new file every several seconds is actually not that bad an approach — in fact, it is the only really portable way to monitor for filesystem changes. If you're running on Linux, answers in the question linked by #sdolan will help you check for new files more efficiently, but they won't help you with the other part of your question.
Detecting that the file has been copied completely is much more difficult than it first appears. Your best best is, when a new file is detected, wait until it hasn't been touched for a while before processing it. The length of the wait period is best determined experimentally. It's a balancing act: make the interval too short, and you're risking operating on incomplete files; make it too long, and the user will notice a delay between the copy operation finishing and your code processing it.