I am using apache.
If we keep it in var/www/ folder permission issues is raised.
I am thinking to keep log files in /tmp/ folder.Is it right place to keep log files?
No, /tmp would not be the right place to save log files.
According to the Filesystem Hierarchy Standard (FHS), the /tmp directory serves a different purpose:
3.17.1. Purpose
The /tmp directory must be made available for programs that require temporary files.
Programs must not assume that any files or directories in /tmp are preserved between invocations of the program.
The intent of writing log files is the ability to debug errors and keep track of program activity. Therefore, non-persistent logs would be of very little use.
For Logging there is the /var/log directory, as being recommended by the FHS:
5.10.1. Purpose
This directory contains miscellaneous log files. Most logs must be written to this directory or an appropriate subdirectory.
For the rights question I can only refer to WiseTechi, a /var/log/mydaemon directory is your way to go.
Related
I have a small Flask application that runs fine locally, however when I go to run the application on my server, it runs but I am not able to get the output to save to a public_html folder.
This is the area I believe I am having the issue, when I run the application remotely:
df.to_html('/home/mydomain/public_html/data/candles.html', index = False)
If I run the application locally, this location works fine:
df.to_html('candles.html', index = False)
I have ensured that the remote folder 'data' has full access - 0777.
What am I doing wrong?
If you don't have an exception occurring, then very likely the file was saved, but not where you think it should have. If you did not provide a full path, the destination will be relative to the application directory. The solution is to be explicit and provide a full path, unless you are using some Flask functions that already have a default setting.
You should never grant 0777 permissions on public_html, that is a potential vulnerability. For example, someone could upload a shell to that directory if they can leverage a security flaw on your website.
There is not enough context, but the user running the process (Apache, Nginx or whatever) should not have write permissions here. If you must grant write permissions, create a dedicated directory (preferably outside the webroot unless they have to be exposed to the user), then add some directives to stipulate that files present in the directory cannot be executed. So that even if a webshell is uploaded it cannot run.
I have previously written a script using python that monitors a windows directory and uploads any new files to a remote server offsite. The intent is to run it at all times and allow users to dump their files there to sync with the cloud directory.
When a file added is large enough that it is not transferred to the local drive all at once, Watchdog "sees" it as it is partially uploaded and tries to upload the partial file, which fails. How can I ensure that these files are "complete" before they are uploaded? Again, I am on Windows, and cannot use anything but Windows to complete this task, or I would have used inotify. Is it even possible to check the "state" of a file in this way on Windows?
It looks like there is no easy way to do this. I think you can put in place something that checks the stats on the directory when it triggers and only actions after a given amount of time that the folder size hasn't changed:
https://github.com/gorakhargosh/watchdog/issues/184
As a side note, I would check out Apache Nifi. I have used it with a lot of success and it was pretty easy to get up and running
https://nifi.apache.org/
I'm building an app with Pyside. Part of what the app does is download some xml files from a website and store them for later use. For example, every time the program is started it checks for these directories and, if they cannot be found, downloads the xml files, creates new directories, and stores the files in them. I'm wondering if there is some compelling reason not to allow the program to create new directories (permissions, security, etc.) and, if so, what other options exist?
EDIT: To be clear, I'll need to read and write to these directories and possibly create new ones as new files appear on the webpage.
Almost all programs need to write to config and/or cache directories. The only thing you need to worry about is making sure you use the appropriate platform-specific locations for doing this. Do not use an arbitrary, application-specific directory.
In Qt4, the QDesktopServices class can be used to determine the correct storage locations, or in Qt5, there's the QStandardPaths class.
Yes, it is normal for programs to store files in config / cache / log folders.
Use the python package appdirs to find the correct folders to use for your program.
I need to listen linux's port to run my service. So, I always run the python program to start with sudo previlage, that made the files created by program like pycahee and .pyc files also get super user's authority,the files and directories can only be removed in sudo mode. That's very inconvenience. So, is there a way to specify python to create normal folder and files?
Running a script as root just so you can listen on privileged ports is not good practise (unless the script really does require root).
If your script doesn't require root, then I would recommend using setuid/setgid to drop privileges after you have set up the privileged port socket;
This has already been answered in detail here;
Dropping Root Permissions In Python
Edit: OP mentioned that the pyc file created still has root permissions. You could use the suid bit (chmod u+s script.py) then setuid(0) to gain root permissions during runtime, ensuring the file ownership is not root. Setting the suid bit for only the file owner also means other users cannot abuse the suid bit.
I was wondering whether there was a best practice for checking if an upload to your ftp server was successful.
The system I'm working with has an upload directory which contains subdirectories for every user where the files are uploaded.
Files in these directories are only temporary, they're disposed of once handled.
The system loops through each of these subdirectories and new files in them and for each file checks whether it's been modified for 10 seconds. If it hasn't been modified for 10 seconds the system assumed the file was uploaded successfully.
I don't like the way the system currently handles these situations, because it will try and handle the file and fail if the file upload was incomplete, instead of waiting and allowing the user to resume the upload until it's complete.
It might be fine for small files which doesn't take a lot of time to upload, but if the file is big I'd like to be able to resume the upload.
I also don't like the loops of directories and files, the system idles at a high cpu usage, so I've implemented pyinotify to trigger an action when a file is written. I haven't really looked at the source code, I can only assume it is more optimized than the current implementation (which does more than I've described).
However I still need to check whether the file was successfully uploaded.
I know I can parse the xferlog to get all complete uploads. Like:
awk '($12 ~ /^i$/ && $NF ~ /^c$/){print $9}' /var/log/proftpd/xferlog
This would make pyinotify unnecessary since I can get the path for complete and incomplete uploads if I only tail the log.
So my solution would be to check the xferlog in my run-loop and only handle complete files.
Unless there's a best practice or simply a better way to do this?
What would the disadvantages be with this method?
I run my app on a debian server and proftpd is installed on the same server. Also, I have no control over clients sending the file.
Looking at the proftpd docs, I see http://www.proftpd.org/docs/directives/linked/config_ref_HiddenStores.html
The HiddenStores directive enables two-step file uploads: files are
uploaded as ".in.filename." and once the upload is complete, renamed
to just "filename". This provides a degree of atomicity and helps
prevent 1) incomplete uploads and 2) files being used while they're
still in the progress of being uploaded.
This should be the "better way" to solve the problem when you have control of proftpd as it handles all the work for you - you can assume that any file which doesn't start .in. is a completed upload. You can also safely delete any orphan .in.* files after some arbitrary period of inactivity in a tidy-up script somewhere.
You can use pure-uploadscript if your pure-ftpd installation was compiled with
--with-uploadscript option.
It is used to launch a specified script after every upload is completely finished.
Set CallUploadScript to "yes"
Make a script with a command like touch /tmp/script.sh
Write the code in it. In my example the script renames the file and adds ".completed" before the file name:
#!/bin/bash
fullpath=$1
filename=$(basename "$1")
dirname=${fullpath%/*}
mv "$fullpath" "$dirname/completed.$filename"
Run chmod 755 /tmp/script.shto make the script executable by pure-uploadscript
Then run a command pure-uploadscript -B -r /etc/pure-ftpd/uploadscript.sh
Now /tmp/script.sh will be launched after each completed upload.