Working with large in memory objects and was wondering if there's a way to check how much memory a python CGI process is allocated from within a script?
If you're on a UNIX machine, you can try the information from the documentation on resource. Not sure exactly what you want but resource.RLIMIT_STACK or resource.RLIMIT_DATA (heap) could be what you're looking for.
http://docs.python.org/2/library/resource.html
It is very unlikely there is a standard way to do this. If the value is not in the environment, you can not find it programmatically. How is the script run (server, module...)?
Related
Is it possible to stop from a python code all network traffic made by my computer?
If yes, how?
Thank you very much!
If you need to do this for testing, I recommend using pytest-socket
If you're looking to do this for real, you can try using the WMI interface
Depending on your machine, you could use NetLimiter to block the networking access for a single process. On Unix you could do something like setting up the iptables to drop all packages coming from Python.
You can run a python script in sandmode using the answer here: https://unix.stackexchange.com/a/212199, if the script is not running already.
If you gave more detail, a more detailed answer could be provided.
From what you're saying, the best answer is: "Cut the cable".
i've got a script that uses the resource-module from python (see http://docs.python.org/library/resource.html for information). Now i want to port this script to windows. is there any alternative version of this (the python-docs are labeling it as "unix only").
if there isn't, is there any other workaround?
I'm using the following method/constant:
resource.getrusage(resource.RUSAGE_CHILDREN)
resource.RLIMIT_CPU
Thank you
PS: I'm using python 2.7 / 3.2
There's no good way of doing this generically for all "Resources"" -- hence why it's a Unix only command. For CPU speed only you can either use registry keys to set the process id limit:
http://technet.microsoft.com/en-us/library/ff384148%28WS.10%29.aspx
As done here:
http://code.activestate.com/recipes/286159/
IMPORTANT: Backup your registry before trying anything with registry
Or you could set the thread priority:
http://msdn.microsoft.com/en-us/library/ms685100%28VS.85%29.aspx
As done here:
http://nullege.com/codes/search/win32process.SetThreadPriority
For other resources you'll have to scrap together similar DLL access APIs to achieve the desired effect. You should first ask yourself if you need this behavior. Oftentimes you can limit CPU time by sleeping the thread in operation at convenient times to allow the OS to swap processes and memory controls can be done problematically to check data structure sizes.
I had a program that Scraped certain data from certain Web-Pages, and when the Web-Pages changed, acted accordingly.
How would one set up the program so it continues to run in the background?
I don't need any specifics
I'm just really confused on this concept and would appreciate whatever help anybody has to offer.
start path-to-pythonw.exe your-code.py
pythonw means without console.
start means start on background.
if your python is installed system-wide, you can probably start your-code.pyw
.pyw is associated with pythonw.exe
remember you cannot use print (to stdout) in this case.
If you want to be able to just start your process and have it background itself and do a few more typical things that "daemon" processes do in Unix, look here: How do you create a daemon in Python?
There is no concept of "background" in Windows. But the UNIX shell concept of a background process can be reasonably emulated by running your Python script as a Windows service. There are a couple of suggestions in this question: Is it possible to run a Python script as a service in Windows? If possible, how?
For casual use, I suggest that you learn how to use srvany from the second answer.
You simply need to leave your program running! Please google "python daemon" and see how to implement a persistent background process in Python.
Now, you cannot know when a website changes unless you poll it. If the website is well designed, the page you are trying to poll will have a "Last-Modified" header, you can make a "HEAD" request every so often (be nice: don't poll like crazy) and act when Last-Modified is >= than the one on record. If the site is not well designed, it will not have a reliable Last-Modified or ETAG header, in that case you will have to parse manually and check for changes yourself.
Cheers.
Sorry wasn't sure how to best word this question.
My scenario is that I have some python code (on a linux machine) that uses an xml file to acquire its arguements to perform a task, on completion of the task it disposes of the xml file and waits for another xml file to arrive to do it all over again.
I'm trying to find out the best way to be alerted an xml file has arrived in a specified folder.
On way would be to continually monitor the folder in the Python code, but that would mean a lot of excess resourses used while waiting for something to turn up (which may be as little as a few times a day). Another way, would be to set up a cronjob, but it's efficiency would't be any better than monitoring from within the code. An option I was hoping was possible would be to set up some sort of interrupt that would alert the code when an xml file appeared.
Any thoughts?
Thanks.
If you're looking for something "easy" to just run a specific script when new files arrive, the incron daemon provides a very handy combination of inotify(7) and cron(8)-like support for executing programs on demand.
If you want something a little better integrated into your application, or if you can't afford the constant fork(2) and execve(2) of the incron approach, then you should probably use the inotify(7) interface directly in your script. The pyinotify module can integrate with the underlying inotify(7) interfaces.
Is there a way to run cProfile or line_profile on a script on a server?
ie: how could I get the results for one of the two methods on http://www.Example.com/cgi-bin/myScript.py
Thanks!
Not sure what line_profile is. For cProfile, you just need to direct the results to a file you can later read on the server (depending on what kind of access you have to the server).
To quote the example from the docs,
import cProfile
cProfile.run('foo()', 'fooprof')
and put all the rest of the code into a def foo(): -- then later retrieve that fooprof file and analyze it at leisure (assuming your script runs with permissions to write it in the first place, of course).
Of course you can ensure different runs get profiled into different files, etc, etc -- whether this is practical also depends on what kind of access and permissions you're getting from your hosting provider, i.e., how are you allowed to persist data, in a way that lets you retrieve that data later? That's not a question of Python, it's a question of contracts between you and your hosting provider;-).