As part of our CI/CD we want to add some check to run code on python files and check if there are some secrets in code (like API, passwords etc.).
I saw only programs that do this, and I want to create a Python script that does it.
Does anyone have some suggestion or example for this?
I'm not sure it exactly what you look for but you can use GitGuardian API,
The GitGuardian API puts at your fingertips the power to detect more
than 200 types of secrets in any text content, as well as other
potential security vulnerabilities.
py-gitguardian can be used to create integrations to scan various data
sources, from your workstation's filesystem to your favorite chat
application.
You can check API details here with all the response codes and expected structures on each method.
just take a look GitGuardian/py-gitguardian Github repository,
You can also check this Youtube video that will help you implement this.
Good luck.
Related
I built a python script (bot) for a game and I do plan on building a GUI for it to make it more user friendly. However I wanted to add some sort of security along with it, something that would only give access to whoever I want, so maybe adding some kind of encryption key to it, I was thinking something along the lines of an encrypted key to unlock the files and with limited use(a few days for example). I am new when it comes to this specific 'security' topic, so I need help better understanding what my options are and what I can do or search for. Thank you for reading.
after days of searching and trying I figured the easiest way was to use a web API to check requests, you can use for example cryptolens web api or any other api and your encrypted file will work just fine.
Consider the following use case. I provide a Python package working as a standalone application. The application can load different more or less lass data sets (assume maybe 5 to 500mb) and do some processing / analysis of the data. The application code is hosted publicly, but I cannot provide and host the data there, since I am not the owner of the data. The data is available in many different public repositories and can be gathered from there. Also, this just helps to limit the application size and not clutter it with potentially unnecessary data (since different users might want to use very different data sets).
To make this work, I would have to provide user instructions like "If you want to use this data set, go to https://foo.bar/a.wav, download the file and place it into ./data". I would like to take that hideous procedure off the user. Hence, I was looking for a package that can help with exactly that. The internal workflow would look like something like this:
Developer defines a project resource
User picks a specific resource that should be loaded on execution (this is also just done via the relative path where the resource is supposed to be, e.g. "a.wav")
Package verifies if resource is available on the user local system
If not available, the package downloads the resource from an online source (specified by the developer)
I expected this to be a very common problem to many people. Hence I expected to easily find packages that help me realize such functionality. But I was not really not successful. Am I lacking just the appropriate terms to search for (lazy loading seems to be usually used for local data access or dynamic package imports)?!
What I found was lazydata which realizes this kind of functionality, but AFAIK in a pretty unsuitable way for me (providing the yourself on an hosted AWS instance).
I also found data-retriever which does kind of exactly what I need, but AFAIK only from predefined data repository (provided by them) and not arbitrary URLs.
Most tests are run using some kind of credentials or keys to access the system under test. When the same suite of tests is run against several different environments, this management of credentials becomes more important.
Presently we are using a Robot Framework variable file to store them locally. But this requires a file per environment, or a single file with a master switch. This works, but is not a secure solution. For this reason I'm seeking your input.
Several solutions seem to be out there, HashiCorp's Vault being one of them, that help with this management of credentials and seek your feedback on what works for you and why. If possible within the context of Robot Framework but generic solutions with Python will work too.
You can try using excel with Python and then import that as a keyword to your RF.
Clear instructions are given here in this blog post, I found that they are suitable for your req: https://blog.aspose.com/2021/06/01/encrypt-and-decrypt-excel-files-in-python/
I am planning to create a file upload website where users register as members and then upload files through both a file upload form and ftp account (each file can be up to 10gb).
For each file uploaded the member gets provided with a link which he can share with other users. Unfortunately I am just an average Django coder/linux user and have not worked on any similar project before.
Problem 1
The storage space used will potentially quickly grow to 1000s of TB's, how do I optimise the server and its storage for this? Should I use a Cloud-service or which type of Hosting would be most suitable? How would you setup the Infrastructure to make this run smoothly?
I was planning to run Freebsd as OS and Django/Python for the Development ...
Appreciate your input and all ideas!
From what you describe, I would start with a cloud service and see how the actual usage turns out. That might be the cheapest and most scalable version.
For setting things up, you have several options (surprise! :-) ). AFAIK, Amazon has some preconfigured images that might take you a long way. Since you're doing python, you could also look at Google and see how their services play together.
As you described yourself primarily as a coder, I would stay away from puppet, chef, ansible and such. While those are great tools, they add a layer of abstraction to managing actual servers. I might be wrong, of course, and such tools are just the help you need in order to set things up.
For many admin-tools, there are ready-to-use modules or templates that might help you achieve your goal.
As a simple battle-plan suggestion:
look at cloud-providers to determine which one suites you well.
look at tools to interact with the cloud-provider you are thinking about using.
try to find user-groups for the cloud-provider/admin tool you chose to learn more about them or get help from other people.
I would like to use SAS or Python to achieve this.
I have multiple points that I would like to map out using mapquest/maps.google/maps.bing, and then download a screenshot of the map on to my C: drive.
This would not be a one time thing. I would need it to download multiple maps (Have a map for route within Chicago, then another for a route within Pittsburgh).
I was hoping there may be a module within Python that enables this.
Also, on a side note, is there a way to call Python scripts from within a SAS program? Could I possibly embed it in to a VBA script which is then called from SAS?
You should be able to do the lot from SAS if the APIs for Google/MapQuest/Bing all support URI requests...
In SAS you can make the API request and retrieve and process the data. An example is shown here.
I'm not sure what the API request is going to return or how you would then use/interpret that. If it's standard XML use the SAS XML mapper. If it's anything else (ie. Image/MIME data) then things may get trickier.
The family of Google Maps APIs lets you call down various services from any language that will let you send a URL request. The Static API might be of special interest to you, though I'm not sure if you can overlay directions, or how.
To do this in Python, you could use urllib for interacting with the API; there's also pymaps.