I have been using the following code to try and extract URLs from a copy of my chrome history, i have been writing this in PyCharm:
import sqlite3
import os
PATH='C:\\Users\\%s\\AppData\\Local\\Google\\Chrome\\User Data\\Default\\History - Copy' % os.environ.get('USERNAME')
HistCop = sqlite3.connect(PATH)
c = HistCop.cursor()
ccp = c.execute('SELECT url FROM urls ORDER BY "id" DESC LIMIT 5')
ccpp=ccp.fetchall()
print ccpp
My main goal is to open this up at least one url in a browser, but when I use the code:
import webbrowser
url = ccpp[4]
webbrowser.open(url)
I end up with an error. I think it does not work because ...
(u'https://stackoverflow.com/search',)
there is a "u" in front of it.
Please let me know why this happens, if there is a way to get rid of it, or if there is a better way for my goal.
It doesn't work because you're passing a tuple into a function that expects a string. cursor.fetchall() returns a list of tuples (since a row with n elements is represented as an n-tuple), so you just need to get the single element contained in the tuple:
rows = cursor.fetchall()
url = rows[4][0]
sqlite's fetchall method returns a list, which contains an item per row in the result of the query. These items are each a tuple (similar to a list) which contain the field data for that row. So:
ccpp # this is a list
ccpp[4] # this is a tuple
You can tell it's a tuple because the output you printed shows that. If you want the data from the first column, the 'url' column, you need to index it (similar to how you would a list):
ccpp[4][0] # get the first column of the fifth row
Related
I have an established connection with a notes database and I am able to loop through all the records in a view. What I am curious about if it is possible to open a document and get the data from it using python. (Like double clicking on a record from an HCL Notes Client).
Here is my code simplified:
import noteslib
db = noteslib.Database('my-domino-server','my-db.nsf', 'mypassword')
view = db.GetView('my_view')
doc = view.GetFirstDocument()
while doc:
print(doc.ColumnValues)
#here after printing the column values, I want to open the document and store it's values in a variable.
doc = view.GetNextDocument(doc)
I tried googling about LotusScript and I found the Open() method, but doc.Open() did not work.
Just use the LotusScript documentation to find examples for everything you need.
In your case you start with the NotesDatabase - class, then get an object of type NotesView and finally get a NotesDocument object.
This doc object does not need to be opened. You can directly access all items in that document either by their name or -if you don't know the name- by cycling through all items.
If you e.g. know the name of an item (can be found in the document properties box on the second tab, found with Alt + Enter) then you can read the value like this:
#Subject of a given mail
subject = doc.GetitemValue( "Subject" )[0]
#Start date of a calendar entry
startdate = doc.GetItemValue( "StartDate" )[0]
# cycle through all items
for item in doc.Items
print(item.Name)
value = item.Values
Take care: items are always arrays, even if they contain only a single value. To get the value of a single value item, always access the element at position 0.
i'm learning and would appreciate any help in this code.
The issue is trying to print the values in the data that are contained in one line of the JSON using Python.
import json
import requests
data = json.loads(response.text)
print(len(data)) #showing correct value
#where i'm going wrong below obviously this will print the first value then the second as it's indexed. Q how do I print all values when using seperate print statements when the total indexed value is unknown?
for item in data:
print(data[0]['full_name'])
print(data[1]['full_name'])
I tried without the index value this gave me the first value multiple times depending on the length.
I expect to be able to access from the JSON file each indexed value separately even though they are named the same thing "full_name" for example.
import json
import requests
data = json.loads(response.text)
print(len(data)) #showing correct value
for item in data:
print(item['full_name'])
#the below code will throw error.. because python index starts with 0
print(data[0]['full_name'])
print(data[1]['full_name'])
hope this help
Presuming data is a list of dictionaries, where each dictionary contains a full_name key:
for item in data:
print(item['full_name'])
This code sample from your post makes no sense:
for item in data:
print(data[0]['full_name'])
print(data[1]['full_name'])
Firstly it's a syntax error because there is nothing indented underneath the loop.
Secondly it's a logic error, because the loop variable is item but then you never refer to that variable.
I am attempting to parse Shodan query results and print only the results that match the criteria I have set. The output need to be in JSON format to be integrated later in Splunk.
I'd like to iterate over the set of elements and removing an element if it doesn't match the location country_code of "US".
Here is my code :
import shodan
import os
import sys
import json
SHODAN_API_KEY = os.environ.get("SHODAN_API_KEY")
api = shodan.Shodan(SHODAN_API_KEY)
query = sys.argv[1]
try:
query_results = api.search(query)
except shodan.APIError as err :
print('Error: {}'.format(err))
for element in query_results['matches']:
if 'US' in format(element['location']['country_code']):
del element
print(query_results['matches'])
But with this code my element won't get removed from query_result['matches'].
There are a few things:
Consider using the Shodan.search_cursor(query) method instead of just Shodan.search(query). The search_cursor() method handles paging through results for you in case there are more than 100 results. Otherwise you need to do that on your own by providing the page parameter to the search() method. Here is an article that explains it a bit further: https://help.shodan.io/guides/how-to-download-data-with-api
You can actually filter out non-US results within the search query! Simply add " -country:US" to your query and you won't get any results for services in the US. I.e. do the following assuming you have Python 3.7:
query_results = api.search(f'{query} -country:US')
this script is meant to parse Bloomberg finance to find the GBP value during the day, this following script does that however when it returns you get this:
{'dateTime': '2017-01-17T22:00:00Z', 'value': 1.6406}
I don't want the dateTime, or the value text there. I don't know how to get rid of it. and when I try it gives me errors like this: list index out of range.
any answers will be greatly appreciated. here is the script (in python3):
import urllib.request
import json
htmltext = urllib.request.urlopen('https://www.bloomberg.com/markets/api/bulk- time-series/price/GBPAUD%3ACUR?timeFrame=1_DAY').read().decode('utf8')
data = json.loads(htmltext)
datapoints = data[1]['price']
print(datapoints)
This should work for you.
print (data[0]['price'][-1]['value'])
EDIT: To get all the values,
for data_row in data[0]['price']:
print data_row['value']
EXPLANATION: data[0] gets the first and only element of the list, which is a dict. ['price'] gets the list corresponding to the price key. [-1] gets the last element of the list, which is presumably the data you'll be looking for as it's the latest data point.
Finally, ['value'] gets the value of the currency conversion from the dict we obtained earlier.
I have created the following Python code that reads a method from a webservice:
def GetWeatherParameters():
""""""
client = Client('www.address.asmx?wsdl')
#WebServiceClient.GetWeatherParameters()
return client.service.GetWeatherParameters()
It works fine and I get the data returned and can print it, however the data returned contains mutltiple columns and this code just prints out everything at once.
Does anybody know how I can extract the returned data column by column?
It all depends on the returned data - a handy way to display it nicely is to use pprint:
from pprint import pprint
pprint(your_data)
That'll format it nicely so it's easier to see the structure. Then if it's a list or similar, to get the first row you can do your_data[0] to get the first one, or loop, to print it row by row:
for row in your_data:
print row
print row[0] # could be the first column...
And go from there...