output list of special letters string with python - python

I have 2 lists of strings, they are constructed from database content but I have difficulty in reading it into a txt file
list_a={u'XYZ-123-456',u'XYZ-123-456',u'XYZ-789-456',....}
list_b={u'Tom \n is creative', u'Sky is blue',....}
What I need to do is get rid of 'u' at the beginning of each string in both lists, getting rid of \n within each string in list_b where applicable.
then write each element in list_b to a text file, one by a line, into the txt file, whose title is the element in list_a. Strings with the same title should be in the same txt file. Anyone can help ?
I think I can use pop function to get rid of u, but how about the '\n' within the string?
I am also not 100% sure about the I/O function. Anyone can help me out with the sample script?
Thanks a lot, a newbie of python


Welcome so SO.
list_a is actually a set. The {} indicates set, which is a list of unique items - so you may not want this type if you expect multiple values in the 'list'. If you want an actual list then encapsulate the 'list' with [...] instead of {...}
The 'u' in the set indicates the text is unicode. if you want this removed:
list_a = map(str, list_a)
\n is a control character indicating a new line. This will be represented as a new line when printed in the console. If you really don't want new lines then do:
list_b= { item.replace('\n','') for item in list_b }
I assume you want each item on a new line in the text file. If so:
with open('c:\file.txt', 'w') as file:
file.write('\n'.join(list_a))
file.write('\n'.join(list_b))
for non duplicates merge the 2 lists into a set. As they are currently already sets this will work:
with open('c:\file.txt', 'w') as file:
file.write('\n'.join(list_a + list_b))

Related

Turning this list off a website into a python list

https://github.com/danielmiessler/SecLists/blob/master/Passwords/10k_most_common.txt
What would be the most efficient way of turning this list into a python list, I only really need the top 100.
Thanks, leon.
FYI, I have looked around the website and have found many thing relating to multiline lists but nothing has helped me.

yourlist = open("10k_most_common.txt", "r").read().split("\n")[:100]
open() : opens the file; takes two arguments, the file name, and what you want to do with it, in this case you want to r read it
read() : so now you have the file object, but you need what's written inside of it; with the function read() you get all the content of the file as a string
split() : it divides the string by the argument, in this case \n (the newlines), so the string becomes a list
[:100] : With this, you can specificates what part of the list you want. So [:100] is the same as [0:100:1], that means: start from index zero, arrive to index 100, with an intervall between element of one (so every element from 0 to 100)
If it is too hard to you to understand you can use this longer form:
file = open("10k_most_common.txt", "r")
string = file.read()
yourlist = string.split("\n")[:100]

unable to convert python type str into a list array

i'm new to python, and i am developing a tool/script for ssh and sftp. i noticed some of the code i'm using creates what i thought was a string array
channel_data = str()
to hold the console output from an ssh session. if i check "type" on channel_data it comes back as class 'str' ,
but yet if i perform for loop to read each item in channel_data , and channel_data contains what appears to be 30 lines from an ssh console
for line in channel_data:
if "my text" in line:
found = True
each iteration of "line" shows a single character, as if the whole ssh console output of 30 lines of text is broken down into single character array. i do have \n within all the text.
for example channel_data would contain "Cisco Nexus Operation System (NX-OS) Software\r\nCopyright (c) 2002-2016\r\n ..... etc. etc.. ", but again would read in my for loop and print out "C" then "i" then "s" etc..
i'm trying to understand do i have a char array here or a string array here that is made up of single string characters and how to convert it into a string list based on \n within Python?

You can iterate a string just like a list in Python. So, yes, as expected, your string type channel_data will in fact give you every character.
Python does not have a char array. You will have a list of strings, even as a single character as each item in the list:
>>> type(['a', 'b'])
<type 'list'>
Also, just for the sake of adding some extra information for your own knowledge when it comes to usage of terminology, there is a difference between array and list in Python: Python List vs. Array - when to use?
So, what you are actually looking to do here is take the channel_data string and make it a list by calling the split method on it.
The split method will, by default, split on white space characters only. Check the documentation. So, you will want to make sure what you want to actually split on and provide that detail to the method.
You can take a look at splitlines to see if that works for you.
As specified in the documentation for splitlines:
Line breaks are not included in the resulting list unless keepends is
given and true.
Your result will then be a list of strings as you expect. So, as an example you can do:
your_new_list_of_str = channel_data.split('\n')
or
your_new_list_of_str = channel_data.splitlines()

string_list = channel_data.splitlines()
See docs at https://docs.python.org/3.6/library/stdtypes.html#str.splitlines

Remove duplicates from a list in Python

I have a python script which parses an xml file and then gives me the required information. My output looks like this, and is 100% correct:
output = ['77:275,77:424,77:425,77:426,77:427,77:412,77:413,77:414,77:412,77:413,77:414,77:412,77:413,77:414,77:412,77:413,77:414,77:431,77:432,77:433,77:435,77:467,77:470,77:471,77:484,77:485,77:475,77:476,77:437,77:438,77:439,77:440,77:442,77:443,77:444,77:445,77:446,77:447,77:449,77:450,77:451,77:454,77:455,77:456,77:305,77:309,77:496,77:497,77:500,77:504,77:506,77:507,77:508,77:513,77:515,77:514,77:517,77:518,77:519,77:521,77:522,77:523,77:403,77:406,77:404,77:405,77:403,77:406,77:404,77:405,77:526,77:496,77:497,77:500,77:504,77:506,77:507,77:508,77:513,77:515,77:514,77:517,77:518,77:519,77:521,77:522,77:523,77:403,77:406,77:404,77:405,77:403,77:406,77:404,77:405,77:526,77:317,77:321,77:346,77:349,77:350,77:351,77:496,77:497,77:500,77:504,77:506,77:507,77:508,77:513,77:515,77:514,77:517,77:518,77:519,77:521,77:522,77:523,77:403,77:406,77:404,77:405,77:403,77:406,77:404,77:405,77:526,77:496,77:497,77:500,77:504,77:506,77:507,77:508,77:513,77:515,77:514,77:517,77:518,77:519,77:521,77:522,77:523,77:403,77:406,77:404,77:405,77:403,77:406,77:404,77:405,77:526,77:362,77:367,77:369,77:374,77:370,77:372,77:373,77:387,77:388,77:389,77:392,77:393,77:394,77:328,77:283,77:284,77:285,77:288,77:289,77:290,77:292,']
It is all fine, but I want to remove the duplicate elements in an element, like in the case above. I tried using the OrderedDict package or just simple list(set(output)), but obvoiusly they both didn't work. Does anyone have a tip for me on how to solve this problem.

You have one element in a list. If you expected it to be treated as separate elements, you need to explicitly split it.
You could split the string on the ',' comma character into a list with str.split():
separate_elements = output[0].split(',')
after which you can use set() (unordered) or OrderedDict (maintaining order) and re-join the string if you still need just the one string object:
','.join(set(separate_elements))
You can put that back into a list with just one element, but there is little point if all you ever handle is that one string.

Why does ['Mr David',23,'City'] become [[\'Mr David\',23,\'City\'] in Python?

My problem is this: ['Mr David',23,'City'] becomes [\'Mr David\',23,\'City\']. Can you please suggest how to fix it?
here is some code if you wanna see..
path = r'D:/ListFile'
rdata = open(path,'rb')
ListNow = []
for ch in rdata:
ListNow.append(ch)
print ListNow
What I am trying to do: I read it from a file and try to rewrite it to memory (because I dont know how to work with list stored in files which are on disk?)

I think you have a file D:\listfile with lines containing python lists such as
['Mr David',23,'City']
which you want to read into a list of lists. However as you loop over the file, each line is read to a string. So ch is a string, but you were expecting it to be a list.
If you trust the contents of the list file to contain safe expressions you can get python to evaluate the strings
ListNow.append(eval(ch))
This is dangerous if the listfile isn't trusted (for example if it contains data collected from a website) because malicious code in the listfile would be run. In that case you would have to analyse the string, starting with ch.split(',')

When you print a list in Python, it uses repr to reproduce the value, which will represent each list item as a Python string literal. If you want to print each value separately, you can do so:
for item in ListNow:
print item
(arshajii’s comment below might explain that more clearly, by the way. Thanks!)

Python - print a list adding empty lines at determined points

I have a list that contains logs, and I am trying to add an empty line between specific blocks. So when I get to the end of a specific block, I can see a bit of separation, instead of looking like a continuous list.
EX:
log a:
kdsaldklsadkaslk
kasldkasldkasldkasldk
log a1:
lkpadkfaldkfdsl
klsdkfldskfsdl
So far I've tried all that I was able to find online, but I was unsuccessful. Either I am forced to add anything but an empty line (like a sequence of ----- for example), or the space added will be added to every single line (which is not what I want).
If I add in the list the empty line, like
log_list.append(" \n")
when I print the list using
print "\n".join(log_list)
all the empty lines that I have added are not printed.
But if I add any character to the append command, then it will be printed.
Is there any option that is automatically taking off the empty lines in a list, when I do the join command? Otherwise I do not understand why I cannot have an empty line in the list.
Is there another way to print out lists? I've always seen printing lists with the join command (all my objects in the list are strings).
Thanks!

For me, this works (showing that there is no monkey business with appending a \n and then joining with a \n):
>>> log_list = ['a']
>>> log_list.append('\n')
>>> log_list.append('b')
>>> log_list.append('c')
>>> print '\n'.join(log_list)
a
b
c
What did you do differently?

If the list is already created, you can for example insert a blank line before every line ending with ":" like this
print "\n".join("\n"+s if s.endswith(":") else s for s in log_list)
It's probably clearer to just use a loop though
for s in log_list:
if s.endswith(":"):
print
print s
You can change the condition to suit your requirements, eg s.startswith("log ")

Categories