I'm using the solution obtained from this question Regular expression to match any character being repeated more than 10 times
The regex you need is /(.)\1{9,}/.
https://regex101.com/ is recognizing it, grep recognizes it, but python does not.
Ultimately I want to replace the match with a single space, for example:
>> text = 'this is text???????????????'
>> pattern = re.compile(r'/(.)\1{5,}/')
>> re.sub(pattern,'\s',text)
'this is text '
However, search, findall, even match do not recognize the pattern, any idea as to why?
re.sub(r'(.)\1{9,}', ' ',text)
The slashes are not part of the regex, they are a syntactic construct by which some languages form regex literals (and in case of PHP's preg module, an oddity).
With your regexp, you would have matched this is text/?????????????/, and transformed it into this is text\s (note that \s has no meaning in the replacement string).
Related
I need a regular expression able to match everything but a string starting with a specific pattern (specifically index.php and what follows, like index.php?id=2342343).
Regex: match everything but:
a string starting with a specific pattern (e.g. any - empty, too - string not starting with foo):
Lookahead-based solution for NFAs:
^(?!foo).*$
^(?!foo)
Negated character class based solution for regex engines not supporting lookarounds:
^(([^f].{2}|.[^o].|.{2}[^o]).*|.{0,2})$
^([^f].{2}|.[^o].|.{2}[^o])|^.{0,2}$
a string ending with a specific pattern (say, no world. at the end):
Lookbehind-based solution:
(?<!world\.)$
^.*(?<!world\.)$
Lookahead solution:
^(?!.*world\.$).*
^(?!.*world\.$)
POSIX workaround:
^(.*([^w].{5}|.[^o].{4}|.{2}[^r].{3}|.{3}[^l].{2}|.{4}[^d].|.{5}[^.])|.{0,5})$
([^w].{5}|.[^o].{4}|.{2}[^r].{3}|.{3}[^l].{2}|.{4}[^d].|.{5}[^.]$|^.{0,5})$
a string containing specific text (say, not match a string having foo):
Lookaround-based solution:
^(?!.*foo)
^(?!.*foo).*$
POSIX workaround:
Use the online regex generator at www.formauri.es/personal/pgimeno/misc/non-match-regex
a string containing specific character (say, avoid matching a string having a | symbol):
^[^|]*$
a string equal to some string (say, not equal to foo):
Lookaround-based:
^(?!foo$)
^(?!foo$).*$
POSIX:
^(.{0,2}|.{4,}|[^f]..|.[^o].|..[^o])$
a sequence of characters:
PCRE (match any text but cat): /cat(*SKIP)(*FAIL)|[^c]*(?:c(?!at)[^c]*)*/i or /cat(*SKIP)(*FAIL)|(?:(?!cat).)+/is
Other engines allowing lookarounds: (cat)|[^c]*(?:c(?!at)[^c]*)* (or (?s)(cat)|(?:(?!cat).)*, or (cat)|[^c]+(?:c(?!at)[^c]*)*|(?:c(?!at)[^c]*)+[^c]*) and then check with language means: if Group 1 matched, it is not what we need, else, grab the match value if not empty
a certain single character or a set of characters:
Use a negated character class: [^a-z]+ (any char other than a lowercase ASCII letter)
Matching any char(s) but |: [^|]+
Demo note: the newline \n is used inside negated character classes in demos to avoid match overflow to the neighboring line(s). They are not necessary when testing individual strings.
Anchor note: In many languages, use \A to define the unambiguous start of string, and \z (in Python, it is \Z, in JavaScript, $ is OK) to define the very end of the string.
Dot note: In many flavors (but not POSIX, TRE, TCL), . matches any char but a newline char. Make sure you use a corresponding DOTALL modifier (/s in PCRE/Boost/.NET/Python/Java and /m in Ruby) for the . to match any char including a newline.
Backslash note: In languages where you have to declare patterns with C strings allowing escape sequences (like \n for a newline), you need to double the backslashes escaping special characters so that the engine could treat them as literal characters (e.g. in Java, world\. will be declared as "world\\.", or use a character class: "world[.]"). Use raw string literals (Python r'\bworld\b'), C# verbatim string literals #"world\.", or slashy strings/regex literal notations like /world\./.
You could use a negative lookahead from the start, e.g., ^(?!foo).*$ shouldn't match anything starting with foo.
You can put a ^ in the beginning of a character set to match anything but those characters.
[^=]*
will match everything but =
Just match /^index\.php/, and then reject whatever matches it.
In Python:
>>> import re
>>> p='^(?!index\.php\?[0-9]+).*$'
>>> s1='index.php?12345'
>>> re.match(p,s1)
>>> s2='index.html?12345'
>>> re.match(p,s2)
<_sre.SRE_Match object at 0xb7d65fa8>
Came across this thread after a long search. I had this problem for multiple searches and replace of some occurrences. But the pattern I used was matching till the end. Example below
import re
text = "start![image]xxx(xx.png) yyy xx![image]xxx(xxx.png) end"
replaced_text = re.sub(r'!\[image\](.*)\(.*\.png\)', '*', text)
print(replaced_text)
gave
start* end
Basically, the regex was matching from the first ![image] to the last .png, swallowing the middle yyy
Used the method posted above https://stackoverflow.com/a/17761124/429476 by Firish to break the match between the occurrence. Here the space is not matched; as the words are separated by space.
replaced_text = re.sub(r'!\[image\]([^ ]*)\([^ ]*\.png\)', '*', text)
and got what I wanted
start* yyy xx* end
This question already has an answer here:
Why is Python Regex Wildcard only matching newLine
(1 answer)
Closed 1 year ago.
The following regular expression is not returning any match:
import re
regex = '.*match.*fail.*'
pattern = re.compile(regex)
text = '\ntestmatch\ntestfail'
match = pattern.search(text)
I managed to solve the problem by changing text to repr(text) or setting text as a raw string with r'\ntestmatch\ntestfail', but I'm not sure if these are the best approaches. What is the best way to solve this problem?
Using repr or raw string on a target string is a bad idea!
By doing that newline characters are treated as literal '\n'.
This is likely to cause unexpected behavior on other test cases.
The real problem is that . matches any character EXCEPT newline.
If you want to match everything, replace . with [\s\S].
This means "whitespace or not whitespace" = "anything".
Using other character groups like [\w\W] also works,
and it is more efficient for adding exception just for newline.
One more thing, it is a good practice to use raw string in pattern string(not match target).
This will eliminate the need to escape every characters that has special meaning in normal python strings.
You could add it as an or, but make sure you \ in the regex string, so regex actually gets the \n and not a actual newline.
Something like this:
regex = '.*match(.|\\n)*fail.*'
This would match anything from the last \n to match, then any mix or number of \n until testfail. You can change this how you want, but the idea is the same. Put what you want into a grouping, and then use | as an or.
On the left is what this regex pattern matched from your example.
How to solve this problem on regex in Python?
I want to filter words regular and text from:
"A regular expression is a special text string for describing a search pattern."
I want the result like this :
"A expression is a special string for describing a search pattern."
Please help me to solve this problem on regex syntax.
import re
txt = "A regular expression is a special text string for describing a search pattern."
pattern = "(.*) regular(.*) text(.*)"
result = re.sub(pattern, r"\1\2\3", txt)
print(result) # for testing only
The explanation:
As you can see, your regular expression is
(.*) regular(.*) text(.*)
Expressions in parentheses are so called capture groups. All 3 have the same form:
.*
which means that they will match everything - . means any character, * means arbitrary number of them, including zero (empty string).
Now we may use the captured texts as \1, \2, \3, respectively, so your original text is in this notation the same as
\1 regular\2 text\3
So in the re.sub() function we keep as substituting string only
\1\2\3
which effectively strip out the parts " regular" and " text".
trying to parse JSON key names within quotes, including escaped quotes.
my thinking is: take anything between quotes not prefixed with \
(?<!\\)\".+(?<!\\)\"
where (?<!\\)\" should screen for " but not \" but Python complains about unbalanced parenthesis.
if I use (?<!\\\)\" Python is happy , but this doesn't work:
re.findall('(?<!\\\)\".+(?<!\\\)\"','"this is \"the\". key"."and this.is.the.child"')
leads:
['"this is "the". key"."and this.is.the.child"']
when I expect:
['"this is "the". key"', '"and this.is.the.child"']
split at the dot which is enclosed with " without escape.
I feel like i need an 'anything but not escaped double quote ' in the middle, but if
[^"] screens for anything but a double quote, I don't know how to negate the (?<!\\\)\" expression within a [ ] set that takes characters as literals.
i would want something like [^(?<!\\\)\"] but that doesn't work.
I tried things like [[^"]|(\")]+ (anything but a double quote, or a \" ) but that doesn't seem to work either...
Can;t seem to find the right way to do this...
Any ideas?
Thanks for help
EDIT:
My real goal is to be able to split full 'text' JSON key names to transform them into alphanum only values. The transform is irrelevant here, but the goal is to split the keys to represent the hierarchy properly. The keys are in text form.
EDIT 2:
even though OmnipotentEntity is most likely right, writing a parser will have to wait..
This solution below doesn't support the "\" or "\\" cases as indicated in his comments.
I settled with
"(?:\\"|[^"])*?"|(?<=\.)[^".]+?(?=\.)|^[^".]+?(?=\.)|(?<=\.)[^".]+?$
inspired by the answer from Avinash Raj
but adding support for keys that are not enclosed in double quotes:
no quotes beginning of line ending with .
.key.
and
.lastkey
when substituting [empty] with the same regex, one should find 1 less element than the number of found strings, or there is an error.
something like .. outside "" will fail that test
Fundamentally, using a regular expression to match quoted strings is impossible in the general case. JSON is not a regular language (all regular languages are LL(1) but not all LL(1) languages are regular, JSON is one of these), so it cannot be matched by a regular expression.
Avinash Raj's regular expression (?<!\\)".*?(?<!\\)", for instance, fails on the the case "\\". Because the quote is preceded by a \ but the backslash doesn't function as an escape. But you can't special case this situation because then "\\\"" will fail. And if you special case this situation, you can just use 4 \ and then 5 \ etc.
Lookbehinds aren't part of standard regular expressions so they can match more grammars than simply regular ones. So you might be able to come up with a regular expression that works in this case. However, I would recommend writing a parser instead, they are very easy to do for LL(1) grammars. It will be easier, more understandable, less brittle, and give you more leverage to deal with non-conformant JSON and give you the ability to write better diagnostic messages in this case.
Try to define your regex as raw string notation.
>>> s = r'"this is \"the\". key"."and this.is.the.child"'
>>> re.findall(r'"(?:\\"|[^"])*?"', s)
['"this is \\"the\\". key"', '"and this.is.the.child"']
DEMO
OR
>>> re.findall(r'(?<!\\)".*?(?<!\\)"', s)
['"this is \\"the\\". key"', '"and this.is.the.child"']
(?<!\\) called negative lookbehind which asserts that the match won't be preceded by a backslash.
" Matches a double quotes.
.*?(?<!\\)" Matches all the characters non-greedily upto the double quotes which is not preceded by a backslash.
Question:
Is there any way to tell a regular expression engine to treat a certain part of a regular expression as verbatim (i.e. look for that part exactly as it is, without the usual parsing) without manually escaping special characters?
Some context:
I'm trying to backreference a group on a given regular expression from another regular expression. For instance, suppose I want to match hello(.*?)olleh against text 1 and then look for bye$1eyb in text 2, where $1 will be replaced by whatever matched group 1 in text 1. Therefore, if text 1 happens to contain the string "helloFOOolleh", the program will look for "byeFOOeyb" in text 2.
The above works fine in most cases, but if text 1 were to contain something like "hello.olleh", the program will match not only "hello.olleh" but also "helloXolleh", "hellouolleh", etc. in text 2, as it is interpreting . as a regex special character and not the plain dot character.
Additional comments:
I can't just search for the plain string resulting from parsing $1 into whatever group 1 matches, as whatever I want to search for in text 2 could itself contain other unrelated regular expressions.
I have been trying to avoid parsing the match returned from text 1 and escape every single special character, but if anyone knows of a way to do that neatly that could also work.
I'm currently working on this in Python, but if it can be done easily with any other language/program I'm happy to give it a try.
You can use the re.escape function to escape the text you want to match literally. So after you extract your match text (e.g., "." in "hello.olleh"), apply re.escape to it before inserting it into your second regex.
To illustrate what BrenBarn wrote,
import re
text1 = "hello.olleh"
text2_match = "bye.eyb"
text2_nomatch = "byeXeyb"
found = re.fullmatch(r"hello(.*?)olleh", text1).group(1)
You can then make a new search with the re.escape:
new_search = "bye{}eyb".format(re.escape(found))
Tests:
re.search(new_search, text2_match)
#>>> <_sre.SRE_Match object; span=(0, 7), match='bye.eyb'>
re.search(new_search, text2_nomatch)
#>>> None