I was asked by my supervisor to convert some Perl scripts into Python language. I'm baffled by few lines of code and I am also relatively inexperienced with Python as well. I'm an IT intern, so this was something of a challenge.
Here are the lines of code:
my %sybase;
my $S= tie %sybase, "DB_File", $prismfile, O_RDWR|O_CREAT, 0666, $DB_HASH or die "Cannot open: $!\n";
$DB_HASH->{'cachesize' } = $cache;
I'm not sure what is the equivalent of this statement in Python? DB_FILE is a Perl module. DB_HASH is a database type that allows arbitrary keys/values to be stored in data file, at least that's according to Perl documentation.
After that, the next lines of code also got me stumped on how to convert this to the equivalent in Python as well.
$scnt=0;
while(my $row=$msth->fetchrow_arrayref()) {
$scnt++;
$srow++;
#if ($scnt <= 600000) {
$S->put(join('#',#{$row}[0..5]),join('#',#{$row}[6..19]));
perf(500000,'sybase') ;#if $VERBOSE ;
# }
}
I'll probably use fetchall() in Python to store the entire result dataset in it, then work through it row by row. But I'm not sure how to implement join() correctly in Python, especially since these lines use range within the row index elements -- [0..5]. Also it seems to write the output to data file (look at put()). I'm not sure what perf() does, can anyone help me out here?
I'd appreciate any kind of help here. Thank you very much.
I have obtained the OHLC values from the iqoption and trying to find out a way to use it with MT5.
Here is how I got the values:
import time
from iqoptionapi.stable_api import IQ_Option
I_want_money=IQ_Option("email","password")
goal="EURUSD"
print("get candles")
print(I_want_money.get_candles(goal,60,111,time.time()))
The above code library is here: iqoptionapi
The line: I_want_money.get_candles(goal,60,111,time.time()) output json as : Output of the command
Now I am getting json in the output so it work like an API, I guess so.
Meanwhile, I try to create a Custom Symbol in MT5 as iqoption. Now I just wanted to add the data of the OHLC from the API to it, so that it will continue fetching data from the Iqoption and display the chart on the chart window for the custom symbol iqoption.
But I am not able to load it in the custom symbol. Kindly, help me.
Edited
This is the code for live streaming data from the iqoption:
from iqoptionapi.stable_api import IQ_Option
import logging
import time
logging.basicConfig(level=logging.DEBUG,format='%(asctime)s %(message)s')
I_want_money=IQ_Option("email","password")
I_want_money.start_candles_stream("EURUSD")
thread=I_want_money.collect_realtime_candles_thread_start("EURUSD",100)
I_want_money.start_candles_stream("USDTRY")
thread2=I_want_money.collect_realtime_candles_thread_start("USDTRY",100)
time.sleep(3)
#Do some thing
ans=I_want_money.thread_collect_realtime.items()
for k, v in ans:
print (k, v)
I_want_money.collect_realtime_candles_thread_stop(thread)
I_want_money.stop_candles_stream("EURUSD")
I_want_money.collect_realtime_candles_thread_stop(thread2)
I_want_money.stop_candles_stream("USDTRY")
Ok, you need to
1. receive the feed from the broker(I hope you succeeded)
2. write it into a file
** (both - python) **
3. read and parse it
4. add it to the history centre/marketWatch
** (both - mt5) **
So, you receive data as a string after
I_want_money.get_candles(goal,60,111,time.time())
this string might be json or json-array.
The important question is of course the path you are going to put the data. An expert in MQL45 can access only two folders (if not applying dll):
C:\Users\MY_NAME_IS_DANIEL_KNIAZ\AppData\Roaming\MetaQuotes\Terminal\MY_TERMINAL_ID_IN_HEX_FORMAT\MQL4\Files
and
C:\Users\MY_NAME_IS_DANIEL_KNIAZ\AppData\Roaming\MetaQuotes\Terminal\Common\Files
in the latter case you need to open a file with const int handle=FileOpen(,|*| FILECOMMON);
In order to parse json, you can use jason.mqh https://www.mql5.com/en/code/13663 library (there are few others) but as far as i remember it has a bug: it cannot parse array of objects correctly. In order to overcome that, I would suggest to write each tick at a separate line.
And the last, you will recieve data from your python application at random time, and write it into Common or direct folder. The MT5 robot will read it and delete. Just to avoid confusion, it could be better to guarantee that a file has a unique name. Either random (random.randint(1,1000)) or milliseconds from datetime can help.
So far, you have python code:
receivedString = I_want_money.get_candles(goal,60,111,time.time())
filePath = 'C:\Users\MY_NAME_IS_DANIEL_KNIAZ\AppData\Roaming\MetaQuotes\Terminal\MY_TERMINAL_ID_IN_HEX_FORMAT\MQL4\Files\iqoptionfeed'
fileName = os.path.join(filePath,"_"+goal+"_"+str(datetime.now())+".txt")
file = open(fileName, "w")
for string_ in receivedString:
file.write(string_)
file.close()
In case you created a thread, each time you receive an answer from the thread you write such a file.
Next, you need that data in MT5.
The easiest way is to loop over the existing files, make sure you can read them and read (or give up if you cannot) and delete after reading, then proceed with the data received.
The easiest and faster way is to use 0MQ of course, but let us do it without dll's.
In order to read the files, you need to setup a timer that can work as fast as possible, and let it go. Since you cannot make a windows app sleeping less then 15.6ms, your timer should sleep this number of time.
string path;
int OnInit()
{
EventSetMillisecondTimer(16);
path="iqoptionfeed\\*";
}
void OnDeinit(const int reason) { EventKillTimer(); }
string _fileName;
long _search_handle;
void OnTimer()
{
_search_handle=FileFindFirst(path,_fileName);
if(_search_handle!=INVALID_HANDLE)
{
do
{
ResetLastError();
FileIsExist(_fileName);
if(GetLastError()!=ERR_FILE_IS_DIRECTORY)
processFile(path+_fileName);
}
while(FileFindNext(_search_handle,_fileName));
FileFindClose(_search_handle);
}
}
this piece of code loops the folder and processes each file it managed to find.
Now reading the file (two functions) and processing the message inside it:
void processFile(const string fileName)
{
string message;
if(ReadFile(fileName,message))
processMessage(message,fileName);
}
bool ReadFile(const string fileName,string &result,const bool common=false)
{
const int handle = FileOpen(fileName,common?(FILE_COMMON|FILE_READ):FILE_READ);
if(handle==INVALID_HANDLE)
{
printf("%i - failed to find file %s (probably doesnt exist!). error=%d",__LINE__,fileName,GetLastError());
return(false);
}
Read(handle,result);
FileClose(handle);
if(!FileDelete(fileName,common?FILE_COMMON:0))
printf("%i - failed to delete file %s/%d. error=%d",__LINE__,fileName,common,GetLastError());
return(true);
}
void Read(const int handle,string &message)
{
string text="";
while(!FileIsEnding(handle) && !IsStopped())
{
text=StringConcatenate(text,FileReadString(handle),"\n");
}
//printf("%i %s - %s.",__LINE__,__FUNCTION__,text);
message=text;
}
And the last but not the least: process the obtained file.
As it was suggested above, it has a json formatted tick for each new tick, separated by \r\n.
Our goal is to add it to the symbol. In order to parse json, jason.mqh is an available solution but you can parse it manually of course.
void processMessage(const string message,const string fileName)
{
string symbolName=getSymbolFromFileName(fileName);
if(!SymbolSelect(symbolName,true))
{
if(!CustomSymbolCreate(symbolName))
return;
}
string lines[];
int size=StringSplit(message,(ushort)'\n',lines);
for(int i=0;i<size;i++)
{
if(StringLen(lines[i])==0)
continue;
CJAVal jLine(jtUNDEF,NULL);
jLine.Deserialize(lines[i]);
MqlTick mql;
//here I assume that you receive a json file like " { "time":2147483647,"bid":1.16896,"ask":1.16906,"some_other_data":"someOtherDataThatYouMayAlsoUse" } "
mql.time=(datetime)jLine["time"].ToInt();
mql.bid=(double)jLine["bid"].ToDbl();
mql.ask=(double)jLine["ask"].ToDbl();
ResetLastError();
if(CustomTicksAdd(symbolName,mql)<0)
printf("%i %s - failed to upload tick: %s %s %.5f %.5f. error=%d",__LINE__,__FILE__,symbolName,TimeToString(mql.time),mql.bid,mql.ask,GetLastError());
}
}
string getSymbolFromFileName(const string fileName)
{
string elements[];
int size=StringSplit(fileName,(ushort)'_',elements);
if(size<2)
return NULL;
return elements[1];
}
Do not forget to add debugging info and request for GetLastError() is for some reason you get errors.
Can this work in a back tester? Of course not. Fist, OnTimer() is not supported in MQL tester. Next, you need some history record in order to make it running. If you do not have any history - Nobody can help you unlees a broker can give it to you; the best idea could be to start collecting and storing it right now, and when the project is ready (maybe another couple months), you will have it ready and be able to test and optimize the strategy with the available dataset. You can apply the collected set into tester (MQL5 is really the next step in algo trading development compared to MQL4), either manually or with something like tickDataSuite and its Csv2Fxt.ex4 file that makes HST binary files that the tester can read and process; anyway that is another question and nobody can tell you if your broker stores their data somewhere to provide it to you.
After second-reading what you wrote (and edited) I can see you want:
a symbol synchronized with iqoption [ through your proxy / remotely ]
The symbol could be used for backtesting
The symbol could be used for on-screen live/strategy/indicator run
That implies operations outside strategy/indicator which MT platforms do not allow in an automated manner - you can achieve it manually by providing a data package, parsing it to CSV and importing to custom symbol creator. Well documented here.
Unfortunately, you choose a platform that by-design stands for self-contained strategies and indicators, more for beginners than professionals taking it seriously.
Refer to the link I provided and see for yourself. The official doc states you can create a custom symbol via mql ref, yet even though they state, in the foreword, it allows 3rd party providers - it's not referenced anywhere else and does not show any integration possibilities.
custom indicators
custom symbol properties
I'm using a library ABPY (library here) for python but it is in older version i think. I'm using Python 3.3.
I did fix some PRINT errors, but that's how much i know, I'm really new on programing.
I want to fetch some webpage and filter it from advertising and then print it again.
EDITED after Sg'te'gmuj told me how to convert from python 2.x to 3.x this is my new code:
#!/usr/local/bin/python3.1
import cgitb;cgitb.enable()
import urllib.request
response = urllib.request.build_opener()
response.addheaders = [('User-agent', 'Mozilla/5.0')]
response = urllib.request.urlopen("http://www.youtube.com")
html = response.read()
from abpy import Filter
with open("easylist.txt") as f:
ABPFilter = Filter(file('easylist.txt'))
ABPFilter.match(html)
print("Content-type: text/html")
print()
print (html)
Now it is displaying a blank page
Just took a peek at the library, it seems that the file "easylist.txt" does not exist; you need to create the file, and populate it with the appropriate filters (in whatever format ABP specifies).
Additionally, it appears it takes a file object; try something like this instead:
with open("easylist.txt") as f:
ABPFilter = Filter(f)
I can't say this is wholly accurate though since I have no experience with the library, but looking at it's code I'd suspect either of the two are the problem, if not both.
Addendum #1
Looking at the code more in-depth, I have to agree that even if that fix I supplied does work, you're going to have more problems (it's in 2.x as you suggested, when you're using 3.x). I'd suggest utilizing Python's 2to3 function, to convert from typical Python 2 to Python 3 code (it's not foolproof though). The command line would be as so:
2to3 -w abpy.py
That will convert it from Python 2.x to 3.x code, and re-write the source file.
Addendum #2
The code to pass the file object should be the "f" variable, as shown above (modified to represent that; I wasn't paying attention and just left the old file function call in the argument).
You need to pass a URI to the function as well:
ABPFilter.match(URI)
You'll need to modify the code to pass those items into an array (I'm assuming at least); I'm playing with it now to see. At present I'm getting a rule error (not a Python error; but merely error handling used by abpy.py, which is good because it suggests that it's the right train of thought).
The code for the Filter.match function is as following (after using the 2to3 Python script):
def match(self, url, elementtype=None):
tokens = RE_TOK.split(url)
print(tokens)
for tok in tokens:
if len(tok) > 2:
if tok in self.index:
for rule in self.index[tok]:
if rule.match(url, elementtype=elementtype):
print(str(rule))
What this means is you're, at present, at a point where you need to program the functionality; it appears this module only indicates the rule. However, that is still useful.
What this means is that you're going to have to modify this function to take the HTML, in place of the the "url" parameter. You're going to regex the HTML (this may be rather intensive) for a list of URIs and then run each item through the match loop Where you go from there to actually filter the nodes, I'm not sure; but there is a list of filter types, so I'm assuming there is a typical procedural ABP does to remove the nodes (possibly, in some cases merely by removing the given URI from the HTML?)
References
http://docs.python.org/3.3/library/2to3.html
I'm writing Python code for IOS automation using Appium, but get stuck on this issue:
the check point is check some cells in tableView, when changing the device location setting (the cells will be different, both number and text), if the text and image appears in cell correctly, and the image matches the text, like football icon matches "football", I try to find webdriver api to capture the snapshot for the cell only, but all the methods I found are for the driver, it just can capture the whole screen.
so we found a way to check this: first we capture the correct screenshot for the cells manually, then running the automation script, use the correct cell_screenshot to check if it appears in the screen
now my workmates has the java code for this:
import org.sikuli.api.DesktopScreenRegion;
import org.sikuli.api.ImageTarget;
import org.sikuli.api.ScreenRegion;
import org.sikuli.api.Target;
public class SnapShot {
static DesktopScreenRegion sr = new DesktopScreenRegion();
static Target image;
static ScreenRegion result;
static String fileURL;
private static String getImageURL(String fileURL){
return "screenshots/" + fileURL;
}`enter code here`
public static boolean imageExists(String fileURL, Double similar, int timeout){
timeout = timeout * 1000;
image = new ImageTarget(new File(getImageURL(fileURL)));
image.setMinScore(similar);
result = sr.wait(image, timeout);
if (result == null){
System.out.println("Can not find image");
return false;
}
return true;
}
}
this code can use imageExists to judge if the image can be found in another
I search for sikuli api for python, but cannot find anything, looks like there's java api only
now I'm stuck at this, could anyone help? Thanks a lot!
Sikuli supports Jython as well. In fact that is default the scripting language for Sikuli. So you can write similar logic in Jython and then call that script from Python(if at all you can). I have seen users using Java to call sikuli script. Refer here for Sikuli details.
I am relativly new to coding with Python.
I recently set up a gps logging device with a raspberry pi and I want to make my log file look cleaner.
My current code for logging is:
logging.info('Altitude:')
logging.info(gpsd.fix.altitude)
It logs:
INFO:root:Altitude:
INFO:root:80
What I want to see in the log is:
Altitude: 80
I tried to do this with my limited knowledge with python, but it only resulted in failure.
Thanks!
Also, any other tips for cleaning up the log file?
If altitude is a decimal then
logging.info('Altitude: %d' % gpsd.fix.altitude)
will do it, there are several other ways to achieve the same thing though as I'm sure others can present!
In Python >=3.6 you can do this :
logging.info(f"Altitude: {gpsd.fix.altitude}")
By adding the "f" at the beginning of the string the value between brackets will be interpreted as a variable and will be replaced by its value.
logging.info('{}:{}'.format("Altitude", gpsd.fix.altitude)
You can use format method. Have a look at the examples to understand format better.
Example:
print '{}:{}'.format("Altitude", 80)
Output
Altitude:80
Try:
logging.info('Altitude:%s' % gpsd.fix.altitude)