Related
I got this code from some 3rd party, the purpose is to "decrypt" an encrypted data (basically just customer's name data). But i don't know what is openssl_decrypt() in Python.
The PHP code i got:
<?php
$secret = "079423bc31698dbaf2d6f49973301b97";
$content = "+oY3RKkVzstJjpaBz523z9960f/KKNfV1hrYW9bUVJ0AMwwQQfa0Zi09VH3WGac8qPpZkRodMxBIMl7suYD8ck95JI4K1mnO84nacf9sh0y8577N0zYWfhaIaapiFi/ZbX3J0aJr3gMjSk+/fIjXw99kTjZEEyDtvqyvc8K8XDaa4uQbAz10Vpf61+6I7cPKfXmdWUD53ZLweoLQZfLhnH1XK3bheX6SKRR33N/NzEwE/qbaANs2f6OL7HhjyiR58BEjg+At9c26pD5n170JoadgKVjsLZk0L2+2zMJKHuxnjtm8KxqimObZB2riIXmobA2tZmQQBx2CQGwDeqeUkxbO+uxxfb4sWKeViU90QpfDNqCTELiTkS0KnVpadzSt908rdx7w3lFuIDbqZXwClutORdASz2s1t856aLnT89UT1Tm25MWcX0kAGIaw8xYwOdoUEyk0g+4d0ZVcGUDFlU2L/8mQ9jG4YTAm/9dQH57o1s+2wTuRpmltdhAXVA==";
$bcontent = base64_decode($content);
$bnonce = substr($bcontent, strlen($bcontent) - 12, strlen($bcontent));
$bcipher = substr($bcontent, 0, strlen($bcontent) - 12);
// default tag
$taglength = 16;
$tag = substr($bcipher, strlen($bcipher) - $taglength, strlen($bcipher));
$acipher = substr($bcipher, 0, strlen($bcipher) - $taglength);
$result = openssl_decrypt( $acipher, "aes-256-gcm", $secret, OPENSSL_RAW_DATA, $bnonce, $tag);
echo $result;
?>
This is the python equivalent that i tried to replicate (the code above):
import base64
secret = "079423bc31698dbaf2d6f49973301b97";
content = "+oY3RKkVzstJjpaBz523z9960f/KKNfV1hrYW9bUVJ0AMwwQQfa0Zi09VH3WGac8qPpZkRodMxBIMl7suYD8ck95JI4K1mnO84nacf9sh0y8577N0zYWfhaIaapiFi/ZbX3J0aJr3gMjSk+/fIjXw99kTjZEEyDtvqyvc8K8XDaa4uQbAz10Vpf61+6I7cPKfXmdWUD53ZLweoLQZfLhnH1XK3bheX6SKRR33N/NzEwE/qbaANs2f6OL7HhjyiR58BEjg+At9c26pD5n170JoadgKVjsLZk0L2+2zMJKHuxnjtm8KxqimObZB2riIXmobA2tZmQQBx2CQGwDeqeUkxbO+uxxfb4sWKeViU90QpfDNqCTELiTkS0KnVpadzSt908rdx7w3lFuIDbqZXwClutORdASz2s1t856aLnT89UT1Tm25MWcX0kAGIaw8xYwOdoUEyk0g+4d0ZVcGUDFlU2L/8mQ9jG4YTAm/9dQH57o1s+2wTuRpmltdhAXVA==";
bcontent = str(base64.b64decode(content),"utf-8");
bnonce = bcontent[len(bcontent) - 12:len(bcontent) - 12 + len(bcontent)];
bcipher = bcontent[0:0 + len(bcontent) - 12];
# default tag
taglength = 16;
tag = bcipher[len(bcipher) - taglength:len(bcipher) - taglength + len(bcipher)];
acipher = bcipher[0:0 + len(bcipher) - taglength];
#####
# I don't know what is the openssl_decrypt in Python. ^_^
#####
result = openssl_decrypt(acipher, "aes-256-gcm", secret, 1, bnonce, tag);
print(result,end="");
I have a minimal sample below to support my question.
I have a collection of dataframes in a dictionary that I used to generate DataTable(s) and storage them in an array with the def fDP, then storage in dictionaries the panels and tabs in order to filter them later with the Select widget as in the example. It works fine still I would like to use another Select widget to filter the tabs even further, in that case by product. I tried few different approaches and none worked out, if someone could help me out please.
I added the second Select widget at the end of the code, to give a better idea what I have in mind.
d1 = {'table1' : [['Product1', 0], ['Product2',0]],
'table2': [['Product1', 1], ['Product2',1]],
'table3': [['Product1', 0], ['Product2',3]]}
dfs = {k:pd.DataFrame(v, columns=['Product', 'value']) for k, v in zip(d1.keys(), [d1[t] for t in d1.keys()])}
def fDP(df):
tables = []
for s in df.keys():
src = ColumnDataSource(df[s])
cls = [TableColumn(field=c, title=c) for c in df[s].columns]
dt = DataTable(source=src,columns=cls, width=600, height=200,editable=True)
tables.append(dt)
return tables
plist = list(dfs['table1']['Product'].unique())
tables1 = fDP(dfs)
panels1 = {t: Panel(child=p, title='') for t, p in zip(dfs.keys(), tables1)}
tabs1 = Tabs(tabs=[x for x in panels1.values()], align='start', width=10)
ls = [x for x in dfs.keys()]
sel1 = Select(title='Select Check:', align='start', value=ls[0], options=ls, width=195, margin = (15, 5, 0, 0))
colT = column([sel1, tabs1], spacing=-26)
sel1.js_on_change('value', CustomJS(args={'sel':sel1, 'tab':tabs1, 'diPanel':panels1}
,code='''
var sv = sel.value
tab.tabs = [diPanel[sv]]
'''))
show(colT)
selP = Select(title='Select Product:', align='start', value=plist[0], options=plist, width=195, margin = (15, 5, 0, 0))
The code below is an example how you could solve it (made for bokeh v2.1.1)
The basic idea is to pass the original dataframe to the callback, then filter that data based on current dropdown values and create a new data dictionary that you then assign to the table data source.
For more practical cases (more data) I would recommend using the dataframe-js JS library.
In that case you need to use Bokeh preamble or postamble template approach like explained in this post to be able to attach this library to your generated HTML. Having dataframe-js in your code allows you to use many basic Python Pandas functions transferred to JS domain and this is more than enough for what you need.
import pandas as pd
from bokeh.models import ColumnDataSource, DataTable, TableColumn, Tabs, Select, Panel, Row, Column, CustomJS
from bokeh.plotting import show
d1 = {'table1' : [['Product1', 11], ['Product2',12]],
'table2': [['Product1', 21], ['Product2',22]],
'table3': [['Product1', 31], ['Product2',32]]}
dfs = {k:pd.DataFrame(v, columns=['Product', 'value']) for k, v in zip(d1.keys(), [d1[t] for t in d1.keys()])}
def fDP(df):
tables = []
for s in df.keys():
src = ColumnDataSource(df[s])
cls = [TableColumn(field=c, title=c) for c in df[s].columns]
dt = DataTable(source=src,columns=cls, width=600, height=200,editable=True)
tables.append(dt)
return tables
plist = list(dfs['table1']['Product'].unique())
plist.insert(0, ' ')
tables1 = fDP(dfs)
panels1 = {t: Panel(child=p, title='') for t, p in zip(dfs.keys(), tables1)}
tabs1 = Tabs(tabs=[x for x in panels1.values()], align='start', width=10)
ls = [x for x in dfs.keys()]
sel1 = Select(title='Select Check:', align='start', value=ls[0], options=ls, width=195, margin = (15, 5, 0, 0))
selP = Select(title='Select Product:', align='start', value=plist[0], options=plist, width=195, margin = (15, 5, 0, 0))
colT = Column(Row(sel1, selP), tabs1, spacing=-26)
sel1.js_on_change('value', CustomJS(args={'sel':sel1, 'selP':selP, 'tab':tabs1, 'diPanel':panels1}
,code='''
var sv = sel.value
tab.tabs = [diPanel[sv]]
selP.value = ' '
'''))
ds = ColumnDataSource(d1)
selP.js_on_change('value', CustomJS(args={'selT':sel1, 'selP':selP, 'tab':tabs1, 'diPanel':panels1, 'ds': ds}
,code='''
var tb_name = selT.value
var p_name = selP.value
var c_tb_ds = diPanel[tb_name].child.source // table datasource to update
var c_tb_ds_data = ds.data[tb_name] // original data to work with
var new_data = {}
var p_index = null
var keys = Object.keys(c_tb_ds.data)
if (index > -1) {
keys.splice(index, 1); // 1 means remove one item only => index
}
for (var k in keys){
new_data[keys[k]] = []
}
for (var i in c_tb_ds_data) {
if(p_index == null) {
for (var j in keys) {
if(c_tb_ds_data[i][j] == p_name) {
if(c_tb_ds_data[i][j].includes('Product')) {
p_index = i
}
}
}
}
}
if(p_index != null) {
for (var j in keys) {
new_data[keys[j]].push(c_tb_ds_data[p_index][j])
}
}
c_tb_ds.data = new_data // update table data source'''))
show(colT)
Result:
I can create files with C#, with accentuated letters in their name and content:
string itemtoorder = itemorderitemlist.SelectedItem.ToString();
int amounttoorder = int.Parse(itemorderamount.Text);
if (unitselect.Text == "gramm")
{
amounttoorder /= 1000;
}
DateTime noww = DateTime.Now;
int result = DateTime.Compare(dateTimePicker2.Value, noww);
if (result <= 0)
{
notifyIcon1.BalloonTipText = "Nem adhatsz fel a múltba rendelést!";
notifyIcon1.Icon = new Icon("Error.ico");
}
else
{
string today = DateTime.Today.ToString().Replace(" ", "").Replace("0:00:00", "");
string selecteddateasstring = dateTimePicker2.Value.Date.ToString().Replace(" ", "").Replace("0:00:00", "");
string[] writethingie = { itemtoorder, amounttoorder.ToString(), unitselect.Text, today,
selecteddateasstring, "M" };
string path = "data\\itemorders\\" + selecteddateasstring + "_" + itemtoorder + ".txt";
if (File.Exists(path))
{
DialogResult dialogResult = MessageBox.Show("Már van ugyanilyen rendelés ugyanerre a napra.", "Hozzáadjam a rendelt mennyiséget?", MessageBoxButtons.YesNo);
if (dialogResult == DialogResult.Yes)
{
string[] dataaaa = File.ReadAllLines(path);
dataaaa[1] = (int.Parse(dataaaa[1]) + amounttoorder).ToString();
File.WriteAllLines(path, dataaaa);
notifyIcon1.BalloonTipText = "A rendelés sikeresen ki lett bővítve!";
}
else if (dialogResult == DialogResult.No)
{
notifyIcon1.BalloonTipText = "Rendelés elvetve!";
}
}
else
{
File.WriteAllLines(path, writethingie);
notifyIcon1.Icon = new Icon("order.ico");
notifyIcon1.BalloonTipText = "Sikeres rendelésfelvétel!";
}
}
I can read the file in C#, that I created. It all works.
The problem is that if I create test data with a python script:
from datetime import datetime
from random import randrange
from datetime import timedelta
import string
def randomitem():
itemlist = [
"Ananász kocka (fagyasztott)",
"Barna rizsliszt",
"Barnarizs - Nagykun (szárazon)",
"Csicseriborsó (szárazon)",
"Csirkecomb filé (Bőr nélkül)",
"Hámozott paradicsom konzerv - Petti",
"Kukorica (morzsolt, fagyasztott)",
"Marhacomb (fagyasztott)",
"Pritamin paprika (fagyasztott)",
"Sárgarépa csík (fagyasztott)",
"Sárgarépa karika (fagyasztott)",
"Sűrített paradicsom - Aranyfácán",
"Vörösbab (szárazon)",
"Vöröshagyma kocka (fagyasztott)",
"Zeller (fagyasztott, csíkozott)",
]
return(itemlist[randrange(len(itemlist))])
def randomitem2():
itemlist = [
"Ananász SŰRÍTMÉNY",
"Citrom SŰRÍTMÉNY",
"Kókuszolaj - Barco",
"Kókusztejszín - RealThai",
]
return(itemlist[randrange(len(itemlist))])
def random_date(start, end):
delta = end - start
int_delta = (delta.days * 24 * 60 * 60) + delta.seconds
random_second = randrange(int_delta)
return start + timedelta(seconds=random_second)
def randomamount():
return randrange(10,70)
for i in range(200):
d1 = datetime.strptime('2022.1.1.', '%Y.%m.%d.')
d2 = datetime.strptime('2023.12.31.', '%Y.%m.%d.')
thida = str(random_date(d1, d2))
d3 = datetime.strptime('2021.4.1.', '%Y.%m.%d.')
d4 = datetime.strptime('2023.6.28.', '%Y.%m.%d.')
thirec = random_date(d3, d4)
if(i>160):
thisitem = randomitem2()
else:
thisitem = randomitem()
file = open(thida.replace("-",".").split()[0] + "._" + thisitem + ".txt" , "w")
file.write(thisitem + "\n")
file.write(str(randomamount()) + "\n")
if(i>160):
file.write("liter" + "\n")
else:
file.write("kilogramm" + "\n")
file.write(str(thirec + timedelta(days=4)).replace("-",".").split()[0] + "\n")
file.write(str(thirec).replace("-",".").replace(" ",".") + "\n")
file.write(thida.replace("-",".").replace(" ",".") + "\n")
Then the C# script can't read the accentuated letters.
Here's the reader:
string[] files = Directory.GetFiles("data/warehouse/");
string[] recipé = File.ReadAllLines("data/recipes/" + allines[0] + ".txt");
List<string> goodpaths = new List<string>();
List<int> goodams = new List<int>();
for (int i = 0; i < recipé.Length - 2; i += 2)
{
for (int j = 0; j < files.Length - 1; j++)
{
string cuf = Regex.Split(files[j], "._")[1];
cuf = cuf.Replace(".txt", "");
//Debug.WriteLine(recipé[i]);
//Debug.WriteLine(cuf);
if (recipé[i] == cuf)
{
//Debug.WriteLine("\n--==## Match ##==--\n");
goodpaths.Add(files[j]);
goodams.Add(int.Parse(recipé[i + 1]));
//Debug.WriteLine(files[j]);
}
}
goodpaths.Add("BREAK");
}
List<int> amos = new List<int>();
List<string> exps = new List<string>();
List<List<string>> UOT = new List<List<string>>();
string curitttttttt = "";
int counter = 0;
for (int i = 0; i < goodpaths.Count - 1; i++)
{
if (goodpaths[i] != "BREAK")
{
string[] thisisthecurrentfile = File.ReadAllLines(goodpaths[i]);
curitttttttt = thisisthecurrentfile[0];
//Debug.WriteLine(File.ReadAllLines(goodpaths[i])[0]);
amos.Add(int.Parse(thisisthecurrentfile[1]));
exps.Add(thisisthecurrentfile[5]);
}
else
{
int[] ams = amos.ToArray();
string[] exs = exps.ToArray();
int ned = goodams[counter];
int amo = int.Parse(allines[1]);
List<string>[] output = DATASET(ams, exs, ned, amo);
//Alapanyag neve
List<string> itnm = new List<string>();
itnm.Add(curitttttttt);
UOT.Add(itnm);
//Részletek
dataGridView1.ColumnCount = 1;
dataGridView1.RowCount = UOT.Count;
dataGridView1[0, counter].Value = UOT[counter][0];
counter++;
amos = new List<int>();
exps = new List<string>();
}
}
So to sum the problem:
I want to create test data with Python for a C# Windows Forms program, but the C# program can read only the files well if it creates and writes them. When the files are created by the Python script, the accentuated letters are displayed as question marks.
I want to get latitude and longitude from a webpage using beautifulsoup but they are in a script:
//<![CDATA[
theForm.oldSubmit = theForm.submit;
theForm.submit = WebForm_SaveScrollPositionSubmit;
theForm.oldOnSubmit = theForm.onsubmit;
theForm.onsubmit = WebForm_SaveScrollPositionOnSubmit;
var GMapsProperties={};function getGMapElementById(mapId,GMapElementId){var _mapId=typeof(mapId)=='string'? mapId : mapId.getDiv().id;var overlayArray=GMapsProperties[_mapId]['overlayArray'];for(var i=0;i < overlayArray.length;i++){if(overlayArray[i][0]==GMapElementId){return overlayArray[i][1];}}return null;}function removeGMapElementById(mapId,GMapElementId){var _mapId=typeof(mapId)=='string'? mapId : mapId.getDiv().id;var overlayArray=GMapsProperties[_mapId]['overlayArray'];for(var i=0;i < overlayArray.length;i++){if(overlayArray[i][0]==GMapElementId){overlayArray.splice(i,1);return;}}}function closeWindows(mapId){for(var i=0;i<GMapsProperties[mapId]['windowArray'].length;i++){GMapsProperties[mapId]['windowArray'][i][1].close();}}var _sg=_sg ||{};_sg.cs=(function(){var p={};p.createMarker=function(opt,id){var m=new google.maps.Marker(opt);if(id && m.getMap())GMapsProperties[m.getMap().getDiv().id]['overlayArray'].push([id,m]);return m;};p.createPolyline=function(opt,id){var m=new google.maps.Polyline(opt);if(id && m.getMap())GMapsProperties[m.getMap().getDiv().id]['overlayArray'].push([id,m]);return m;};p.createPolygon=function(opt,id){var m=new google.maps.Polygon(opt);if(id && m.getMap())GMapsProperties[m.getMap().getDiv().id]['overlayArray'].push([id,m]);return m;};return p;})();function addEvent(el,ev,fn){if(el.addEventListener)el.addEventListener(ev,fn,false);else if(el.attachEvent)el.attachEvent('on'+ev,fn);else el['on'+ev]=fn;}GMapsProperties['subgurim_GoogleMapControl'] = {}; var GMapsProperties_subgurim_GoogleMapControl = GMapsProperties['subgurim_GoogleMapControl']; GMapsProperties_subgurim_GoogleMapControl['enableStore'] = false; GMapsProperties_subgurim_GoogleMapControl['overlayArray'] = new Array(); GMapsProperties_subgurim_GoogleMapControl['windowArray'] = new Array();var subgurim_GoogleMapControl;function load_subgurim_GoogleMapControl(){var mapDOM = document.getElementById('subgurim_GoogleMapControl'); if (!mapDOM) return;subgurim_GoogleMapControl = new google.maps.Map(mapDOM);function subgurim_GoogleMapControlupdateValues(eventId,value){var item=document.getElementById('subgurim_GoogleMapControl_Event'+eventId);item.value=value;}google.maps.event.addListener(subgurim_GoogleMapControl, 'addoverlay', function(overlay) { if(overlay) { GMapsProperties['subgurim_GoogleMapControl']['overlayArray'].push(overlay); } });google.maps.event.addListener(subgurim_GoogleMapControl, 'clearoverlays', function() { GMapsProperties['subgurim_GoogleMapControl']['overlayArray'] = new Array(); });google.maps.event.addListener(subgurim_GoogleMapControl, 'removeoverlay', function(overlay) { removeGMapElementById('subgurim_GoogleMapControl',overlay.id) });google.maps.event.addListener(subgurim_GoogleMapControl, 'maptypeid_changed', function() { var tipo = subgurim_GoogleMapControl.getMapTypeId(); subgurim_GoogleMapControlupdateValues('0', tipo);});google.maps.event.addListener(subgurim_GoogleMapControl, 'dragend', function() { var lat = subgurim_GoogleMapControl.getCenter().lat(); var lng = subgurim_GoogleMapControl.getCenter().lng(); subgurim_GoogleMapControlupdateValues('2', lat+','+lng); });google.maps.event.addListener(subgurim_GoogleMapControl, 'zoom_changed', function() { subgurim_GoogleMapControlupdateValues('1', subgurim_GoogleMapControl.getZoom()); });subgurim_GoogleMapControl.setOptions({center:new google.maps.LatLng(35.6783546483511,51.4196634292603),disableDefaultUI:true,keyboardShortcuts:false,mapTypeControl:false,mapTypeId:google.maps.MapTypeId.ROADMAP,scrollwheel:false,zoom:14});var marker_subgurim_920435_=_sg.cs.createMarker({position:new google.maps.LatLng(35.6783546483511,51.4196634292603),clickable:true,draggable:false,map:subgurim_GoogleMapControl,raiseOnDrag:true,visible:true,icon:'/images/markers/Site/Tourism/vase.png'}, 'marker_subgurim_920435_');}addEvent(window,'load',load_subgurim_GoogleMapControl);//]]>
and I want information in this part:
{position:new google.maps.LatLng(35.6783546483511,51.4196634292603)
is it possible to access that information by using beautifulsoup or any other web-scraper?
Use Regular expression for this purpose.
import re
#Suppose script is stored in variable script_file
m = re.search('LatLng(\(.+?\))', script_file)
latlng = m.group(1)
latlng = eval(latlng)
print(latlng) #(35.6783546483511, 51.4196634292603)
import re
s = 'position:new google.maps.LatLng(35.6783546483511,51.4196634292603)'
lat, lng = map(float, re.search(r'\(([^,]+),([^)]+)', s).groups())
If you want to get Latitude and Longitude separately, use regex expression in this way:
import re
s = 'position:new google.maps.LatLng(35.6783546483511,51.4196634292603)'
Lat, Lng = map(float, re.search(r'LatLng\(([\d.]+),([\d.]+)\)',s).groups())
I am trying to convert the Java Code to Python Code and i have done it so far. Java Code works but Python Code doesn't work. Please help me.
Python Code
import random
class QLearning():
alpha = 0.1
gamma = 0.9
state_a = 0
state_b = 1
state_c = 2
state_d = 3
state_e = 4
state_f = 5
states_count = 6
states = [state_a, state_b, state_c, state_d, state_e, state_f]
R = [[0 for x in range(states_count)] for x in range(states_count)]
Q = [[0 for x in range(states_count)] for x in range(states_count)]
action_from_a = [state_b, state_d]
action_from_b = [state_a, state_c, state_e]
action_from_c = [state_c]
action_from_d = [state_a, state_e]
action_from_e = [state_b, state_d, state_f]
action_from_f = [state_c, state_e]
actions = [action_from_a, action_from_b, action_from_c, action_from_d, action_from_e, action_from_f]
state_names = ["A","B","C","D","E","F"]
def __init__(self):
self.R[self.state_b][self.state_c] = 100
self.R[self.state_f][self.state_c] = 100
def run(self):
for i in range(1000):
state = random.randrange(self.states_count)
while(state != self.state_c):
actions_from_state = self.actions[state]
index = random.randrange(len(actions_from_state))
action = actions_from_state[index]
next_state = action
q = self.Q_Value(state, action)
max_Q = self.max_q(next_state)
r = self.R_Value(state, action)
value = q + self.alpha * (r + self.gamma * max_Q - q)
self.set_q(state, action, value)
state = next_state
def max_q(self, s):
self.run().actions_from_state = self.actions[s]
max_value = 5
for i in range(len(self.run().actions_from_state)):
self.run().next_state = self.run().actions_from_state[i]
self.run().value = self.Q[s][self.run().next_state]
if self.run().value > max_value:
max_value = self.run().value
return max_value
def policy(self, state):
self.run().actions_from_state = self.actions[state]
max_value = 5
policy_goto_state = state
for i in range(len(self.run().actions_from_state)):
self.run().next_state = self.run().actions_from_state[i]
self.run().value = self.Q[state][self.run().next_state]
if self.run().value > max_value:
max_value = self.run().value
policy_goto_state = self.run().next_state
return policy_goto_state
def Q_Value(self, s,a):
return self.Q[s][a]
def set_q(self, s, a, value):
self.Q[s][a] = value
def R_Value(self, s, a):
return self.R[s][a]
def print_result(self):
print("Print Result")
for i in range(len(self.Q)):
print("Out From (0)".format(self.state_names[i]))
for j in range(len(self.Q[i])):
print(self.Q[i][j])
def show_policy(self):
print("Show Policy")
for i in range(len(self.states)):
fro = self.states[i]
to = self.policy(fro)
print("From {0} goto {1}".format(self.state_names[fro], self.state_names[to]))
obj = QLearning()
obj.run()
obj.print_result()
obj.show_policy()
Java Code
import java.text.DecimalFormat;
import java.util.Random;
public class Qlearning {
final DecimalFormat df = new DecimalFormat("#.##");
// path finding
final double alpha = 0.1;
final double gamma = 0.9;
// states A,B,C,D,E,F
// e.g. from A we can go to B or D
// from C we can only go to C
// C is goal state, reward 100 when B->C or F->C
//
// _______
// |A|B|C|
// |_____|
// |D|E|F|
// |_____|
//
final int stateA = 0;
final int stateB = 1;
final int stateC = 2;
final int stateD = 3;
final int stateE = 4;
final int stateF = 5;
final int statesCount = 6;
final int[] states = new int[]{stateA,stateB,stateC,stateD,stateE,stateF};
// http://en.wikipedia.org/wiki/Q-learning
// http://people.revoledu.com/kardi/tutorial/ReinforcementLearning/Q-Learning.htm
// Q(s,a)= Q(s,a) + alpha * (R(s,a) + gamma * Max(next state, all actions) - Q(s,a))
int[][] R = new int[statesCount][statesCount]; // reward lookup
double[][] Q = new double[statesCount][statesCount]; // Q learning
int[] actionsFromA = new int[] { stateB, stateD };
int[] actionsFromB = new int[] { stateA, stateC, stateE };
int[] actionsFromC = new int[] { stateC };
int[] actionsFromD = new int[] { stateA, stateE };
int[] actionsFromE = new int[] { stateB, stateD, stateF };
int[] actionsFromF = new int[] { stateC, stateE };
int[][] actions = new int[][] { actionsFromA, actionsFromB, actionsFromC,
actionsFromD, actionsFromE, actionsFromF };
String[] stateNames = new String[] { "A", "B", "C", "D", "E", "F" };
public Qlearning() {
init();
}
public void init() {
R[stateB][stateC] = 100; // from b to c
R[stateF][stateC] = 100; // from f to c
}
public static void main(String[] args) {
long BEGIN = System.currentTimeMillis();
Qlearning obj = new Qlearning();
obj.run();
obj.printResult();
obj.showPolicy();
long END = System.currentTimeMillis();
System.out.println("Time: " + (END - BEGIN) / 1000.0 + " sec.");
}
void run() {
/*
1. Set parameter , and environment reward matrix R
2. Initialize matrix Q as zero matrix
3. For each episode: Select random initial state
Do while not reach goal state o
Select one among all possible actions for the current state o
Using this possible action, consider to go to the next state o
Get maximum Q value of this next state based on all possible actions o
Compute o Set the next state as the current state
*/
// For each episode
Random rand = new Random();
for (int i = 0; i < 1000; i++) { // train episodes
// Select random initial state
int state = rand.nextInt(statesCount);
while (state != stateC) // goal state
{
// Select one among all possible actions for the current state
int[] actionsFromState = actions[state];
// Selection strategy is random in this example
int index = rand.nextInt(actionsFromState.length);
int action = actionsFromState[index];
// Action outcome is set to deterministic in this example
// Transition probability is 1
int nextState = action; // data structure
// Using this possible action, consider to go to the next state
double q = Q(state, action);
double maxQ = maxQ(nextState);
int r = R(state, action);
double value = q + alpha * (r + gamma * maxQ - q);
setQ(state, action, value);
// Set the next state as the current state
state = nextState;
}
}
}
double maxQ(int s) {
int[] actionsFromState = actions[s];
double maxValue = Double.MIN_VALUE;
for (int i = 0; i < actionsFromState.length; i++) {
int nextState = actionsFromState[i];
double value = Q[s][nextState];
if (value > maxValue)
maxValue = value;
}
return maxValue;
}
// get policy from state
int policy(int state) {
int[] actionsFromState = actions[state];
double maxValue = Double.MIN_VALUE;
int policyGotoState = state; // default goto self if not found
for (int i = 0; i < actionsFromState.length; i++) {
int nextState = actionsFromState[i];
double value = Q[state][nextState];
if (value > maxValue) {
maxValue = value;
policyGotoState = nextState;
}
}
return policyGotoState;
}
double Q(int s, int a) {
return Q[s][a];
}
void setQ(int s, int a, double value) {
Q[s][a] = value;
}
int R(int s, int a) {
return R[s][a];
}
void printResult() {
System.out.println("Print result");
for (int i = 0; i < Q.length; i++) {
System.out.print("out from " + stateNames[i] + ": ");
for (int j = 0; j < Q[i].length; j++) {
System.out.print(df.format(Q[i][j]) + " ");
}
System.out.println();
}
}
// policy is maxQ(states)
void showPolicy() {
System.out.println("\nshowPolicy");
for (int i = 0; i < states.length; i++) {
int from = states[i];
int to = policy(from);
System.out.println("from "+stateNames[from]+" goto "+stateNames[to]);
}
}
}
Traceback
C:\Python33\python.exe "C:/Users/Ajay/Documents/Python Scripts/RL/QLearning.py"
Traceback (most recent call last):
File "C:/Users/Ajay/Documents/Python Scripts/RL/QLearning.py", line 4, in <module>
class QLearning():
File "C:/Users/Ajay/Documents/Python Scripts/RL/QLearning.py", line 19, in QLearning
R = [[0 for x in range(states_count)] for x in range(states_count)]
File "C:/Users/Ajay/Documents/Python Scripts/RL/QLearning.py", line 19, in <listcomp>
R = [[0 for x in range(states_count)] for x in range(states_count)]
NameError: global name 'states_count' is not defined
To access all of the class attributes you define (i.e. everything between class QLearning and def __init__), you need to use self or the class name:
self.states_count
or
QLearning.states_count
I don't know the algorithm, but it is possible that these class attributes should be instance attributes (i.e. separate for each instance of the class, rather than shared amongst all instances) and therefore defined in __init__ (or other instance methods) using self anyway.