We Keep Coding

Visualizing pie charts on a map

I have an excel where each row has latitude and longitude data associated with some integer values. I would like to visualize this as pie charts on a map.
What I tried so far:
Google Data Studio: The only drawback here is that we cannot zoom on the map and the map is zoomed at country level but all my data is about a district in a city.
Python with Folium: Folium is a wrapper for Leaflet.js which is excellent for geographical visualization. However it lacks pie chart feature. I looked at integrating with Vega but that is only good for popups on Markers. This is not good, I want the pie charts on the map directly.
Can you recommend any free tool, or Python solution for this?
I come from a Python background mainly, but I welcome JS based solutions as well.

I think Highcharts can help you with what you are looking for. They are based on javascript.
An example for precisely what you are looking for -
https://jsfiddle.net/gh/get/library/pure/highcharts/highcharts/tree/master/samples/maps/demo/map-pies/
{
title: {
text: 'USA 2016 Presidential Election Results'
},
chart: {
animation: false // Disable animation, especially for zooming
},
colorAxis: {
dataClasses: [{
from: -1,
to: 0,
color: 'rgba(244,91,91,0.5)',
name: 'Republican'
}, {
from: 0,
to: 1,
color: 'rgba(124,181,236,0.5)',
name: 'Democrat'
}, {
from: 2,
to: 3,
name: 'Libertarian',
color: libColor
}, {
from: 3,
to: 4,
name: 'Green',
color: grnColor
}]
},
mapNavigation: {
enabled: true
},
// Limit zoom range
yAxis: {
minRange: 2300
},
tooltip: {
useHTML: true
},
// Default options for the pies
plotOptions: {
mappie: {
borderColor: 'rgba(255,255,255,0.4)',
borderWidth: 1,
tooltip: {
headerFormat: ''
}
}
},
series: [{
mapData: Highcharts.maps['countries/us/us-all'],
data: data,
name: 'States',
borderColor: '#FFF',
showInLegend: false,
joinBy: ['name', 'id'],
keys: ['id', 'demVotes', 'repVotes', 'libVotes', 'grnVotes',
'sumVotes', 'value', 'pieOffset'],
tooltip: {
headerFormat: '',
pointFormatter: function () {
var hoverVotes = this.hoverVotes; // Used by pie only
return '<b>' + this.id + ' votes</b><br/>' +
Highcharts.map([
['Democrats', this.demVotes, demColor],
['Republicans', this.repVotes, repColor],
['Libertarians', this.libVotes, libColor],
['Green', this.grnVotes, grnColor]
].sort(function (a, b) {
return b[1] - a[1]; // Sort tooltip by most votes
}), function (line) {
return '<span style="color:' + line[2] +
// Colorized bullet
'">\u25CF</span> ' +
// Party and votes
(line[0] === hoverVotes ? '<b>' : '') +
line[0] + ': ' +
Highcharts.numberFormat(line[1], 0) +
(line[0] === hoverVotes ? '</b>' : '') +
'<br/>';
}).join('') +
'<hr/>Total: ' + Highcharts.numberFormat(this.sumVotes, 0);
}
}
}, {
name: 'Separators',
type: 'mapline',
data: Highcharts.geojson(Highcharts.maps['countries/us/us-all'], 'mapline'),
color: '#707070',
showInLegend: false,
enableMouseTracking: false
}, {
name: 'Connectors',
type: 'mapline',
color: 'rgba(130, 130, 130, 0.5)',
zIndex: 5,
showInLegend: false,
enableMouseTracking: false
}]
}

react-table not rendering server-side data in table (with manual pagination, filtering & sorting)

I'm having trouble getting data from the backend (Python API) to show in react-table manually. I've read the documentation and I'm trying to use the example here: https://react-table.js.org/#/story/server-side-data
I'm only seeing data in one column and only for 6 records which is really weird. It's probably the way I'm mixing in async/await syntax with the example code which uses a promise. I was able to create a simple react-table fetching data with the same async/await syntax, but when I added the server-side data code from the example (the requestData function) it wouldn't work.
I've spent days on this and looking all over Stackoverflow and the internet. I'm a newbie so please go easy on me. Here's what I have:
import React from 'react'
import { render } from 'react-dom'
import ReactTable from 'react-table'
import api from 'src/api'
import { orderBy } from 'lodash'
// importing react-table css would not work so I added it using cdn link
const requestData = async (pageSize, page, sorted, filtered) => {
// api is a wrapper for axios.create()
const rawData = await api.admin.exercise.feed()
return new Promise((resolve, reject) => {
let filteredData = rawData;
if (filtered.length) {
filteredData = filtered.reduce((filteredSoFar, nextFilter) => {
return filteredSoFar.filter(row => {
return (row[nextFilter.id] + "").includes(nextFilter.value);
});
}, filteredData);
}
const sortedData = orderBy(
filteredData,
sorted.map(sort => {
return row => {
if (row[sort.id] === null || row[sort.id] === undefined) {
return -Infinity;
}
return typeof row[sort.id] === "string"
? row[sort.id].toLowerCase()
: row[sort.id];
};
}),
sorted.map(d => (d.desc ? "desc" : "asc"))
);
const res = {
rows: sortedData.slice(pageSize * page, pageSize * page + pageSize),
pages: Math.ceil(filteredData.length / pageSize)
};
resolve(res);
});
};
export class ExerciseList extends React.Component {
constructor() {
super();
this.state = {
data: [],
pages: null,
loading: true
};
this.fetchData = this.fetchData.bind(this);
}
setLoading(loading) {
this.setState({ loading })
}
fetchData(state, instance) {
this.setLoading(true);
requestData(
state.pageSize,
state.page,
state.sorted,
state.filtered
).then(res => {
this.setState({
data: res.rows,
pages: res.pages,
loading: false
});
});
}
render() {
const { data, pages, loading } = this.state;
return (
<div>
<ReactTable
columns={[
{
Header: "Name",
accessor: "name"
},
{
Header: "Movement",
accessor: "movement"
},
{
Header: "Equipment",
accessor: "equipments"
},
{
Header: "Channel",
accessor: "channel"
},
{
Header: "Level",
accessor: "skill_level"
},
{
Header: "Duration",
accessor: "duration",
filterable: false
},
{
Header: "Injuries",
accessor: "injuries"
},
{
Header: "Is Substitute",
accessor: "has_video",
Cell: ({ value }) => (value? 'Yes': 'No'),
filterable: false
}
]}
data={data}
pages={pages}
loading={loading}
onFetchData={this.fetchData}
manual
filterable
defaultPageSize={10}
className="-striped -highlight"
/>
</div>
);
}
}
render(<ExerciseList />, document.getElementById('datatable'));

Please refer the link for server-side sorting, pagination and manual filtering within the grid
// Component related to methods for sorting, pagination server side and filtering manual filtering with in the grid
import React from 'react'
import 'react-table/react-table.css'
import ReactTable from 'react-table'
import autoBind from 'react-autobind'
import {filterCaseInsensitive} from '../../helper/commonMethods'
class ServerSideAtomGrid extends React.Component {
super(props)
const userDetails = getUserDetails()
this.state = {
page: 0,
pageSizeOptions: [500, 1000, 2000, 4000],
pageSize: 500,
totalRecords: 0,
nextCursor: '*',
cursorList: [{
page: 0,
cursor: '*'
}],
sortFields: {
field: 'created_dtm',
sort: 'desc'
},
columnData,
}
autoBind(this)
}
handlePageChange (page) {
const pageNumber = (page)
const cursorList = this.state.cursorList
let cusrsorMark = ''
_.each(cursorList, (list) => {
if (list.page === pageNumber) {
cusrsorMark = list.cursor
}
})
this.setState({
nextCursor: cusrsorMark,
page: pageNumber
}, () => this.searchData(cusrsorMark, pageNumber))
}
handleSizePerPageChange (pageSize) {
this.resetData(pageSize)
this.searchData('*', 0)
}
handleSorting = (state, instance) => {
const sorted = state
let field = 'created_dtm'
let sort = 'desc'
sorted && sorted.length > 0 && sorted.map(fld => {
field = fld.id
sort = fld.desc ? 'desc' : 'asc'
})
this.setState({
sortFields: {
field,
sort
}
}, () => this.searchData('*', 0))
}
////
searchData('*', 0) {
//Axios call you cna have
}
filterCaseInsensitive (filter, row) {
const id = filter.pivotId || filter.id
return row[id] ? row[id].toString().toLowerCase().includes(filter.value.toLowerCase()) : true
}
render () {
const {
classes, gridData, gridColumns, defaultFilter, totalRecords,
gridPageSizeOptions, gridPage, gridPages, gridPageSize, gridLoading
} = this.props
return (
<div>
<ReactTable
columns={gridColumns}
data={gridData}
onSortedChange={(state, instance) => {
this.handleSorting(state, instance)
}}
filterable={defaultFilter}
defaultFilterMethod={filterCaseInsensitive}
noDataText="Ops No result found!"
defaultPageSize={this.state.pageSize}
className="-highlight"
style={{height: `${totalRecords < 25 ? '' : `800px`}`, width: '100%', textAlign: 'center'}}
pageText={`Total Count : ${totalRecords.toLocaleString()} Page: `}
loading={gridLoading}
page={this.state.page}
pages={this.state.pages}
showPaginationTop
pageSize={this.state.pageSize}
pageSizeOptions={gthis.state.pageSizeOptions}
minRows={25}
manual
onPageChange={page => {
this.setState({page})
this.handlePageChange(page)
}}
onPageSizeChange={(pageSize, page) => {
this.setState({
page,
pageSize
})
this.props.handleSizePerPageChange(pageSize)
}}
showPageJump={false}
/>
</div>
)
}
}
export default (ServerSideAtomGrid)
My Fiddle: https://jsfiddle.net/gowthamguruju/o9ybxqaj/8/

Best way to refresh graph without page refresh (Python Django, ajax)

A bit of a general question - I am looking for ways to refresh a graph on a Django page based on user choices. The page has a graph, a few drop boxes where you can select parameters and a refresh button. Currently, I can capture the selections via ajax to my Django view and generate new data from database for the graph. I now need to feed that newly-generated data back into the graph and refresh it without a page refresh. Could anyone recommend the best methods of doing this?

Use JQuery to refresh graph without refreshing page.
I am using chart.js to create graph. first create a graph and on change event get updated data using Ajax URL call and assign values to chart data sets.
/** Graph Start Here */
window.chart = null;
$(document).on('change', '.graph-year-earning', function () {
var year = $(this).val();
$.get($('.graph-ajaxload-context').data('href'), { 'year': year, 'number': Math.floor(Math.random() * (1000000 - 10 + 1) + 10) }, function (response) {
window.chart.data.labels = response.labels;
window.chart.data.datasets[0].soldProductLabel = response.product_sold_label;
window.chart.data.datasets[0].totalCommissionLabel = response.monthly_commission_label;
window.chart.data.datasets[0].dataLabel = response.your_share_label;
if (response.total_commission == 0) {
window.chart.options.scales.yAxes[0].ticks.suggestedMin = 0;
window.chart.options.scales.yAxes[0].ticks.suggestedMax = 140000;
} else {
window.chart.options.scales.yAxes[0].ticks.suggestedMin = '';
window.chart.options.scales.yAxes[0].ticks.suggestedMax = '';
}
$.each(response.data, function (index, value) {
window.chart.data.datasets[0].soldProduct[index] = value[2];
window.chart.data.datasets[0].data[index] = Math.round(value[0]);
});
window.chart.update();
$(".txt-total-commission-by-year").html(response.total_commission)
$('.graph-ajaxload-context .inline-loader').hide();
});
});
if ($('.graph-ajaxload-context').length > 0) {
showLoader()
$('.graph-year-earning').trigger('change');
var ctx = $('#userEarningGraph');
window.chart = new Chart(ctx, {
type: 'bar',
data: {
labels: [],
datasets: [{
soldProductLabel: '',
soldProduct: [],
dataLabel: '',
data: [],
backgroundColor: '#ADAEB1',
hoverBackgroundColor: '#48C6B9'
}]
},
options: {
legend: {
display: false
},
scales: {
yAxes: [{
ticks: {
beginAtZero: true,
maxTicksLimit: 8,
userCallback: function (value, index, values) {
value = value.toString();
value = value.split(/(?=(?:...)*$)/);
value = value.join(',');
var currency_code = ' ₩'
if ($('.graph-ajaxload-context').data('currency-code') && $('.graph-ajaxload-context').data('currency-code') != 'None') {
currency_code = $('.graph-ajaxload-context').data('currency-code')
}
return value + ' ' + currency_code;
}
},
}]
},
tooltips: {
mode: 'label',
callbacks: {
label: function (tooltipItem, data) {
var soldProduct = data.datasets[tooltipItem.datasetIndex].soldProduct[tooltipItem.index];
var soldProductLabel = data.datasets[tooltipItem.datasetIndex].soldProductLabel;
var dataPro = data.datasets[tooltipItem.datasetIndex].data[tooltipItem.index];
var dataLabel = data.datasets[tooltipItem.datasetIndex].dataLabel;
return [soldProductLabel + ':' + soldProduct, dataLabel + ':' + dataPro + ' ₩',];
}
}
}
}
});
}
$(document).on('click', '.showgraph', function (e) {
$('.graph-year-earning').trigger('change');
});
/** Graph End Here */

d3.js Multi-Series Graph with JSON

I have parsed JSON from those models in the view to a JsonResponse and now I am trying to my multi-line plot working using d3.js.
The example code I am trying to use is here.
I am not sure how to go about changing the tsv call or if I should change my JSON format.
d3.tsv("data.tsv", function(error, data) {
if (error) throw error;
color.domain(d3.keys(data[0]).filter(function(key) { return key !== "date"; }));
data.forEach(function(d) {
d.date = parseDate(d.date);
});
var cities = color.domain().map(function(name) {
return {
name: name,
values: data.map(function(d) {
return {date: d.date, temperature: +d[name]};
})
};
});
x.domain(d3.extent(data, function(d) { return d.date; }));
y.domain([
d3.min(cities, function(c) { return d3.min(c.values, function(v) { return v.temperature; }); }),
d3.max(cities, function(c) { return d3.max(c.values, function(v) { return v.temperature; }); })
]);
And here is how my JSON is formatted where the Site name is top level, and the `Metric is one level deep.
{ "Site1":
[{"da": 1.0, "date": 2015-09-01},
{"da": 2.0, "date": 2015-09-04}],
"Site2":
[{"da": 1.0, "date": 2015-09-01},
{"da": 2.0, "date": 2015-09-04}
{"da": 5.0, "date": 2015-09-04}]
}
How can I setup d3.js to work with these nested JSON objects versus the top level JSON in the example?

Here is the full working code fiddle; I have added comments in the code to help you understand the code.
Fiddle
Full code snippet:
var margin = {
top: 20,
right: 80,
bottom: 30,
left: 50
},
width = 960 - margin.left - margin.right,
height = 500 - margin.top - margin.bottom;
var parseDate = d3.time.format("%Y-%m-%d").parse;
var x = d3.time.scale()
.range([0, width]);
var y = d3.scale.linear()
.range([height, 0]);
var color = d3.scale.category10();
var xAxis = d3.svg.axis()
.scale(x)
.orient("bottom");
var yAxis = d3.svg.axis()
.scale(y)
.orient("left");
var line = d3.svg.line()
.interpolate("basis")
.x(function (d) {
return x(d.date);
})
.y(function (d) {
return y(d.da);
});
var svg = d3.select("body").append("svg")
.attr("width", width + margin.left + margin.right)
.attr("height", height + margin.top + margin.bottom)
.append("g")
.attr("transform", "translate(" + margin.left + "," + margin.top + ")");
//your dataset
var myData = {
"Site1": [{
"da": 1.0,
"date": "2015-09-01"
}, {
"da": 2.0,
"date": "2015-09-04"
}],
"Site2": [{
"da": 1.0,
"date": "2015-09-01"
}, {
"da": 2.0,
"date": "2015-09-04"
}, {
"da": 5.0,
"date": "2015-09-04"
}]
};
//make fulldataset to get the extent of x axis and yaxis
var fullDataSet = []
for (var key in myData) {
fullDataSet = fullDataSet.concat.apply(fullDataSet, myData[key]);
}
fullDataSet.forEach(function (d) {
d.date = parseDate(d.date);
d.da = +d.da;
});
//get the xaxis extent i.e. min max
x.domain(d3.extent(fullDataSet, function (d) {
return d.date;
}));
//get the yaxis extent i.e. min max
y.domain(d3.extent(fullDataSet, function (d) {
return d.da;
}));
//make the x axis
svg.append("g")
.attr("class", "x axis")
.attr("transform", "translate(0," + height + ")")
.call(xAxis);
//make the y axis
svg.append("g")
.attr("class", "y axis")
.call(yAxis)
.append("text")
.attr("transform", "rotate(-90)")
.attr("y", 6)
.attr("dy", ".71em")
.style("text-anchor", "end")
.text("DA ");
//iterate through your nested data and make the line graph
for (var key in myData) {
makeLine(myData[key], key);
}
//This function will make the line chart
function makeLine(data, title) {
svg.append("path")
.datum(data)
.attr("class", "line")
.style("stroke", function (d) {
return color(title);
}).attr("d", line);
}

Various queries - MongoDB

This is my table:
unicorns = {'name':'George',
'actions':[{'action':'jump', 'time':123123},
{'action':'run', 'time':345345},
...]}
How can I perform the following queries?
Grab the time of all actions of all unicorns where action=='jump' ?
Grab all actions of all unicorns where time is equal?
e.g. {'action':'jump', 'time':123} and {'action':'stomp', 'time':123}
Help would be amazing =)

For the second query, you need to use MapReduce, which can get a big hairy. This will work:
map = function() {
for (var i = 0, j = this.actions.length; i < j; i++) {
emit(this.actions[i].time, this.actions[i].action);
}
}
reduce = function(key, value_array) {
var array = [];
for (var i = 0, j = value_array.length; i < j; i++) {
if (value_array[i]['actions']) {
array = array.concat(value_array[i]['actions']);
} else {
array.push(value_array[i]);
}
}
return ({ actions: array });
}
res = db.test.mapReduce(map, reduce)
db[res.result].find()
This would return something like this, where the _id keys are your timestamps:
{ "_id" : 123, "value" : { "actions" : [ "jump" ] } }
{ "_id" : 125, "value" : { "actions" : [ "neigh", "canter" ] } }
{ "_id" : 127, "value" : { "actions" : [ "whinny" ] } }
Unfortunately, mongo doesn't currently support returning an array from a reduce function, thus necessitating the silly {actions: [...]} syntax.

Use dot-separated notation:
db.unicorns.find({'actions.action' : 'jump'})
Similarly for times:
db.unicorns.find({'actions.time' : 123})
Edit: if you want to group the results by time, you'll have to use MapReduce.