We use Elasticsearch + Kibana for data analysis, and we love it. Very often, we share the results of our analysis with the team. Kibana expects us to visualize data in the form of graphs and charts, but some use cases are more tuned for data-table with raw filtered query results. Surprisingly, there is no way to export CSV or create a data table in Kibana. A quick Google search gave these results, not only us; many are waiting for this feature.
We did a quick hack to GTD in the startup way; we have written a chrome plugin, it injects the CSV export functionality in Kibana Discover tab.
Business Users: Go ahead install this plugin from the chrome web store.
Developers: Here is the Github link, and it is an open source project under MIT license. Please go ahead and tweak for you need. It would be great if you can provide a link back to this blog from the places you refer.
Future: We can write an elasticsearch plugin and pair it with this chrome plugin to make it powerful. We know Kibana team is trying to solve this problem in a holistic way, it takes time, so we thought this might help users like us. Let us know, what you think?
In one of his blogs Avinash Kaushik makes a very important distinction between reporting and analysis. Reporting is a series of metrics and data points and analysis(typically) is the inferences and recommendations that are then made on that data. By definition most analytical tools lean heavily into reporting and this makes it very easy for them to crossover into data puke territory. At MineWhat we’ve included a few features that will help you avoid just this.
Viewing data in the right context is key to ensuring that you are always focussing on useful data. A spike in purchases on your store means little unless you know the story behind it, maybe there were a series of bulk purchases or the local league game caused a surge in traffic to your store. The event overlay feature on MineWhat lets you upload different events, for example a series of print ads you ran in your local daily. These events are then overlaid on a trendline so you can see how they affected your sales.
Human analysis on the data
The value of a good analyst is in the inferences they make on the data and the actions they recommend based on this. Avinash Kaushik sums this up well “Dashboards are not reports. Don’t data puke. Include insights. Include recommendations for actions. Include business impact.” On MineWhat users can add their comments and recommendations to the reports available on the app.
In eCommerce not all things are made equal, some categories might have a lot more products than others. As a result of this these categories will always appear at the top of most sales and traffic reports. Unless there’s some form of normalization on the data, you might end up not noticing interesting patterns like a smaller category having a higher conversion rate. In addition to normalization by conversion rate on MineWhat you can also normalize the data by the number of products in the group
Analytics, arguably began when American industrialist Frederick Winslow Taylor attempted to improve efficiency by running some time management analysis. Later Henry Ford, began collecting data to study and measure the pacing of the Model T assembly line. Since those days analytics has come a long long way. But there’s a steady trend one can map over the years: a quest for more detail, low level data if you will.
Each major shake up of the industry brings about a whole new crop of startups that promise metrics in more and more exhaustive detail. When Google acquired Urchin and released Google Analytics, the idea that one could find out which webpages users viewed and so on must have been every marketers idea of heaven. Since then, the quest for more low level data has been relentless. Every once in a while a new crop of startups pop up, offering detailed insights that Google Analytics doesn’t offer yet. We now have more than pageview based metrics, we have event analysis, user analysis and even in-page tracking of mouse movements. And all the firms involved are constantly blurring the lines between each of these.
Now where does the online retail business fit into all this? In addition to the increase in detail there’s been something else that’s quite noticeable in the industry: Verticalization. One rather recent and prominent example of this is the numerous unstructured data/text analysis tools for social media analysis. eCommerce analytics in itself is also fairly new (we count ourselves as among the first startups to get here, but i digress, that’s another story 🙂 ) here are some of our thoughts on the topic.
Why eCommerce Analytics?
Why indeed? Should all online stores just install GA, or some other web analytics tool and be done with it. Well of course not, most of these clickstream analytics tools are very good at giving a bird’s eye view of the proceedings.
Say I run an online store and I have to make a decision on how to spread my marketing budget across a few ad platforms(FB, Google, twitter). What web analytics will give me is, how much revenue each of these generate. While i can make a decision based on that alone, the decision would be quite uninformed because I don’t know the “why” of it all – did the facebook shoppers see something that didn’t appeal to their casual intentions?. I also don’t know what i need to do next – Which products should I display to FB shoppers?. Find the “why” behind your marketing data.
Another common approach among eCommerce analytics companies is to focus on the shopper itself. This works really well for marketing personalization. You could identify the highest paying shoppers (whales) on your store and understand their preferences to ensure they always come back. The problem with this is that it isn’t very scalable and the ROI will not justify the effort. Once you start getting a lot of traffic to your store, it’ll quickly become impossible to take care of each one individually. The way out of this is to track shoppers as cohorts or segments showing similar behaviour instead. For example “Shoppers who have an average order value of over $250”.
More metrics but at what cost
All this in-depth analysis is perfectly fine, but a rather worrying trend is that most tools achieve this by giving a very low level framework, which can be then customised by writing custom code. What this does is that it puts analytics almost completely in the hands of tech savvy analysts and IT folk. One thing that we’ve worked on constantly at MineWhat is that this doesn’t happen. An user can create a custom report tracking pretty much anything without having to write any code.
How far can a business intelligence tool go to convert insights into decisions? Typically the most that can be done is to give the user as much information as possible and then hope that it leads to some action. One way out of this is to take that last step and automate the actions themselves. Say for example if a product analytics tool that has picked out the right products for a landing page banner, can then give out the SKU ids via an API you can update the banner ad directly.
Do you folks think that an analytics tool can take over the decision making in this regard? Will it require human supervision or not? let us know in the comment section below.
I’ll leave you with this quote by the great Sir Arthur Conan Doyle – Data! Data! Data! I can’t make bricks without clay.