Data visualization using Drupal 8 and Angular 5. A practical example.
This article is a thought piece about the potential of using Open Source web technologies for the purpose of data visualization. It uses a practical example to illustrate the potential of using three opensource technologies in tandem, Drupal 8, Angular 4 and the d3 data visualization library. You can see the prototype produced in the article here: https://www.theline.digital/chart. We are considering a technical guide as well so if anyone is interested please let us know in the comments. Anyone interested in the content of the article is welcome to engage with me via twitter and LinkedIn.
1. Introduction
Technology is moving at a faster pace than ever before, and value in today’s market can be obtained by using existing technologies in combination in innovative ways.
I am a veteran and long term supporter of Drupal, the leading opensource CMS for enterprise. I have spent the last 10 years working on high profile digital projects almost exclusively in Drupal. Many people will be looking at the subject of this article with curiosity. Drupal is branded as a heavyweight CMS for enterprise, how does that play nicely with data visualization?
At first glance, it shouldn’t. The PHP based rendering system, the latest iteration working in tandem with twig templates, makes the front-end clumsy when compared to more lightweight javascript technologies (such as Angular JS / React JS). Drupal’s data storage is mature and robust, but comes with an overhead making it less than ideal for the storage of large amounts of data. This is why at scale, as is the theme of this article, Drupal would probably decouple the data storage and use a built for purpose datasource. It is possible to implement the below example bypassing any data storage in Drupal.
For context on why we are excited about decoupling Drupal, we have to dig in to how Drupal is evolving, and in particular the context of Drupal as a headless API. The founder of Drupal Dries Buyteart writes about this subject on his blog: https://dri.es/reservoir-a-simple-way-to-decouple-drupal
The suggestion in the market is (and it is one that I subscribe to) that the future of a PHP based system to manage every aspect of large web applications is limited. One technology cannot do everything optimally, and a number of best in class components should be used in any modern technology stack.
There is no doubt that the future of digital is using multiple technologies in combination, each one being dedicated to a single function or component of the final application. That means a lightweight front-end framework for client side rendering, and an efficient manageable user-friendly API on the backend. This is where Drupal’s role in new digital projects changes from a conservative all encompassing CMS, to being at the very core of the backend, serving multiple components with a robust, configurable API.
To examine in detail how Drupal could be suitable as a decoupled API, we have to look into how the software is put together. Drupal 8 ships with a functionality that is the pride of the Drupal community, this is called ‘views’. This is the way in which Drupal queries and contextualizes data and has been the tool that Drupal Developers have used to create contextual, scaleable websites over the last 10 years. Views continues to be ‘best of class’, a data querying tool that is user friendly, massively extendable, scaleable and cacheable. Although Drupal CMS has always used this (by default) to render data via PHP templating, there is now the native capability to expose this power as a Web service. Views to a simple JSON data feed. This is Drupal as an API and it has incredible potential.
To demonstrate the potential I’m going to work through a theoretical example with a mockup that we have developed.
2. Practical example
Imagine that we have a client (Shoe Brand LTD) that wants to analyse client feedback about their new range of shoes. Shoe brand LTD draw up a feedback form and go to work. A few weeks later they have a pile of 200 questionnaires and want to really understand the data. Their brief is for a way of controlling, personalizing and visualizing their data. Their questionnaire has the following aspects:
- Gender
- Age
- Location
- Satisfaction with the brand (1–5)
- Satisfaction with product (1–5)
We are going to take this data and put it into Drupal, creating 200 users and 200 nodes of type questionnaire. All I have to do to prepare Drupal is using the UI (no code) add 3 fields to the users providing options for Gender, Age and Location, and two fields to a node type ‘questionnaire’ to define my results data.
In my data sample I’ve distributed the users evenly in terms of Gender, Age and Location and will randomly generate 200 questionnaire results. This whole process has taken me about 30 minutes.
We now have 200 questionnaires, but the exciting thing is a little hidden box, called relationships. What this means in laymans terms is that we can relate and map these results to the users (and their data). I’ll try and show the value of this detail and what I believe to be immense potential.
First, I load a standard Drupal HTML page based on this view.
In the HTML page you can see the results of the Drupal view. I can filter my data set by the different users, and if I wanted to I could incorporate PHP calculations and processes to create a form of interactive reporting, a common feature in many Drupal websites.
In the example I am only showing questionnaires submitted by users from location ‘A’, therefore we have at our fingertips all the information that we want to use for our initial analysis. The problem with this approach, is that the PHP rendering means that visualizing this data is not being done in the most intuitive way. It is possible to adapt the Drupal theme to make this better, however it would be still not be the most ‘natural’ way to consume the data. The data is there, but I would like to explore a different way to visualize it. So I’m going to take the data that Drupal is providing, and consume it with a dedicated front-end framework, in this case Angular 4.
So we just need a simple Angular 4 app for consuming and manipulating this data. Luckily, our corporate website is built in Angular 4. I’m just going to add a component using the excellent https://github.com/swimlane/ngx-charts module, this is an extension of angular for the native rendering of the d3 data visualization library. In their words:
ngx-charts is unique because we don’t merely wrap d3, nor any other chart engine for that matter. It is using Angular to render and animate the SVG elements with all of its binding and speed goodness, and uses d3 for the excellent math functions, scales, axis and shape generators, etc. By having Angular do all of the rendering it opens us up to endless possibilities the Angular platform provides such as AoT, Universal, etc.
So in my angular app I create a component and service to consume the Drupal API. I now have full access to the exposed JSON data being provided by Drupal. Next the fun bit, I install the NGX charts modules into angular 4, and use angular to retrieve the data from my Drupal 8 view. I now have all my Drupal data loaded instantly from a lightweight JSON.
Look at the difference in how I can interact and visualize my data. I have in a matter of hours, created and defined a visualization of the Shoe Companies questionnaire. The best part is that the every stage has been implemented using mature and scaleable technology, I have a Drupal API that is easily configurable by the end user, and lightweight HTML from Angular 4 to consume it. This means all the complicated processing and storage done by Drupal is adding absolutely no overhead to the final user.
Using Angular 4 with the NGX Charts component, means that I have native consumption of the data being provided by Drupal. The value that it adds is the look and feel of our final application, instead of twisting Drupal to our requirements, we are using the correct technology for every part of our application. This massively improves efficiency and quality, meaning that I’ve been able to easily create a product that just looks and feels ‘right’.
3. The power of relationships
Now that we are consuming this data, I want to explore the power of the Drupal 8 (given that I am championing it as the API of the future). I’m going to select ‘Location C’ in my Angular app. You can see that with no processing or page loading the data animates and changes and I am visualizing and comparing the data segment I’ve chosen. This has literally just made a http request to my Drupal view and passed through my filters, Drupal doesn’t blink an eyelid and the efficiency of the data transition Angular is impressive (all with native animations).
It looks and feels fantastic. All of the calculations are done instantly client side via Angular, this means there is no need for processing on the backend, and perhaps best of all, I have lightweight code and mature structures to quickly program the visualization of any kind of data. I am using opensource technologies with a simple UI to interact with and manipulate my data, and the potential and possibilities feel endless.
What we have built here in very little time, is a tool where we have defined and related two datasets using a native lightweight data visualization tool. There is so much we could do with every aspect of this simple example, we could use more complicated and useful data sets, we could use the whole suite of tools available to us for data visualization, we could theoretically add as many parallel data-sets for comparison as we want. This is just a taste of what is capable with mature open source tools already available for use by anybody. I hope that you find this combination of technologies as exciting as me, and can’t wait for people to come up with exciting applications for this technology.
About the author
If you enjoyed the article please contribute your grain of sand by sharing / clapping. User engagement is used by Medium to recommend content so if you like the content then giving a clap or share is an extremely helpful way to support me and allow me to spend time on further articles.
These are exciting times and I hope you take something away from my article, feel free to connect with me on my twitter and LinkedIn accounts.☺
It goes without saying that thoughts / opinions expressed in this article are my own, not associated with Chainfrog!
Further Reading:
Other articles I have written about similar subjects.
