Tag: application

The prototype’s architecture – revised

Ash on

“Trialling Wikibase for our data layer” described how we evaluated the use of Wikibase as a key implementation component in our bi-layer architecture. The conclusion was that Wikibase, although a brilliant product, does not fit our immediate purpose.

In our revised architecture…​

Wikibase is replaced with (dcs-easier-open-data) a simple set of data files (CSV and JSON) hosted in a public repository (GitHub). These data files are generated by the Waste Data Tool (dcs-wdt). Together, dcs-easier-open-data and dcs-wdt implement the architecture’s data layer.

In the architecture’s revised presentation layer, the webapp reads (CSV/JSON formatted) data from the dcs-easier-open-data respository, instead of reading (via SPARQL) data from the Wikibase.

The prototype’s bi-layered architecture - revised

A mock-up website for functionality & navigation

Ash on

Introduction

A prototype website will be one of the outcomes of this research project. The website should help non-experts discover, learn about and understand the open data about waste in Scotland.

To date, we have build a couple of mock-ups [1]:

  1. A functionality & navigation mock-up for exploring ideas about functionality and navigation for our eventual website.
  2. A look’n’feel mock-up for exploring looks/visual aesthetics.

This document concentrates on the functionality & navigation mock-up…​

The splash page of the functionality & navigation mock-up

Functionality

This mock-up ties together a lot of the elements we’ve been working on:

Data Direct access to download the underlying datasets.
A simple, consistent set of CSV and JSON files.
Maps Interactive, on-map depictions of the information from the datasets.
Data grids with graphs A tool for slicing’n’dicing the datasets and visualising the result as a graph.
To make this easier, this tool will provide useful slicing’n’dicing presets: starting points from which users can explore.
SPARQL A query interface to a semantic web representation of the datasets.
This is unlikely to be of use to our target audience, so we’ll probably remove it from the UI but may use its semantic graph internally.
Articles Themed articles and tutorials that are based on evidence from the datasets.
Uses Asciidoc mark-up to make the articles easy to format.
The articles may incorporate data visualisations that are backed by our datasets.

Navigation

The mock-up provides 3 routes to information:

Themes The clickable blocks on the splash page allows users to explore a waste theme by taking the user to a specific set of of articles and tutorials.
Navbar The menu bar at the top of each page, provides an orthogonal, more ‘functional’ classification of the website’s contents.
Search At present, this is a very basic text & tag search. In the future, a predicative/auto-suggestion search based on a semantic graph of the contents, will be provided.

Users navigation histories may help power a further-reading recommender subsystem.

Architecture

Building this mock-up has required some architectural decisions that may help inform the design of our eventual website.

Static website The mock-up has been implemented as a so-called ‘static website’. This means that page content is not dynamically generated by (or saved to) the server-side. The server-side simply serves ‘static content files’.

Pros Implementation-wise, it is an order of magnitude simpler and more scalable than a ‘dynamic’ website.
There are several good, free, open source ‘static website generators/frameworks’.
Static websites can be served for free on hosting platforms such as GitHub (as used for this mock-up).
Cons It can’t support a whole class of functionality, including user uploads, and on-line content editing.
Computation is forced towards the client-side (i.e. into users’ web browsers) which sometimes can have a negative impact on the speed of the UI.
Off-line updates The content of the website can be updated – just not updated on-line. The website maintainers can add new/edit existing datasets, articles, etc. via off-line means.
For off-line updates to this mock-up we use: (i) WDT – a rough’n’ready software script that helps us to curate the datasets that underlay this mock-up; (ii) Cryogen – a static website generator; (iii) Git – to upload updates to our GitHub hosting service.
Client-side computation Page content is dynamically manipulated (e.g. datasets are slice’n’diced) on the client-side (in users’ web browsers) using JavaScript. This enables, for example, the mock-up’s web pages to take the static content that is served by the server-side, and manipulate it so that it can support interactive data visualisations.
Progress in client-side technology even makes it possible to implement a semantic graph supporting triple store in a web browser!

Conclusion

This mock-up website…​

  • provides concrete test-bed for evolving the functionality & navigation aspects of our eventual website, and
  • forces us to think about architectural trade-offs.
1. We use the term “mock-up” to mean an incomplete representation/model – useful for demonstration, design evaluation and acquiring user feedback.

A prototype data grid & graph over data about waste

Ash on

The interactive data grid with a linked graph is a tool that is often used to aggregate, dissect, explore, compare & visualise datasets. Might such a tool help our users explore and understand open data about waste? To help answer this, I have hacked together a web-based prototype…​

The working prototype

The working prototype can be accessed via this link.

The data

The prototype pulls together 4 datasets:

  1. “Generation and Management of Household Waste” (SEPA).
  2. “Carbon footprint [CO2e]” (SEPA)..
  3. “Population Estimates (Current Geographic Boundaries)” (NRS).
  4. “Mid-Year Household Estimates” (NRS).

The datasets are fetched from statistics.gov.scot and Wikidata, using SPARQL; then matched; and finally, the per-citizen and per-household values are calculated.

The result is 17,490 data records.

The build

The data was assembled using this executable Jupyter notebook. For a production-class implementation, that could easily be coded as automated, periodic process.

The web app containing the interactive data grid with a linked graph, was built using the DevExtreme web component library. Alternative libraries are viable, but the DevExtreme one is modern and free for non-commercial use.

The resulting data assembly and web app are stored as static files in the project’s GitHub repositories.

Its features

The prototype’s web page contains a graph and a configurable data grid. The graph automatically reflects the data selected in the data grid.

700

Detailed information about a graph’s data point is shown when the user hovers over it with the cursor.

screenshot graph hover

The graph can be zoomed/unzoomed, and its current contents can be printed or saved as PNG, PDF, etc.

screenshot graph saving

The data grid’s expand/collapse arrow-head icons allow the user to drilldown into slices of data. Below, we’ve expanded the Stirling → Recycled slice to reveal the data values per-material.

screenshot drilldown

The data grid’s “Show Filed Chooser” icon pops up a control panel to allow the user to select data dimensions, axis assignments, value ranges, value filters, display order, etc., etc.

50
screenshot field chooser panel

The data grid’s “Export to Excel file” icon will export the grid’s currently selected data to an Excel spreadsheet.

50

The resulting Excel files are nice because the export functionality preserves user-friendly fixed headers and some other formatting.

700

Finally, the prototype operates well on phones and tablets (although there is a sizing issue with pop-up panels that I haven’t investigated).

200
400

But, is it useful?

So, might (a production-class version of) such a tool, help our users to explore and understand open data about waste? Well, we won’t know until we have user tested it, but my guess is that:

  1. users with no data analysis experience will find its configurability difficult to navigate.
  2. users with low-to-medium data analysis experience may find it a useful as a single tool containing multiple datasets.
  3. users with medium-to-high data analysis experience will prefer to use their own tools.

A presets feature has been added to the tool so that users can go to a particular configuration & data selection by simply clicking on a hyperlink. This supports an easy-access route to the tool for users with no data analysis experience, by answering their potential questions through presets such as:

Mocking-up features in a placeholder WCS web application

Ash on

The narratives in Anna and Hannah’s “Scenarios” document, tantalise with mentions of the features supported by their fictional Waste Commons Scotland (WCS) web application. This week, mocked versions of some of those features have been added to the placeholder WCS web application (source code) – with the idea that their animation will make the features easier to understand and assess.