Network Explorer: Data Pipeline and HTTP API

Worker(s): Christopher King Reward: 5 Stars WIP


network explorer screenshot


Build a public, open HTTP API for exploring metrics about the Urbit network.


The project known as the Network Explorer has been in the works at Tlon for quite some time. The idea is to build a publicly-accessible tool (hosted on that provides key metrics and insights into the state of the Urbit network. A series of high-fidelity designs have been produced.


This bounty covers the creation of the:

The Network Explorer interface will be covered in a separate bounty.


Data Model

Let's start with the data model. There are two sources of data that we need:

Most of the required information in the Network Explorer will come from Azimuth. It's important to know though that the source of PKI information will likely change in the near future in light of ongoing work to reduce the cost of bringing nodes onto the network.

The below entity-relationship diagram defines a data model in terms of a relational database:

erd diagram

Some notes about the above diagram:

Data Ingestion

Data from Azimuth and radar will need to be regularly extracted, parsed, and persisted to the local database for querying.

For Azimuth, using a tool like infura should be used.. Tlon currently uses infura, so it should be possible to use a free account for development and Tlon's pre-existing service in production.

The output of the radar script is saved to a JSON file that can be found here. This file is updated as new ping information is obtained, so data ingestion from this service should happen on some regular schedule.


The following are are necessary API endpoints for the Network Explorer UI to function properly. It's likely that some details have been missed, so apply your own judgement when designing the API.

This interface is meant to be agnostic to the API mechanism you choose, which could be either REST or GraphQL.


Produce the details for a single node.




Produce a list of nodes, optionally with parameters. Absence of parameters queries all nodes.


Returns An array of node in ascending order by point.


Produce the stream of PKI events for a specific Urbit ID, or globally in descending order of time.


Returns An array of pki_event in descending order of time.


Produce the network activity of a node or nodes.


If urbit_id is unspecified, implies all ships.

If since and until are unspecified, produces all activity for all time. If only since is provided, produces activity from since to the present. If only until is provided, produces activity for all of time up until until. If both are provided, queries activity within a range.

Returns Array of:

Technology Requirements/Guidelines

That which is not explicitly specified here is up to the choice of the implementer.




Data ETL

2 stars The database has been designed and data from both sources is ingested (extracted, transformed and loaded) on a regular schedule. Analysis can be performed at the database layer.


2 stars An HTTP API is written per the above specification and deployed to

UI Integration

1 star The Network Explorer UI will be a separate proposal, but integration with an interface inevitably surfaces oversights in the original design. An additional star will be awarded for providing ongoing maintenance as-needed during front-end integration.