Database Deployability – #5 Source Business Intelligence from a Data Warehouse

This post was written by Matthew Skelton co-author of Team Guide to Software Operability.

Deployability is now a first-class concern for databases, and there are several technical choices (conscious and accidental) which band together to block the deployability of databases. Can we improve database deployability and enable true Continuous Delivery for our software systems? Of course we can, but first we have to see the problems. 

Until recently, the way in which software components (including databases) were deployed was not a primary consideration for most teams, but the rise of automated, programmable infrastructure, and powerful practices such as Continuous Delivery, has changed that. The ability to deploy rapidly, reliably, regularly, and repeatedly is now crucial for every aspect of our software.

This is part 5 of a 7-part series on removing blockers to Database Lifecycle Management that was initially published as an article on Simple Talk (and appearing here with permission):

  1. Minimize changes in Production
  2. Reduce accidental complexity
  3. Archive, distinguish, and split data
  4. Name things transparently
  5. Source Business Intelligence from a data warehouse
  6. Value more highly the need for change
  7. Avoid Production-only tooling and config where possible

The original article is Common database deployment blockers and Continuous Delivery headaches.

Opportunistic Business Intelligence data

While we’re at it, having a single, large, data-rich database sometimes results in the database being co-opted for Business Intelligence (BI) purposes: simply provide a BI person with a read-only database login, and let them write SQL directly against the database, right? This ‘opportunistic BI’ pattern naturally leads to strong resistance to database change from the BI team; their SQL queries are often hand-crafted and brittle, and database changes tend to break them. This direct use of application & integration databases for BI actively pushes against more frequent changes and deployments by arguing for stability-at-all-costs (albeit in a particularly narrow domain). BI folk also tend to have the ear of the finance department, which usually overrules IT requests for change. The result: a tendency to avoid changing and deploying the database.

Diagram5.1
Diagram 5.1 – All data is kept in one online data source which can be accessed not only by the application, but by BI people as well

Remedy: Source Business Intelligence from a data warehouse

We can reduce the friction around database changes and deployments by explicitly and consciously servicing the BI team. We should not just get BI ‘for free’ by aggregating data in the same transactional database, but design a proper system that provides the necessary data from multiple data sources, aggregated into a data warehouse, and accessed via a specific datamart. If there is a need for BI, address that properly, not just as a happy result of the data being conveniently in the same place.

Diagram5.2
Diagram 5.2 – Data is split into online data source and data warehouse. Application can access both databases, but BI can access only data warehouse

 

<< 5 >>

 

Database Lifecycle Management
Read more about Database Lifecycle Management in this eBook co-authored by Grant Fritchey and Matthew Skelton.

 

 

 

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: