The Big Guess Up Front (BGUF)
The way we used to deliver Business Intelligence projects was to do what I call the “big guess up front” (BGUF).
Documents for Africa
We would create massive Business Intelligence Strategy documents, which took months to create and get “signed off” at a senior level in the organisation, often by the Board. This strategy document then became set in stone, and it could never change, even if every assumption it was based upon was found to be invalid or had changed.
We would define a project plan early with a standard list of tasks and dependencies based on other projects we had estimated, and then guess the effort required to deliver these tasks and the milestones dates we might be able to achieve. These got signed off as the official plan before we ever gathered the details we needed to see if these estimates were reasonable or achievable.
We used to spend months acquiring and documenting requirements, which became a massive wishlist of every piece of data, every BI tool feature and function the users could think of as it was their one chance to get this capability delivered by a project.
We would spend months standing at whiteboards and stuck gathered around screens with Microsoft Visio defining a single enterprise data warehouse model to rule them all. Or worse leave it to a single data architect to huddle over their screen on their own for weeks, weaving their modelling magic.
The testers would build test plans in infinite detail, with spreadsheets of the hundreds if not thousands of the detailed test they would run to prove this thing we would build would match the things we wrote down.
Intro the change management process
And of course, this was all based on the assumption nothing would change, or if it did change a “change request” would be raised and the “change process” would ensure all the documents and all the people would change in fluid unison. It never worked.
So at some stage, problems were escalated and a “Change Manager” would be added to the team. The Change Manager was not there to help the users transition through the changes that were required to successfully adopt this new capability. This unfortunate person was there to “manage” the changes within the project and would be akin to either the plumber or a police officer managing traffic at a roundabout.
If somehow the Change Managers role was deemed to involve fixing issues when they are raised, I liken their role to that of a plumber. Anybody that had an issue would handball it to the change manager and expect them to resolve it. If there was something messy to deal with, that you didn’t want to deal with yourself, then you called in the ‘plumber’. Of course, the Change Manager was normally just a team of one with minimal authority and budget to make things happen. Eventually, the Change Manager would become swamped with the raft of issues they now had to deal with, causing a massive blockage.
If the Change Managers role was defined as being the channel that managed any change issues but wasn’t responsible for actually resolving them, then I liken the role to that of a police officer directing traffic at a roundabout. If you had an issue, you couldn’t (or wouldn’t) resolve you put it into the roundabout, and the Change Manager made sure it got off the roundabout at the stop of the person or team who could resolve it. In this scenario you often also made sure that you blocked your exit lane on the roundabout as effectively as possible, to ensure the Change Manager couldn’t offload any new issues to you or your team. You did everything you could to stop these problems being allocated to your team due to your team being flat out trying to deliver to the scope in the estimated timeframe. Eventually, the roundabout would become clogged, and there would be a moratorium on any new issues. As issues still eventuate even when a moratorium for managing them exists, there was often a concept of a “parking lot” where they were placed, so they were supposedly visible but with no chance of them ever getting cleared.
Of course, all this change is bound with documents, templates and processes that are meant to streamline the management of this change, but in fact often makes it too onerous to raise any change issues at all. At some stage, a “Change Administrator” would be added to the team to manage the process, track and report the current status of each change.
In some organisations, I have seen projects terminated when they ran out of time and money, based on the initial project plan BGUF, regardless if the core tasks are completed or if the outputs had successfully passed full testing.
And the project was called a success.
The project closure left a massive amount of work for the Business as Usual (BAU) team to complete as soon as the environment went live. These BAU teams were not given additional funding or resources to undertake this work, in fact, often there was an expectation the team headcount would be reduced based on the efficiencies this new system promised to deliver.
These BAU teams then had to struggle to manage the raft of calls they got from the introduction of a new system, as well as find the time to finish the tasks left over from the now closed project, all the while maintaining their previous day jobs
It’s not a team sport
One of the major outcomes of these big guesses upfront are a lack of accountability for the team that receives them. They are handed documents and designs that they didn’t create and told to implement them. The context, discussions and insights that went into these artifacts have been lost as soon as they were written down and handed over, making their task difficult if not impossible.
Also, the change management process removes their accountability for resolving any issues that they identify. Any issues that are raised are typically not in their sphere of control to fix.
And this is why it fails, it’s a bunch of individuals or teams working in isolation.
I have encountered project plans that were created by project managers who had never worked on a business intelligence or data warehousing project before.
They would use a previous plan as the basis for the new development with no multiplier for the level of complexity difference between the two projects. For example, in one organisation the project manager scoped out the estimate to add a new source system to the current data warehouse.
Unfortunately, the new source system had some large text columns that needed to be extensively parsed to meet the business requirements, the previous source system on which the estimates were based did not. The development team was not involved in the estimates, and the budget was locked in based on the initial plan.
The team was setup to fail.
It’s not the people it’s the approach
The problem is not that the Change Manager is incapable, project managers are trying not to scope properly, or the delivery teams don’t want to be accountable for success, the problem is we are doing big guesses upfront. And this causes some issues:
- We create strategies, documents, designs, processes and roles that cannot easily be changed when required
- The documents are based on a large number of assumptions (guesses) as there is a raft of things we don’t know yet
- We spend far too much time creating these when we could be using that time and resource to deliver value upfront
- We communicate via documents not ongoing conversations
- We restrict the teams to deliver small chunks of the entire outcome
- We start with a culture that change is bad and should be avoided at all cost
- We somehow believe that creating these big guesses upfront reduce risks during the delivery phase, when in fact it increases them.
AgileBI is an approach which is all about providing value continuously, collaborating on the right thing at the right time, and ensuring we can manage change at anytime. Big guesses up front make this very very difficult.
Start your AgileBI journey with us.
Gartner 2020 Magic Quadrant for Analytics and Business Intelligence Platforms is out
Its that time of year again, New Year has been and gone, Christmas is but a memory, in half the world Winter is mid flow and in the other half (including my half) are mid summer wearing shorts and jandles.As well as good friends, good barbecues and great...
Gartner 2019 Magic Quadrant for Analytics and Business Intelligence Platforms is out
It's that time of the year (where the hell did 12 months go!) where Gartner announce the latest version of their Magic Quadrant for Business Intelligence tools, or this year what they call "Modern Analytics and BI Platforms".Im always interested in which...
Adding members to the Agile Team makes you go slower
As a Stakeholder or Product Owner I want I want to understand if constantly adding Agile Team members is a good idea So that I can have a valid conversation with my Product Owner and Scrum Master Initially at least! VelocityWhen building a new AgileBI delivery team,...
Gartner Magic Quadrant for Business Intelligence 2018 – Augmented Capability will be the New Black
February has come around again, the Gartner BI conference is running in Sydney Australia, people I know are in Sydney drinking beer, I am not, and the Gartner Magic Quadrant for Analytics and Business Intelligence Platform is out.Can I take a little space to rant (of...
Who should attend Steering Committees?
As a Stakeholder or Product Owner
I want to understand who should attend the steering committee
So that I know I have the best representation
Project Managers who have done standups
As a Stakeholder or Product Owner
I want to understand what experience a Project Manager needs
So that I know if they are suitable as a scrum master
3. AgileBI Things: Agile Data Science, Governed Data Lake, The Agile Data Warehouse A Practical Approach
Source: Pixabay Shane writes an AgileBI series called "3 AgileBI Things" published on LinkedIN Pulse. This article below is a copy of "3. AgileBI Things - 2017-04-09". Shane also writes on AgileBI concepts at AgileBI Guru. 1. Agile Data Science When the new world of...
3. AgileBI Things: Big Data Papers; V, V, V v,v,v,v,v,v,v; Extended Data Warehouse
Source: Mosman Library Shane writes an AgileBI series called "3 AgileBI Things" published on LinkedIN Pulse. This article below is a copy of "3. AgileBI Things - 2017-04-02". Shane also writes on AgileBI concepts at AgileBI Guru. 1. Big Data Papers I still hear...
3. AgileBI Things: Automated Testing Goals, Data Warehouse Test Checklist, Business Rule Game
Shane writes an AgileBI series called "3 AgileBI Things" published on LinkedIN Pulse. This article below is a copy of "3. AgileBI Things - 2017-03-26". Shane also writes on AgileBI concepts at AgileBI Guru. 1. Automated Testing Goals So I have finally been lucky...
3. AgileBI Things: Big Model Upfront, Data Warehouse Project Failures, Guestimates
Shane writes an AgileBI series called "3 AgileBI Things" published on LinkedIN Pulse. This article below is a copy of "3. AgileBI Things - 2017-03-19". Shane also writes on AgileBI concepts at AgileBI Guru. 1. Big Model Upfront When delivering an AgileBI project one...
3. AgileBI Things – SQL Server Temporal tables, Changing Data, ODE
Shane writes an AgileBI series called "3 AgileBI Things" published on LinkedIn Pulse. This article below is a copy of "AgileBI Things - 2017-03-12". Shane also writes on AgileBI concepts at AgileBI Guru. Image source: Pixabay 1. SQL Server Temporal tables, automating...
3. AgileBI Things – Data Lakes vs Data Warehouses, Change Management in an Agile approach, So you wanna be a Scrum Master
Shane writes an AgileBI series called "3 AgileBI Things" published on LinkedIN Pulse. This article below is a copy of "3. AgileBI Things - 2017-02-19". Shane also writes on AgileBI concepts at AgileBI Guru. 1. Data Lakes vs Data Warehouses I think in the next few...