New plan for system design

 I recently began a project to redesign and refactor a legacy data storage and processing system. This project has some serious challenges, one being that the legacy system and backend storage is based on technology that is 20+ years old (customer use Emacs to access and operate). As the engineering team gathering requirements we find ourselves soliciting the customers for more and information to provide clarification on system requirements.


Our challenge is that technology has come so far in the last twenty years that the customers have the ability to describe what they currently have, but they have no ability to describe and reconcile what they could have. In meetings the Systems Engineers (SE's), draft notional requirements we think would be useful and the customers shrug their shoulders and say things like "I guess that would be cool". The team finds themselves fighting about what is needed vs. what is extra at every Architecture Review Board (ARB) meeting. The funny thing is that most of the debating and fighting is centered on disagreements between the development and systems engineering team. The group is always going back and forth between trying to ensure we've captured the functionality of the current operational system while designing the new one.

Every time I capture a notional view of the systems functionality it receives a 30 day back and forth virtual and physical review before I'm informed that my design is premature because we haven't decided on the systems node or this message format. I finally became fully frustrated last week as I was shot down in the ARB when after several minutes of bantering I suggested rather than continuing to guess what the rest of the system looks like (we were discussing the repository node), we finally come to an agreement on what a physical view of the system is - one that attaches specific software products to the actual design. I mean six friggin months later and one actual software baseline release and the customer isn't ready to finalize the physical design. Often this is because the customer can't reach a decision with the white noise from the team constantly buzzing around (well we could do that, and maybe we try this new product instead, or do we think about this yet)... never-ending.

This led me to devising a strategy that basically forces all the SE's and developers to reach a consensus and plan on what the systems is and how it will be designed and integrated. Agree on all the derived artifacts detailing the design and then as a group submit to the customer for review. The customer willingly admitted that it's often difficult to make a decision with so many different opinions and suggestions’ coming at him from different directions. He agreed we need to bring a unified plan for him to adjudicate and approve and that even including him in the final stages of design planning would probably be sufficient to mitigate the lengthy delays. Hopefully this works and we can finalize the system design prior to releasing the 2.0 baseline.