Epic Story of Waterfall to Agile
A Sprint to Agile Story
How we went from waterfall to agile.
The technical industry is filled with buzzwords and 3 letter acronyms. Agile is one of those buzzwords. I looked at agile in 2011 because all the cool kids were doing it. First off, agile is a philosophy. Agile has several cool sounding methodologies like, Scrum, Lean, Kanban, Crystal (OK not so cool), Dynamic Systems Development Method (DSDM), Feature-Driven Development (FDD) or my favorite sounding one... Extreme Programming (XP)! Before you are done reading this, there will probably be a new one. Agile is not only for software development. Companies like to say they are agile. I often wonder in which areas they apply it and what methods they use. Being the buzzword it is, using it for the mere purpose to attract new customers is common.
I have a favorite local diner I like to eat at that has been in business since 1957. The owner told me that business was slow and he didn’t know how to attract new customers and younger wait staff. I came up with an ad campaign for him.
“Joe’s Diner. Agile since 1957”
“All food prepared using Agile Lean”
Isn’t this catchy? Wouldn’t you try it at least once? Sorry, the story is not real so don’t ask me for the address.
We rejected agile in 2011 but fully embraced it in 2016. How it happened for us starts in an era of big hair and the best music ever created.
<< Rewind to the 1980s
My personal story begins as a programmer in 1985 in a small company called Konami. For those not familiar with Konami, it is a video game company based in Japan. It was a great honor to be the 1st American programmer ever hired by Konami to work in R&D and still holds true today. There was only one challenge this opportunity presented to me. I was part of a Japanese-only-speaking team of programmers and artists with all documentation written in Japanese.
My role was solely that of a programmer. I had very little, if any, input on feature, assignments, quoting or scheduling. I was handed a task in Japanese, given some direction in broken English and off I went to create code. Thinking back on this, the only real common language we shared was 6809, 68000 and z80 assembly.
I would go for days with little or no sleep to meet a deadline for a vision I could not even read. My work schedule was 7 days a week, minimum 16 hours a day, for 7 to 9 months in a row.
As crazy as that may sound, I learned a lot about the culture and adopted some of the work ethics that stayed with me to this day. I still encounter moments where peoples’ eyes light up talking about the arcade games I worked on and the hours they spent playing with brothers or friends.
Jump to the 1990s
After Konami, I worked for several other video game companies throughout the 90’s and became, what I would call, the assistant (or for you Linux people sudo) project manager.
What I mean by this is that I was required to formulate the project tasks, apply hours to each task, assign the tasks, set milestones and lay it out in Microsoft Project or on a spreadsheet. I did that for the full team of programmers and artists. After handing off the project plan and control to the owners of the company, my role immediately switched to lead programmer.
From there, the features creep (ongoing expansion or addition of new features not defined in the original quote) would begin between the owners and the client. New features would be added, tasks reassigned and people moved from one project and assigned to another with no replacement, chiseling away on the sculpture I had worked so hard on to build.
But one thing on the schedule did not change.
Can you guess the one thing?
If the thought of delivery date came to mind, you are correct! In the video game industry, the magical delivery date cannot be missed or moved to a later date. It’s the same deadline every year and happens to be Black Friday.
If Mom or Dad didn’t find the game the child asked for on the shelf in the stores on Black Friday, it meant a missed opportunity for skyrocketing sales. I know things have changed since the 90’s. Christmas starts in October, companies miss the turkey delivery deadline, but this is not how it worked in my day.
>> Fastforward to the 2000s
Up to 2003, still working in the video game industry, I began looking for an exit strategy. I felt it was time to move on from video games into the corporate world. My goal was to slow down, take on different challenges and learn something new. And boy did I ever! HIPAA compliance laws were coming into effect and insurance companies were required to accept electronic claims to help cut down on cost. I thought “This is a good place to start!”… WRONG!
Insurance and financial companies are slow, large and likely to reject change. I do not fault them on the slowness of change, but do on the willingness. The facts are, they provide a service to hundreds of thousands or even millions of people. One mistake and the company and customers can suffer a loss or inconveniences.
I got what I asked for and didn’t like it one bit. On the project management side, the problems switched from a chaos style with too much to do by delivery date, unrealistic goals, features creep to improve the product, backed by a team of programmers and artists working 23 hours a day to meet a deadline
legal and corporate red tape, clients not adhering to schedule testing, missed deadlines, programmers and employees working from 9:00 am to 4:59:59 pm with exactly 1 hour for lunch, non-existent willingness to adopt new technology, hours-long meetings a day that seemed more like a social gathering. Features/change requests had to be approved by a committee which could take months to sign off. In short, the pace was too slow and responsibilities to few for me.
2005 - Starting the SAP Business One chapter
I found a company that was looking for a .net programmer for SAP Business One development. The role was somewhere between video game chaos project management/programming and corporate boring project management/programming.
This was actually what I was looking for! From the project management perspective, I had more control over the projects, but not the level needed. Upper management would poke their heads in every few weeks and adjust priorities. The ripple effect forever changed the course of the projects.
2009 - KB1Pro is born
In 2009, I started KB1Pro to continue the journey of SAP Business One development. Now was my chance to put into action the things I had learned. The company model was to provide our services directly to SAP Partners.
One of the services we wanted to provide was direct access to the project status. This was at the top of our list as it would add transparency while creating a more active participation role of the SAP Partner. We looked at several web based applications and chose Redmine. Although it didn’t have all the functionality important to us, it was close enough.
From 2009 until 2010, we added functionality to allow transparency and promote Partner engagement. SAP Partners had access to schedule, milestones, tasks, Gantt charts, hours, bugs, PDF invoices for Partner with KB1Pro’s billable rate, PDF invoices for customer with Partner billable rate, access to project documents, access to download each add-on version, support tickets and more.
If you are a technical person like me, you are probably saying “COOL!!! How can I get this?” At one point we had 9 SAP Partners with access to the system. Of the 9, however, only one actively used the system saying it was outstanding. The other 8 Partners would use it very sparsely, mostly after a phone call and KB1Pro directing them to the website for the information or download.
The customization we created for Redmine also came at a cost. With each Redmine update, there was a risk in breaking the custom code. Over the years, we spent hundreds of hours supporting a system that seemed to have little value to our Partners. The
Real world conclusion
My career experiences and my attempt to create a system for our partners to help make their lives easier clearly came up short, forcing me to take a step back and ask myself a series of questions.
Why were non-technical people consistently messing up the projects?
The schedule was created, milestones set and the tasks assigned. Couldn’t they comprehend a project was like a large ship that had set sail? The weather alone was already a huge factor that we weren’t able to predict. Now we had to contend with ship passengers coming in spinning the ship’s wheel? Each and every change off course required new calculations and delayed the arrival date, or worse, left us stranded on an uncharted island.
Was it possible to remove the non-technical people from the project after start?
No, but if we didn’t have non-technical crew members changing the scope of a project, who could we blame for the countless tasks we guessed at 1 hour turning into 40 hours? Non-technical members have a reputation for making project changes. They easily become the excuse for missed milestones and deadlines while their job is to help improve the outcome to give customers what they want. For that reason, they need to be part of and throughout the entire process.
“You can lead a horse to water, but you can't make it drink”.
Partner participation is essential in creating a product that meets the needs of the customer and we believed we had found the answer in Redmine. So what did go wrong? Why was only one Partner using the system? Didn’t the rest of the Partners care?
I’ll get to the answer in just a moment, but before I do that, here’s a hint: The Partner that described the system as “outstanding” was technical, the other 8 non-technical.
To continue the ship references, this is were I “missed the boat”. I was feeding technical data to non-technical individuals. What did I expect? I started replaying my past experiences in my head, writing down what I recalled. Revisiting a few days later with more memory recalls leading to an eye opening moment.
Perhaps the approach to project management was the issue? Maybe using a large ship with the limited ability to maneuver was the problem?
Even though I had reviewed the agile methodology many years ago, I rejected it because I had a hard time relating to the concepts. If you spend any time reading about it and talking to people that use agile, you would understand the confusion.
With waterfall, you collect ALL the features up front, break it into tasks, guess the hours, link dependencies and add it to a milestone. If you work on a project that will have no changes, this model works great and should be used. However, in reviewing my past projects, change happened every time. I quickly realized that I had to correct my approach to factor in the inevitable. What could I do to make room for change? And how could I correct my approach?
Agile to the rescue
To create better products, joining the technical with the non-technical members became our goal. Removing the friction between the two crews allowed flexibility in a project and in turn better products.
How can Agile do that?
Unlike waterfall, where a project has a start date and a finish date, agile allows for multiple check points to gauge features, functions and usability throughout the project. After each check point, changes can be made, stories/tasks can be rearranged, features added or removed.
You may be saying “Waterfall has check points, they call them milestones.” That is true and a great place to validate it performs as designed. But again, by its very nature and name, it is no place for making changes.
Stories replace Gantt charts
I am not an artist, but I have created gantt charts. Hundreds of pieces of paper carefully taped together and hung on a wall. Art is subjective, its beauty is in the eye of the beholder.
A technical person viewed the gantt chart with amazement, examining each line and task. A non-technical person viewed it with confusion. When I think of the time and energy it took to create a masterpiece, it is disheartening how quickly it lost its worth.
If the goal was for, both, technical and non-technical individuals to see the art, a different form of light was needed to highlight its value without altering the artist’s vision.
Replacing the gantt chart with stories allows all participants in the project to clearly grasp the intent and objective of the project. No more technical jargon, but a book of collective short stories everyone can understand.
2015 – Search for Agile software begins:
We started the search by first defining the requirements and objectives we needed in the project management software.
- Web based to allow access to project information
- Flexible workflow per project type
- Product Version Release downloads
- Support ticket submission and tracking
- Change Request submission
- Bug submission and tracking
- Document storage
- Rest API interface
- Easy to navigate website interface
- Data display in graphical chart format for a crystal clear overview
Seven products met our requirements. After reviewing and testing each one for 7 to 14 days, the winner by a landslide was Tuleap. It quickly became our top choice that all other had to measure up to.
With that, the story doesn’t end. It’s just the beginning of endless possibilities.
After using Tuleap, a vision of support ticket, change request and bug submissions over a secure and direct connection from SAP Business One to Tuleap emerged. The idea is to allow SAP Business One end users the ability to submit requests to Tuleap while never leaving the SAP Business One GUI environment to
1) Eliminate the need to go outside of SAP Business One to additional websites and URLs
2) Improve business partner/client relationships
We feel it is going to be a long time before we outgrow Tuleap.
Departing words of wisdom:
Evaluate your process, see if there is room for improvement. Re-examine things you might have rejected as a fad. It might be in style (just like those skinny leather ties in the 80s).
OK, maybe a bad example.
Was this blog helpful? Share your thoughts with us.