Tuesday, August 18, 2020

Agile Methods Over Time pt1, to the eXtreme

Is Agile Dead? Does this even matter in 2020? What kind of person writes a manifesto?

For the sake of pragmatic memory management, I'd like to have a TL;DR, but what lead the leads who developed the Unified Process before (and several others) to meet and put out the Agile Manifesto is inherently important as iterative methods are delta-driven, requiring one to know what about their current position should carry forward and what should be removed and/or replaced. That knowledge can not be perfect, so experimentation, agility, and a solid foundation in measures which are based on the product of the work, not the work for work itself.

Timeline

from https://www.researchgate.net/publication/275654650_COMBINING_LEAN_THINKING_AND_AGILE_SOFTWARE_DEVELOPMENT_How_do_software-intensive_companies_use_them_in_practice [1]


Methods Along a Path

While my career is not normal, it does mirror a class of developers who have had the persistence and care for the field to not seek early exits or hone their skill in the way of the shiny object at the expense of the clients' outcomes nor measure their own personal 10x without recognizing the value of a team. The key process change Agile methods milestones I experienced with some local color follow:

Hobbyist

On seeing my uncle run a swarming bees near runways model on a computer that occupied the length of a bedroom and requiring the heat to be exhausted, then near after doing many odd jobs to earn enough to buy a personal computer that could fit on a desk, monitor and all, I had found my first financier and customer of software developed, myself. My uncle did not fully appreciate that he inspired me to learn computing, so did not mentor me, but he did connect me with resources including magazines covering this personal computing wave as well as manuals for C, Pascal, and BASIC. At the time, constructing memory models for the 40 keywords of C and graphs of how they could be strung together to form logical and procedural constructs were unconscious acts. Compiling a Pascal program of relatively simple complexity, such as reducing all of the permutations of eight queens placed on a chess board to list only those for which all queens were safe, took twenty-four hours to compile, give or take an accidental power-cord pull. And BASIC provided the much more proximal feedback, though required manual line numbering and GOTOs of various forms. Each experience was repeated with exuberance and each with significant difficulty at first and each difficulty was minimized along an apparent course from the invisible unknown to the approximately rendered visible, known. And my customer was over joyed most of the time.  And when not, my customer told me exactly what I should have done better. And I was eager to please my customer, so I dove right back in to the keywords, well actually not. I drew out pictures of the solution as my customer stated it should look like. I drew out key components of the solution and how they should interact. I listed out sequences and forks in the sequence based on key variables. And then I dove right back in to fix what I had partially succeeded in delivering. With hopes that my customer would be happy with this next development, yet knowing that there should always be the next "could be better".

Waterfall

After doing here-and-there contract work through university, then owning/operating a small data backup solution for local businesses, when I encountered the contract-oriented nature of Waterfall in my first full-time job as a software developer, I found the meeting of deadlines at the expense of maintainability, passive-aggressive needling over details which were in the contract or not or to whatever not-so-specific level they were in the contract, and wait aren't we in the same company why is there a contract?, whether someone will help or not because they are too busy/behind on their obligations, and generally getting past all of this and remembering that we are in an infinite game with our team and our customers, so getting ahead by working harder, worker smarter, but also working together. There weren't many books or blogs...haha the internet was nascent and my use was more reading RFCs and learning the processes of obtaining the necessities to launch an eCommerce business and how to adjust to the search engines emerging algorithms.

Spiral Model

On my second big project that I lead w/i the Waterfall first job, a contractor who had had eight successful stints at the company (each stint requiring a year between to encourage conversion to full-time for such good contractors, yet most remained in the more highly lucrative contractor status) determined to mentor me in the Microsoft version of a marked improvement on the Waterfall model. In the Microsoft form and presentation of the the Spiral Model, the process was likened to and was highly cohesive to the Rapid Application Development tooling. This continuous improvement model was also being studied and touted in academia, ie the Capability Maturity Model from Carnegie Mellon University. While it was a bit of marketing to not only lockstep all of the Visual Studio tool versions to 6.0 (ASP was previously 1.0) but also to tie the software to a specific methodology, there is and was a lot of merit to reducing cognitive load for developers, putting up an easy to grok interface while tweaking and tuning performance in the framework, and making previously enterprise-only operability features of the OS and web services (IIS, not SOAP and later REST) into the developer mainstream, all leading to actual rapid milestone deliveries from prototype through to first full delivery and on to year-over-year updates and in some cases, the next Agile increment, continuous engagement w/ the business unit.

eXtreme Programming

Concurrently emerging as documentation and other up-front quality and maintainability artifacts that were requirements/byproducts of successful Waterfall were waning was a call for technical and business subject matter experts to step up their personal processes and contribute to team processes while also taking their apparent skills and ability to speak, understand, and deliver software solutions that operate in the domain of the business as agents of the business. In hindsight, it is no wonder why they called this eXtreme Programming. There are always aspects of the solution that we need to develop that are not known, some requiring skills that we must learn or teach others in the delivery team rather rapidly, so being "in the fish bowl" where the developer's management and the developer's business customer is awaiting deliverables and can witness little to no output during these capital development efforts, character is required and XP is gained. At the time that I faced this shift, I was working under the lead of the co-author of Use Cases: Requirements in Context (slightly lesser known than Use Cases, but no less useful). Where I had previously lauded the gods of software from Kernighan and Ritchie and Stroustrup through to the Rational Unified Process / UML and "gang of four" design patterns, I was now working with such a titan and he was just like me, attempting to help the business and help others that are doing likewise. Much was learned, but much was unlearned. Where we had in the Waterfall praised the artifacts, we in eXtreme Programming were including the artifacts in the early and often deliverables to help ensure that the entirety of the business understood the business in its current and next increment states. Quite a number of merit-based human-resource strategies and mechanics helped render those who achieved to further refine the game in their favor. Some of us were far outperforming our peers, so much so that many within a team and in some cases entire teams were clearly lost in the wake. There were also various eXtreme Piles of spaghetti that made for some juicy contract work to untangle and refactor and redux entirely on more solid architecture than the poor souls who were hurried under such watchful eyes without such a foundational layer.

To be continued in pt2, from eXtreme to Kanban, aka "Where is that TPS Report?"

[1] Rodríguez, Pilar. (2013). COMBINING LEAN THINKING AND AGILE SOFTWARE DEVELOPMENT How do software-intensive companies use them in practice?.