Flow, Flex, and Ship:

The Daily Practice of Agile Engineering

UCSF Education IT

Team Hosts, and Develops, supports, and configures Moodle, Kaltura, Ilios, TurnItIn, PollEverywhere, and other edtech tools

Ilios

Open Source Curriculum Management System
100% remote team
Up to date on the latest versions of PHP, Symfony, Ember, and a huge number of dependencies
In use at health science schools around the world
Highlights of the work that won us a Sautter Gold Award:
Rave Reviews from Students
Focus on accessibility
- Nearly perfect siteimprove accessibility score
- Manual a11y tester called the webapp the best of its kind that they had ever tested
- Code splitting, lazy loading, and data optimization to ensure support on very low powered devices
JSON Web Token (JWT) app-specific B2B authentication token system
GraphQL API (in addition to our existing JSON:API implementation)
Container Based Cloud Native AWS Fargate Deployment (which is an awesome service, I highly recommend it)

Story Time

A few years ago I stepped in a very small hole and wrecked my back
It wasn't a long way down, but the change was dramatic enough that I seized up completely
I was in a lot of pain
and needed help to walk for a couple of weeks
On the advice of like 50 different people I took up yoga to help with my back
It was Hard
Kept doing it, and it's still hard
I probably would have quit, but my wife bullied me into showing up most days
And then I had a bit of a revelation when I tripped again, but this time I didn't hurt myself
I finally internalized what I'd been hearing for a long time.
Practicing this difficult, painful thing was making me better at it

What does this have to do with agile engineering?

Last year my entire team moved from the UCSF Library to IT
It was a huge change
Different from falling in a hole, but all change is traumatic
The years we'd spent building agility in our team paid off in a huge way
We were able to adapt to the new environment
We were able to keep shipping
Our work pace slowed down a lot, but we kept moving forward
This talk is about building agility for our projects, teams, and ourselves in the same way
It's hard, but it's worth it!
Because the only constant is change

Let's set the stage

Programming is Hard!

The computer resembles the magic of legend… if one of the incantation is not strictly in proper form, the magic doesn't work. Human beings are not accustomed to being perfect, and few areas of human activity demand it. Adjusting to the requirement for perfection is, I think, the most difficult part of learning to program

-Fred Brooks, Mythical Man Month, 1975

In 1975 Fred Brooks wrote the Mythical Man Month
It's an attempt to understand why so many projects failed to deliver anywhere close to on time or on budget
This talk, like this book, is going to be a lot about software development because that is my point of view
But I think the principles are going to apply to any kind of project or team
Building anything worth building is hard!

Fast forward 25 years

A new millennium had dawned
Software development was still hard
a group of 17 heavy hitters in software development process got together in Utah to talk about how to make it easier
The result is what we know today as the Agile Manifesto

Manifesto for Agile Software Development

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools

Working software over comprehensive documentation

Customer collaboration over contract negotiation

Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

-Agile Manifesto https://agilemanifesto.org/

It's a set of values and principles, not a methodology
Missing are a lot of things most people think of as "Agile"
Standups
Scrum
Kanban
Sprints
Retrospectives
User Stories
Almost 25 years later, agile is still the best way I've found to build anything

UCCSC 2009, UC Davis

My Agile Journey Started Right Here at Davis

at this conference (then called UCCSC)

UCB Won a Sautter Award that year for GLOW "Grad Link on the Web"
I went to hear how they had done it
Instead of talking about the tech or the systems, They talked about Agile
I didn't know it was possible to build things that way at UC
I was hooked

Today's Big Takeaway

Building Agility vs Being Agile

Good news is you can call yourself agile whenever you want
Bad news is you'll never be done building agility
Even better news, you can start today even if it's just you on a team of one building something amazing

Let's Build Some Agility Together

Flow, Flex, Ship

Flow

Flow is the melting together of action and consciousness; the state of finding a balance between a skill and how challenging that task is. It requires a high level of concentration.

-Mirvis, Philip H. (July 1991) doi:10.5465/amr.1991.4279513

It might seem like a weird place to start, but I think flow is the most important part of agility
It's the thing that makes everything else possible
Optimizing your team, process, and tools for flow is the most important thing you can do
It's the 10x multiplier that let's us build the best tool for curriculum management in the world

Time really does seem to behave differently around the act of making software. When things go well, you can lose track of passing hours in the state psychologists call "flow." When things go badly, you get stuck, frozen between dimensions, unable to move of see a way forward.

-Scott Rosenberg, Dreaming in Code

Flow is individual, different people find flow in different ways
I like a absolute silence, surrounded by dogs, feet up, and a full water bottle
Some people like music, some people like to be in a crowd
Make room in your process for people to find their flow

Flow is More than a Metaphor

We Can Pursue Flow

and we can get better at finding flow with practice

The Biggest Enemies of Flow

Distraction
Ego Depletion
Lack of Feedback
Poorly Defined Objectives
While finding flow is individual, there are some things that are universally bad for flow

Let's Get Rid of Them!

Deep Work Removes Distractions

Deep work is the ability to focus without distraction on a cognitively demanding task. It’s a skill that allows you to quickly master complicated information and produce better results in less time. Deep work will make you better at what you do and provide the sense of true fulfillment that comes from craftsmanship. In short, deep work is like a super power in our increasingly competitive twenty-first century economy. And yet, most people have lost the ability to go deep—spending their days instead in a frantic blur of e-mail and social media, not even realizing there’s a better way.

-Cal Newport https://www.calnewport.com/books/deep-work/

Deep work is a big topic, I talked about it at UCTech in San Diego a few years ago and it's worth an entire talk
The highlight is:
You have to make time in your day for serious concentration, not 15 minutes between meetings, but hours at a time
Those that practice Deep Work
Who Schedule time in their day for focus and concentration
Are finding that they can do so much more and better work
AND find much more balance and joy away from work

Ego Depletion

Your ability to make good decisions is finite

Seemingly Trivial Choices add up and take away from your ability to make good decisions
This is why grocery stores put the candy at the checkout
By the time you get to the checkout you've made so many decisions you're more likely to give up and pickup a shiny orange reeses peanut butter cup
The tools we use, the processes we have, the meetings we attend, the emails we send, the code we write, all of these things deplete our ability to make good decisions

-Yehuda Katz https://www.rubyvideo.dev/talks/keynote-10-years

When your process encourages rapid decision making, you are going to hit this issue over and over again
It's impossible to know if you're even making a good choice on how critical an issue is
You have to make time for every decision

Open the Feedback Loop

Feedback is a gift

It should be easy to give even if it is hard to receive.

Feedback is critical to flow
The mechanism for getting feedback should be well defined and easy to use
Automate this:
Netlify
Sentry
Tugboat
When you can't get feedback right away, you may need to move on to something else
You're process should make room for Flexibility in choosing tasks
I like to have big stuff I'm working on and a variety of little things too
We'll talk about releases in a moment, they're a great source of feedback

Speaking of Feedback…

Let's Talk about Automated Tests!!

In 2013 I learned about CI at Irvine. Afterwards I was so blown away I had to take a walk and ended up sitting under a tree thinking about how amazing it would be to test every change.
How freeing that would be (I was pretty bogged down in an endless cycle of fixing bugs and causing other bugs)

Automated Testing FTW!

Tests provide immediate feedback, and keep you in the flow

Tests are a gift to your future self

Tests provide a universal place to store your knowledge

Tests are not just a way to prevent bugs, they are a way to prevent ego depletion

That's a Lot of Work!

Start by Taking One Small Step

One of the most important things in Yoga is just showing up on your mat
It's the same with building agility
Take one small step

Finding Flow, One Small Step

Find a day in December to block off 4 hours in your calendar for deep work

This doesn't have to be a permanent thing, just a one time thing to see how it feels
If you like it, try it again in January
If you don't like it, try it again in January

Flex

You get really good at the things you do everyday, and not so good at the things you don't.

-Maya Magennis

Flex

Agile is a Verb, Not a Noun

You have to practice it everyday

Flexibility requires practice, it requires understanding your limits and your teams limits
Flexibility isn't disorganization
Chaos is inevitable, but that doesn't make organization impossible

Flexibility Requires Process

Kanban
sprints
standups
retrospectives
But the goal can't be the process, the goal has to be agility

Communication is How We Define Objectives

Which is Absolutely Critical for Flow

Meetings, project management tools, email, slack, zoom are, for most people, the antithesis of flow and deep work
It's why they can feel so frustrating
You're trying to find flow and you're being pulled in a million different directions
Without clear objectives, you can't find flow
- And these are the best tools we have for defining objectives and communicating them
- Create your process with deep work in mind, make sure you're making time for flow

Lines of Communication

This is, more than anything else, the hole we fell into when moving to EdIT

3 people, 3 lines
4 people, 6 lines
5 people, 10 lines
6 people, 15 lines
7 people, 21 lines
8 people, 28 lines
9 people, 36 lines
10 people, 45 lines
11 people, 55 lines
12 people, 66 lines
13 people, 78 lines
14 people, 91 lines
Spend time optimizing your communication
When you add someone to your team, do it all again
It's hard to see just how much adding a single person to a team can slow everyone down
Be willing to change and change again

Building Flexibility, One Small Step

Spend 5 minutes as soon as you get back to work thinking about one thing you learned this week that you want to try.

Doesn't have to be something you learned from me!

Ship

Release early. Release often. And listen to your customers

-Eric S. Raymond, The Cathedral and the Bazaar

Releasing is a superpower
The more often you release the better you'll be at it
The shorter the time is between releases the more stable each release will be
The more often you release the more you can learn from your customers
The more you learn from your customers the better your product will be

Ship

Use releases to define your process

(sprints, communication, feedback, retrospectives)

Automated testing makes releases easier

You should start releasing now, even if you don't have tests

Seriously, like, now! What are you waiting for?

Developers will make mistakes You can choose to blame someone else (users, QA, designers, code reviewers, project managers, product owners) or you can accept that correcting your mistakes should be as fast as possible.

Ship

Take your ideas
Get them to users
Repeat!
Write Tests
Make it easy to preview changes and experiment

Ship, Ship, Ship!

I hope you take more than one thing away from this talk
But if you only take one thing away, I hope it's that you should be shipping more often

Shipping Styles, Seen in the Wild

Waterfall

Break down the release into stages, each Stage depends on the one before
Requirements, Design, Implementation, Testing, Deployment
Frontloads feedback at the design stage, and makes it very difficult to adjust

-- Sprint Waterfall

Sprint Waterfall

Way to make Waterfall more agile by breaking releases into smaller pieces
But each sprint often includes the same phases: Requirements, Design, Implementation, Testing, Deployment
Frontloads feedback at the sprint planning stage
Trouble dealing with bugs because sprints tend to be feature focused

Continuous Deployment

Release everything that's ready as soon as it's ready
I don't like continuous change, I think it can be exhausting for users
It's a great way to get feedback

Release Train

Pick a date and release everything that's ready.
Nice balance between communication and feedback cycles
Can be scary to get started because it requires a lot of trust

Small Steps to Build Agility

1. Attend this talk
2. Find a day in December to block off 4 hours in your calendar for deep work
3. Spend 5 minutes as soon as you get back to work thinking about one thing you learned this week that you want to try.
4. If you don't have any tests, write one, even if it's just a test that the page loads
5. Identify the biggest single blocker preventing you from shipping tomorrow
Congrats you've already completed the first step!
This isn't a list of things you need to do
It's just a list of things you can do

The only way to do great work is to love what you do.

-Steve Jobs