Tackling Hard Problems Head-On (Can Be a Mistake)

A really hard problem can feel like a great chance to flex our skills and finally use some of those architectural patterns we’ve been reading about. But instead of blasting The Eye of the Tiger and diving into the fray, we should indulge that inner voice that says, “But I don’t like my software to be complicated and messy”. Not out of laziness or fear (Fear is the Mind Killer, after all), but because there’s a good chance we’re missing an opportunity to simplify things.

Simple software is faster to implement, cheaper to maintain, easier to staff up, and more enjoyable to work on and use. Not only should we strive for the simplest solution that meets our requirements, but we should resist the idea that a problem is inherently nasty and requires an equally nasty solution. More often, we’re just misunderstanding the problem, applying inaccurate or inefficient conceptual models, or simply overlooking more elegant solutions.

What might we be missing?

There’s a whole internet full of software development techniques for wrangling complexity, so I won’t get too deep into that except to say that if it’s a difficult and interesting problem, somebody else probably already wrote a library for it. What I really want to focus on is the mindset we should have when approaching and defining problems before we get into coding. Or failing that, things we can return to if we get into coding and discover the problem is more vague or tricky than we anticipated.

1. Define the problem before you code

Maybe everybody knows this, and even if you didn’t, maybe your company has solid agile (or other) processes to make sure this happens, but it’s surprisingly easy to let it slip. Do you ever find yourself in a product meeting, while the user’s needs are still being discussed, and you’re thinking, “OK, we’ll just create a new REST endpoint for resource X and it’ll handle…?” We can’t help it, and maybe it even makes us better engineers, but it’s really important not to get attached to those initial ideas if they don’t end up fitting the real problem once it’s fleshed out. Every time we learn something about the problem domain or user needs, we should take a couple of steps back and see if it invalidates things we already thought were “done” or “solved.”

2. Really understand the problem, and double-check that

Sometimes it takes living with a problem for a while to fully understand it, so it may not be possible to do this before coding. But sometimes during coding, we keep running into special cases or extra conceptual layers, and this can be a sign that our framing and modeling of the problem does not match reality. This is the time to adjust mental models, data structures, and even workflows if necessary. At every step, share your evolving thoughts with the team (including stakeholders) to bring in additional knowledge, squash misunderstandings, and keep everybody on the same page. Doubts, questions, and gaps in knowledge should be surfaced and explored right away, because course corrections only get more costly as time goes on.

3. No stone tablets.

Division of labor is a great thing, but specs and issues and even acceptance criteria aren’t perfect and shouldn’t be treated as such. When implementation begins on a feature, we sometimes find that 90% of the effort and complexity comes from a small detail in the requirements. It may be that this detail provides 90% of the feature value, but if it seems otherwise, we should at least explore other options and ask questions about the feature’s underlying purpose and value.. Sometimes there’s a solution that delivers all or most of the tangible value for 10% of the effort, but often it takes a cross-discipline, assumption-busting discussion to generate that idea.

4. Remember, we’re not that smart

It can be tempting to think, “Programming is hard, so only really smart people can understand this, and only when they’re thinking really hard.” This may be true sometimes, but it shouldn’t drown out that inner voice that hates complexity.

Remember, if it’s not obvious to you when you’ve got all of the context in your head, it’ll be ten times harder for the next person to dip in and fix a bug or implement an enhancement. Comprehensibility, maintainability, and even review-ability in the code are of real value… not just as attributes of a codebase but also in conceptual frameworks and processes. It’s worth thinking about how long it will take somebody new to the business domain to wrap their head around the problems and solutions as you’ve defined and modeled them before they even get to the code.

How does this affect process and teamwork?

The advice above is targeted toward engineers, but of course, we do our work in teams, so it’s important to think about how these ideas affect those interactions as well. I think the most crucial thing is to be flexible and “agile”, in the English language sense of the word. In general, if your process includes isolating disciplines and “throw it over the wall” moments, that’s going to preclude necessary rethinking and reframing. Some concrete ideas from my day-to-day work at Revelry:

Pokering

Before we work on a user story, it gets scored by the whole team in a round of poker. At this stage, the story has been written and approved and should contain all of the detail and background required for implementation, but it may be the first time the story is seen by some team members. It can be tempting to gloss over details in the interest of keeping poker moving forward, but in fact, this is the perfect time to ask questions, flush out assumptions, and get a crystal clear understanding of what the story is trying to achieve. If that can’t be accomplished by a short discussion, the story is rejected. Those things need to be resolved before the story can be scored in a future poker session.

Open communication

A fair bit of time and effort has gone into a story by the time it makes it to poker, though, so it’s nice to have an opportunity to chime in earlier. This is where we rely on a culture of mutual ownership and open lines of communication. Everybody can see stories taking shape via email notifications and GitHub bots in Slack, and everybody can post comments and questions at any time through the process. Those comments help inform the process and stay with the story through scoring and beyond so everybody’s concerns are out in the open and addressed to the team’s satisfaction.