Building the Ads Auction Troubleshooting Tool

The context

Twitter's Ads Revenue Product Organization supported the advertising business that funded the platform. Inside that org, Technical Operations was a small team of specialists embedded alongside product and engineering, each person owning a product area and responsible for understanding it deeply enough to triage and resolve issues before they reached engineering.

When I joined the team, I started with ownership of measurement and mobile app promotion. Over time I expanded my scope to cover the full campaign setup and review experience, and took on something nobody had specifically asked me to: becoming the subject matter expert on Twitter's ads auction system.

The auction is the engine underneath every ad that runs on Twitter. It determines which campaigns serve, in what order, at what price, based on a combination of bid, creative quality, and targeting relevance, each weighted differently depending on campaign type. Understanding how it worked, and why a given campaign might be underperforming within it, was one of the most technical things anyone in a non-engineering role could do.

There was a tool that made this possible: RASAM. It required running a script and returned a plain text output of how campaigns were ranking in the auction. It worked, but it was rough. Interpreting the output required knowing what to look for: the weights applied to different campaign types, the patterns that indicated a real issue versus a campaign simply needing improvement. I got good at it. Most people didn't have access to it at all.

The team at the time was five people total, myself included. My manager had recently moved to a new role and I had assumed most of those responsibilities. I was the most technical person on the team, and I knew it. So I started identifying the people on my team with an appetite for that kind of investigation, and reached out to our sister team, Ad Ops, to find two members who wanted to learn what I was doing with RASAM.

The problem

The pattern I kept seeing was this: roughly 90% of the escalations that reached me didn't require resolution. They required interpretation. A seller or an Ad Ops team member would flag that a campaign wasn't performing as expected, and the answer was almost always findable in the auction data. A weak creative score, a targeting parameter narrowing reach, a bid that wasn't competitive in the relevant auction tier. Real auction bugs were rare. Perceived auction problems were constant.

Every one of those escalations took time. My time, or an engineer's time. And that time was spent explaining something that, with the right tool, the person asking the question could have found themselves.

The more I sat with it, the clearer the problem became. The bottleneck wasn't knowledge or engineering capacity. It was access and interpretability. RASAM existed, but it was effectively gated behind me. The output it produced required context most people didn't have. And that meant every auction question eventually found its way to my desk, regardless of whether it needed to be there.

The idea and the pitch

What I had in my head was simple: take what RASAM was doing and put a real interface on it. Clear labels for each auction stage. Scores presented in a way that made the weighting visible. A stack ranking of which factors were having the biggest impact on a campaign's performance, something a seller could look at and understand without needing to ask a follow-up question.

The goal wasn't to make me redundant. It was to push the answers further upstream, so that the questions reaching me were the ones actually worth my time.

The timing helped. The engineering team had just brought on a contractor for special projects, and when I surfaced the problem, the team lent the resource to the effort. There was a lot of enthusiasm from the start. Everyone understood that having engineers or me interpret plain text auction output on a case-by-case basis wasn't a valuable use of time. There was also something genuinely motivating about the idea of making one of the most technical aspects of advertising legible to the people closest to customers.

Building it

The contractor was heavily backend-focused and needed help understanding who the tool was actually for and what problem it needed to solve. So before any real development started, we sat down and worked through it together. I broke down the experience that sales and Ad Ops were having, what they were trying to understand, and what "clear enough to act on" actually meant for someone without a technical background.

Out of those conversations came the core requirements: visible auction stage scores, a ranked breakdown of which campaign attributes were most impacting performance, and callouts clear enough that a seller could identify the issue and know what to do about it without escalating further.

The build took roughly a quarter. We checked in every two weeks. I'd review progress, give feedback on whether the output was readable to a non-technical eye, and flag anything that needed adjusting. It was a light process by most standards, and in hindsight I would have invested more time upfront in formal discovery, talking to sales and Ad Ops teams more deeply before development started rather than validating as we went. At the time I didn't have those skills yet. I was building a PM muscle I didn't fully have a name for.

Rollout and results

We didn't open it up all at once. We started with the most technical teams, TechOps and Ad Ops, and brought in a small group of SMB sellers for early access to validate that the outputs landed the way we intended. The signal we were watching for: fewer escalations from the teams using it.

When we saw that, we expanded. Slowly, by team, until the tool was broadly available across the sales and operations functions.

During initial rollout, we could see that operations teams upstream were resolving auction-related issues in about two days on average, down from five. They weren't escalating as often because they could find the answer themselves. After full rollout, the remaining day count likely dropped further, though we didn't measure it precisely.

What changed for me was more qualitative. The escalations that did reach my team tended to be actual issues rather than perceived ones. Fewer interpretation questions meant I could move real auction bugs into engineering hands faster, and spend the time I was saving on larger problems and projects that had been sitting on the backlog.

What I'd do differently

I would have done proper discovery before development. More time with sales teams and Ad Ops upfront, understanding the full texture of their frustration rather than building from my own interpretation of it. The tool worked, but it was shaped mostly by what I understood the problem to be. With more structured discovery, it might have been shaped by what customers actually needed, and those two things aren't always the same.

At the time I just didn't have those skills. Looking back, this project is part of how I started developing them.

What this project means to me

I had no authority over the engineering team. No PM title, no formal product ownership of the auction system, no mandate to build anything. What I had was a reputation: for technical depth, for understanding our customers and systems, for not escalating things that didn't need to be escalated. The fact that the team trusted me with a contractor and the space to pursue this reflected something I'd built over time without fully realizing it.

That's the thing I'm most proud of. Not the tool itself, but what it took to make it happen.

It also validated something I'd been feeling for a while: that I wanted to move toward product. And it showed me, in equal measure, how much I still needed to learn to get there.