My work
MY ROLE: PROMPT WRITING | UX WRITING | CONTENT STRATEGY | CONVERSATION DESIGN | CONTENT AT SCALE
I was part of a team that began building an AI chatbot to serve Amazon employees resolve their Human Resource questions more efficiently. I created voice, tone, and conversation flow guidelines while working with my HR partners, workforce trust, and legal teams. I translated this into system prompts that we tested. Our initial user feedback indicated that our users felt the chatbot's responses were robotic and that they were skeptical of AI. In addition to flagging this metric, I enlisted the help of a data scientist and developed a response evaluation framework that I presented to the project leaders. I also proposed a name change for the product that made it feel more human and approachable. I wrote a document supporting my proposal and successfully convinced the team and leadership to make the change. This cross-team collaboration drove measurable improvements in response quality, making interactions feel more natural and human-centered.
Foundation
​Most products need systems to ensure good content is repeatable. For Aza, that meant establishing naming conventions, content standards, and voice principles that shaped the core product as well as every plugin built on it. I aligned stakeholder groups early, balanced competing priorities, and delivered high-quality UX content, source prompts with examples, conversation design guidelines across multiple product iterations and under tight deadlines. In some cases, this meant partnering with stakeholders to assess whether an AI-conversational approach was the right fit for the use case or whether leveraging AI in another way was optimal.
Framework
While working with the team, I realized we weren't truly measuring whether our conversations were resonating with our users, we were only checking if we delivered the right answer. To address this, I built a human annotation rubric and crowdsourced team members to analyze over 300 user queries. I partnered closely with Applied Scientists to push AI evaluation models past factual accuracy into readability, response optimization, and sentiment. I helped bridge the gap between measuring only response accuracy by adding additional metrics such as readability scores. I also led cross-functional efforts to establish content guardrails and quality standards, ensuring the bar stayed high as the product scaled.
Impact
-
Response ratings climbed to 62% between September 2024 and February 2025.
-
Writing style maintained strong positive metrics despite increased query complexity.
-
The HR plugin achieved a 4.5/5 rating for clarity and a 4/5 success rate for new users.