CONTEXT

1 of 5 product designers

Stakeholder Interviews, Personas, JBTD, Journey Mapping, Prototyping, User Testing

The Ontario Digital Service (ODS) is using Notify to send digital reminders and reduce paper mail

5 weeks

MY ROLE

METHODS

TIMELINE

THE PROBLEM STATEMENT

Notify is a stand alone, self-service, OPS-wide platform designed for use by any ministry service to send on-demand or batch communications . It is a government Software-as-a-Service accessed via API or a web interface by ministry cluster and communications teams to:
      • Create and manage templates
      • Send emails, SMS, automated phone calls
      • Obtain reporting and analytics

Current government partners include:
      • ServiceOntario Digital Reminders
      • MOH Travellers SMS

We designed a dashboard and statement of charges for the Notify team to to use when billing other ministries for using the service. These invoices would be sent every 3 months (according to the billing cycle). 

We also decided to use a free tier model to bill other ministries. Hence, the dashboard and statement of charges clearly indicate which usage is covered under the free tier, and which usage must be paid for. 

SOLUTION

GOALS

Business (ODS) Goals: 

How might we improve so that our billing process of Notify is more successful based on price competitiveness, transparency and stability?”

We have observed that Notify, as a free service, isn’t financially sustainable or price competitive for the ODS, and does not have the level of cost transparency that ministry clients would need, which is causing uncertainty in the adoption of this platform service.

User (Ministry) Goals: 

•  increased number of program areas adopt Notify
•  in-depth knowledge of usage stats 
•  price competitiveness & sustainability
(we don't lose money)

•  cost savings for each program area
•  transparent breakdown of consumption costs 
•  good usability / comprehensiveness / learnability 

PROJECT SCHEDULE

Given that the ODS is a rapid prototyping lab, we completed this entire project in a span of 5 weeks. 

Review background material and existing information. Identify key users and stakeholders for interviews. Go over project goals and scope.


Create stakeholder and user interview plans. Interview stakeholders and users, analyze interview notes and map out current state.


Continue stakeholder interviews and synthesize main takeaways. Create personas, jobs to be done and ideal-state journey maps.


Follow up with stakeholders. Design and iterate low-medium fidelity prototype and test with Susmita and Louis. Analyze and synthesize feedback and reiterate.


Finalize prototype based on feedback. Wrap up project engagement and compiled handoff materials. 

Kick-off

Sprint 1: Jan 11 - 15          

Sprint 2: Jan 18 - 22

Sprint 3: Jan 25 - 29

Sprint 4: Feb 1 - 12

When we started off the project, we had no idea where to start or who to talk to because government billing was such a new concept to all of us on the team. 

Therefore, we used a snowball approach to recruit individuals to interview. At the end of each interview we would ask for clarity on who else we should talking to, something that proved to be very helpful. We tracked all of this information in a stakeholder tracker.

01. STAKEHOLDER INTERVIEWS

Conducting interviews with over 20 stakeholders across the Ontario Government

In order to prepare for out interviews, we created interview guides based on the different roles / personas we were interviewing for. Although these guides helped us stay organized, I found that I preferred going off script at times and found it more valuable to ask probing questions based on what the interviewee was already saying. This helped us gain additional insights. 

Improving our Notetaking Method

Creating Interview Guides

During one of our weekly retros I brought up a concern that our current note taking method wasn't very effective. This is because we had 3 notetakers writing in the same box, which often led to multiple sticky notes saying the exact same things. 

We reflected on what we liked and didn't like about our current note taking method in order to find a better solution. 

The solution we came up with as a team was to have two note takers only, with one focused on broader insights and the other writing direct quotes to reference.






I also found it helpful to add a section of 'Consolidated Notes' after each interview so that we could use our key takeaways as insights or create next steps. This kept me organized and gave me a high level overview of what we had learned, and what we had yet to learn and find out more about.

Writing direct quotes
Dividing content based on theme
Using sticky notes instead of paragraphs
Lots of repetitive notes
Overwhelming to find key findings 

•  Users receive invoices for their usage over a set time
•  Users pre-buy a bulk amount of services for use over time


•  Differing prices/rates based on tier of usage
•  Flat rate charged per use


•  Select based on usage and user need
•. Reconciliation and journaling can be time consuming

02. SYNTHESIZING INTERVIEW NOTES

Findings from user interviews, market research and customer journey mapping

"The main feedback we can give you is to make your pricing fair and based on usage"

"The tricky part is you don't want to punish ministries for using the service a lot"

Payment Model: pay-per-use vs. prepay

Pricing: tiered vs. flat rate

Billing Frequency: annual, quarterly, monthly

Takeaways, Constraints and Tradeoffs

Based off of the user interviews, we found that there are a few constraints in regards to billing within the government. For example, even though all ministries within the Ontario government are paying from the same 'account', each ministry and program area has allocations they need to stay within. Therefore, it is important to bill / journal ministries internally so that all program areas are spending within their allocation.  This is a process we were hoping to avoid due to the resources needed to bill back other ministries/program areas. 

Another constraint that we faced was related to the lack of usage of Notify so far. We had to make a lot of assumptions in regards to what data would be important to show or which features are necessary since we didn't have any current usage that we could analyse. Furthermore, we were constrained by the information that AWS does and doesn't collect.This means that there were features our interviewees want to see such as breakdown of phone codes which is not possible due to AWS. We also couldn't install an API which shows cost per message in real time. 

Lastly, we wanted to be able to automate the entire internal billing process but that was not possible at the time due to tech constraints. However, this is a goal for future iterations. 

03. PROTOTYPING FLOWS

User Personas & Jobs To Be Done

Journey Maps

Based on the different individuals we interviewed we created four key personas that interact together during the billing journey map. For each persona we also highlighted their 'jobs to be done' to ensure that our solution would meet each of their goals. 

For each individual persona we noted their user flow context, their goals and process, pain points and gains as well as their empathy map so that we could create a solution tailored towards them. 

ODS Getting Billed

Ministry/Cluster Getting Billed

Once we were able to identify the key stakeholders who would be interacting together, we mapped out their journeys and how they work together in the billing process. We decided to create two journey maps - the first one shows the current process of how the ODS is getting billed from AWS. We got this information from our stakeholder interviews. 

The second journey map we made shows how the ODS would bill another ministry/cluster for using Notify. Based on our own research as well as findings from stakeholder interviews, this is the journey we recommend the Notify team to use. 

Feeback from Show-n-Tell

•  Allow for the ability to search by specific date range
•  Quarterly - based on fiscal year

 
•  Break down fixed costs vs. variable costs
•  Variability in AWS pricing based on location, providers, etc.
•  Notify will set a flat rate per message based on averages to charge users

04. PROTOTYPING DASHBOARD + INVOICES

Creating a final product that meets all three goals

Must have

We started our third sprint by feature prioritization. Based on our interviews and our own brainstorming we had come up with many features that would be beneficial to the platform, but given the scope of the project we decided to primarily focus on the 'must have' and 'should have' features, and leave the 'could haves' for future releases. 

Should have

Could have

Will not have

•  Overall usage by channel and associated cost

•  Usage trends - averages, outliers

•  Filtering by date range (fiscal year, quarter, month)

•  Views by service
(ex. Trial vs live)

•  Export data

•  Filtering by user
     •  Cluster
     •  Ministry
     •  Program area
     •  Project

•  Billing based on success rate

Based on the features we wanted to include we started dashboard prototyping. We wanted our information to be broken down easily into two views. The first is the macro view which is a high level overview of Notify’s performance as a platform and would have many performance metrics. We also wanted a messo view (which from our research is the middle ground between macro and micro) which would have mid level breakdown.

•  Showcase billable quantity: prices based on message lengths rather than number sent
     •  SMS segments 
     •  Voice call minutes


•  Consolidate total service cost and highlight who is paying what (Notify vs. Ministry)

Filtering: Billing Cycles

Pricing

Billable Parts

Cost Payee Breakdowns

We presented all of our work from the Discovery phase in a one hour show-n-tell. After our presentations we got asked some great questions and received feedback.

Required information:
•    Issuing and receiving offices
•    Service name
•    Usage period
•    Breakdown of charges
•    Free tier usage and overall usage

Macro and Messo Dashboard View

We prototyped potential designs for each of the must and should have prioritized features for the MVP.  Each one of the product designers on the team brainstormed their own designs and then we merged them together through a voting exercise. 

This is the prototype that we made for user testing. We broke down the information into 2 views. 

Macro View
•   High level overview of Notify as a platform
•   Performance metrics



Messo View
•   Mid-level breakdown of messages sent per channel and service and their associated costs

Statement of Charges

We also prototyped what a statement of charges would look like for finance admins from other ministries. This statement can be exported from the admin billing dashboard above. 

05. EVALUATING AGAINST USER FEEBACK SESSIONS

User Testing Sessions

In order to test our prototypes we conducted user testing twice with a total of 14 user testing sessions. 

Key User Testing Findings: 
1)  More clarity is needed surrounding what the free tier entails and how it would be shown on their bill

2)  Users want more transparency into the breakdown of where the messages went to in terms of sub-services within their service

3)  Clarification between number of messages sent vs. billable quantities

Different Iterations

Final Designs