If you build apps, you probably spend time thinking about your App Store rating. It may even keep you up at night. So, what does it take to get your app to a rating you’re happy with so you can sleep? At Anonyome Labs, our entire organization put forth effort to get our App Store rating to a 4.8 out of 5 stars for our flagship app MySudo. It required everyone being hyper focused on quality for us to reach our goal.
There are several ways to drive quality into a product from within your company. Start by creating an atmosphere where quality is everyone’s responsibility. I was fortunate when I started as a QA Lead at Anonyome that they already had a QA-centric culture established. I chatted with Anonyome Labs executives Paul Ashley, Steve Sartor and Paul Jensen to learn more about how they achieved this cultural attitude geared toward quality. According to Paul Ashley, quality is a higher priority than new features: “We will go slower on features and focus on quality.”
We also discussed other methods to driving higher levels of quality. Some additional approaches we discussed were setting clear quality goals, guiding quality from design, and providing exceptional customer support. Steve Sartor pointed out that sometimes the traditional severity/priority categories aren’t enough to drive the right focus in prioritization of defect backlog, and we need another dimension. Sartor added, “We call this Tranquility; how does it make the user feel, how often is it reported through customer support/feedback channel.” We also discussed some ways to drive quality through testing, specifically through all-hands testing.
In our discussions, it became clear that the more user testing our product undergoes, the more insight into the level of qualitywe will have. There are some creative approaches you can implement to help keep ‘all eyes on quality.’ First, ensure all team members are also users. Second, implement cross-functional regression testing. Third, run frequent internal betas.
Everyone is a User
One of the ways Anonyome Labs drives quality through our people is that every employee is an avid user of our communications app, MySudo. MySudo is a privacy-based telecommunication app that allows you to control your personal data by creating and maintaining multiple profiles for different roles, each including a phone number and email. Because one of the use cases of MySudo is to compartmentalize your communications, it’s very natural for employees to use it for work contact info. When you walk around Anonyome Labs, you will notice that there isn’t a single desk phone on anyone’s desk; that is because all employees use MySudo for their work phone numbers. You will frequently see employees on their devices using MySudo.
Anonyome team members use MySudo to call each other, call vendors, video call, and message. We have group chats to discuss work, share memes or let each other know we are running to an appointment. On occasion we will implement a ‘No-Slack Wednesday’ where we encourage folks to communicate over our app instead of Slack.
This method is especially useful for raising awareness of the app quality because we are all using our product every day. When we experience product strengths and weaknesses first hand, it allows us to prioritize upcoming releases accordingly. As executive Paul Jensen says, “ ‘We would rather be drinking our own champagne than eating our own dog food.’” We want the app to be the best, because we are using it. Jensen added, “Everyone is encouraged to report quality and tranquility issues no matter how small they are. It is rare that we receive an issue via our users that has not already been reported by one of our employees.”
Another benefit of having your co-workers test is that they will be using production environments and data. This gets your app in front of production data sooner which can help surface issues.
Steps you can take
Identify use cases in your apps that make sense for your team to run in their everyday lives. Are there any tools that can be replaced by your app? Are there communications that can be done in your app? You may have to get creative in this approach, but the point is to get your people in your app every day.
Regression Testing
If you build apps, you have no-doubt been in a crunch to get testing done before a release. According to Marlene Hopkins in Quality — Cost — Time: they say ‘pick any two— “As a ‘rule’, the higher the standard of quality that is required, the higher the cost will be because more time is needed”. As startups we don’t often have the money or resources to match the level of quality we want. So, we need to get creative with what is within our control. One creative solution is to have your cross-functional teams help out with regression testing.
Regression testing is a great area of testing to pull in non-testing resources to help. Usually regression tests are solid, non-brittle test cases that should be passing.
Steps you can take
During your next regression run, assign groups of tests to cross-functional teams, which can include marketing, support, sales, data and analytics. Assign a QA person as the point person to review results, answer questions about test cases and log any bugs. Get buy-in from everyone’s managers and communicate schedules as well as results. Be sure to thank your participants.
Internal Beta
Another extremely effective testing tool is internal beta tests. These are scheduled, time-boxed periods where your co-workers dedicate time to testing your app. You can run these in coordination with a release or as needed. It may be a good idea to run the internal beta before regression as you are likely to get some feedback you want to implement. The most important aspect of internal beta is empowering your employees to provide feedback. Make sure you acknowledge their requests, concerns, and feedback and ensure you follow-up on the feedback. You’ll need to monitor what kind of responses you are getting so you can prioritize what is really happening. It’s a great way to predict how your app will do in the wild.
Steps you can take
Before your next release, schedule an internal beta for 1-2 days. Communicate the plan ahead of time. Ask co-workers to spend a set amount of time on testing. You can even get explicit and ask them to cover certain functional areas. Give specific instructions on providing feedback and empower your users to provide feedback. Make sure leadership across the company is behind the initiative.
In summary, get your people using your app as users and testers! You will identify issues quicker, cover more edge cases, utilize production data sooner and ultimately be able to prioritize quality in your releases.
References
Quality- Cost-Time- They say pick any two — Marlene Hopkins https://www.linkedin.com/pulse/20141124220757-117728303-quality-cost-time-they-say-pick-any-two/
