How I made my app go from 12M reads a day to 1M? (Flutter/Firestore)

Context: I am the developer for Prism Wallpapers, which has 100K+ downloads on the Play Store. In the last 3 months we had seen a spike in our Cloud Firestore reads, although active users were almost the same. We actually tripled our project costs in that time, so for the first time in almost a year of the app being stable, we had no revenue.

Now back to the problem. Here are some graphs showing what happened and what it looks like now (after the problem has been solved).

Having 12M reads on 7th June

Having constant 1M reads for 1 week.

Active Users (Watch the graph is nearly constant (Drop isn’t that much) between 30 May to 22 June)

The project cost graph showing cost rising linearly for May.

The project cost garph showing cost rising intially but then becoming constant for June.

How was I able to pull this off?

This was made possible due to four key steps we did —

i) Revise user models from the ground up to curb unnecessary repetition for things like the profile photo or username.

We used the Google login within our app, and we used the username/profile photo that Google provided for the user within our app. But since the profile photo or username is sometimes changed externally, we need to ensure that we retrieve the updated data ourselves to maintain consistency. For that, we were saving the updated links for profile photo and updated username in our Cloud Firestore. To fix this, we have redesigned the model to store profile photos in Firebase Storage and allow users to add a display name so there are no unnecessary read requests.

The updated user model

We also wrote a function into our app that simply logs everyone out of their account on this update, so we can kill two birds with one stone. First, everyone can easily switch to a new model without any hassle, and second, unnecessary bot accounts, or like dead accounts are removed.

ii) Switch from FutureBuilders to StreamBuilders.

The switch to streams helps in two ways. First, we now don’t have to call a Future every day to fetch new walls, or every time the user starts the app even if there is no new wall, and it’s the same as with the local cache. The stream now does this automatically and curbs unnecessary requests. Second, the Firestore stream only fetches changes, which automatically means fewer read requests.

Streams vs Futures is a long long debate, on what you should use. I would say you first go to this https://stackoverflow.com/a/59610814/16036745, to understand their differences and then figure out the best solution for your use case.

iii) Move data handling from client side to Firebase.

We also used more and more of Firestore side handling (and using Cloud Functions) by using things like ArrayUnion instead of first fetching that array, adding something into it, and then writing it back. So 1 read — 1 write request, becomes 1 write request.

Yeah, that’s it. It’s that simple.

Use ArrayUnion, ArrayRemove and other FieldValue provided in the package to curb these kinds of requests.

And also, I have said that you map your model intelligently so that you are able to use these kinds of things. This is another reason why we had to change the user model so we can use this in follower/following arrays.

iv) Paginated the hell out of everything. Stopped automatic fetching/retrival, and made user click a button to see more.

I know, I know, this is a pretty big step in terms of user experience, and we’ve already had so many messages from people wanting to bring back auto-loading walls instead of them clicking a button to see more now. But this is necessary.

Understand my situation. I was responsible for lowering the graph of reads. It was like we were taking money out of our revenue, and if we didn’t fix it, we’d have to shut down the servers, or we wouldn’t be able to pay as much (the cost prediction for that month was off the charts, and more than we’d make that month from Prism).

This is the see more button.

Using that limit function to handle pagination

You can use this article to achieve pagination in your app.

Conclusion -

Party time.

So, this is how we pulled it off (Yes, I didn’t do this alone). Huge thanks to LiquidatorAB (my brother in crime).

Although this was difficult and took us a long long time to plan and then code everything, we were able to implement this without any major consequences to the user experience. There were a few minor UI bugs here and there, but you know we’ll fix them soon.

This was a big challenge, of course, and I learned a couple of things from it. First, how much better I am at programming now (logical aspect) than I was last year. For example, I now know what’s wrong with the code and am able to fix it. Secondly, that I never take a bug for granted. Like even the smallest things can sometimes be blown out of proportion. That’s the reason I love this and will continue to do this. Also, don’t think I won’t be cringy about this story for years to come. I will, because that’s exactly what it is. Cringy.

And also, you can check out the whole code for Prism here (https://github.com/Hash-Studios/Prism). Yes, it is open-source.

Hash-Studios/Prism

If this story helped you, you can follow me on GitHub, to access more such codes, and examples. I also post such articles on my blog; you can check it out too.

Thanks for reading, and Happy Fluttering!

codenameakshay - Overview