How I made my app go from 12M reads a day to 1M? (Flutter/Firestore)
Context: I am the developer for Prism Wallpapers, which has 100K+ downloads on the Play Store. In the last 3 months we had seen a spike in our Cloud Firestore reads, although active users were almost the same. We actually tripled our project costs in that time, so for the first time in almost a year of the app being stable, we had no revenue.
Now back to the problem. Here are some graphs showing what happened and what it looks like now (after the problem has been solved).
How was I able to pull this off?
This was made possible due to four key steps we did —
i) Revise user models from the ground up to curb unnecessary repetition for things like the profile photo or username.
We used the Google login within our app, and we used the username/profile photo that Google provided for the user within our app. But since the profile photo or username is sometimes changed externally, we need to ensure that we retrieve the updated data ourselves to maintain consistency. For that, we were saving the updated links for profile photo and updated username in our Cloud Firestore. To fix this, we have redesigned the model to store profile photos in Firebase Storage and allow users to add a display name so there are no unnecessary read requests.
We also wrote a function into our app that simply logs everyone out of their account on this update, so we can kill two birds with one stone. First, everyone can easily switch to a new model without any hassle, and second, unnecessary bot accounts, or like dead accounts are removed.
ii) Switch from FutureBuilders to StreamBuilders.
The switch to streams helps in two ways. First, we now don’t have to call a Future every day to fetch new walls, or every time the user starts the app even if there is no new wall, and it’s the same as with the local cache. The stream now does this automatically and curbs unnecessary requests. Second, the Firestore stream only fetches changes, which automatically means fewer read requests.
Streams vs Futures is a long long debate, on what you should use. I would say you first go to this https://stackoverflow.com/a/59610814/16036745, to understand their differences and then figure out the best solution for your use case.
iii) Move data handling from client side to Firebase.
We also used more and more of Firestore side handling (and using Cloud Functions) by using things like ArrayUnion instead of first fetching that array, adding something into it, and then writing it back. So 1 read — 1 write request, becomes 1 write request.
Yeah, that’s it. It’s that simple.
Use ArrayUnion, ArrayRemove and other FieldValue provided in the package to curb these kinds of requests.
And also, I have said that you map your model intelligently so that you are able to use these kinds of things. This is another reason why we had to change the user model so we can use this in follower/following arrays.
iv) Paginated the hell out of everything. Stopped automatic fetching/retrival, and made user click a button to see more.
Understand my situation. I was responsible for lowering the graph of reads. It was like we were taking money out of our revenue, and if we didn’t fix it, we’d have to shut down the servers, or we wouldn’t be able to pay as much (the cost prediction for that month was off the charts, and more than we’d make that month from Prism).
You can use this article to achieve pagination in your app.
So, this is how we pulled it off (Yes, I didn’t do this alone). Huge thanks to LiquidatorAB (my brother in crime).
Although this was difficult and took us a long long time to plan and then code everything, we were able to implement this without any major consequences to the user experience. There were a few minor UI bugs here and there, but you know we’ll fix them soon.
This was a big challenge, of course, and I learned a couple of things from it. First, how much better I am at programming now (logical aspect) than I was last year. For example, I now know what’s wrong with the code and am able to fix it. Secondly, that I never take a bug for granted. Like even the smallest things can sometimes be blown out of proportion. That’s the reason I love this and will continue to do this. Also, don’t think I won’t be cringy about this story for years to come. I will, because that’s exactly what it is. Cringy.
Prism brings you exclusive wallpapers & setups straight to your Android device. With unlimited downloads and…
If this story helped you, you can follow me on GitHub, to access more such codes, and examples. I also post such articles on my blog; you can check it out too.
Thanks for reading, and Happy Fluttering!