Context

I have a webpage named TFT Double Up that finds the best compositions and pairs so players can coordinate their strategies without manually checking conflicts.

I did this project as a side project because I was playing with one friend and we did this manually by looking at the best pairs and trying to not have conflicts.

At first I created custom backend API hosted in Digital Ocean and when user asked opened the webpage I was returning the results and caching them for 1 day. You can see that code in these two lines of code:

@cache(expire=86400) if not settings.debug else (lambda f: f)
async def get_best_pairs(request: Request) -> JSONResponse:

I didn't play this very often, but it was still a useful webpage. Four dollars is not a lot, over 12 months it's 48 dollars, which was enough to make me think about finding a free alternative.

As each request was almost the same, only the data was changing from one day to the other, I thought about using GitHub actions.

I found in their documentation that: GitHub Actions usage is free for self-hosted runners and for public repositories that use standard GitHub-hosted runners. See Choosing the runner for a job.

As my API was basically a chron job I refactored the code to include a daily workflow and save it in a json file.

Key insight

All users were requesting the same data. The only thing changing was the dataset, which updated once per day. That meant my API was effectively acting as a cron job, not a real backend service.

Refactor

The refactor was quite simple, first of all I created a new file named daily_fetch.py to call the two functions and compare if the results are different from the previous day, if they are different change the current.json and create a new entry in the historic folder.

Secondly in the .github/workflows directory and to call the daily_fetch.py and push it to the data folder if there are differences. The only remarkable point you have to change is to allow Read and Write permissions in the settings/actions.

I also changed the FE to fetch the data from directly from the generated JSON then I turned off my Digital Ocean server and now I'm saving 4 dollars per month, which is not a lot but it was worth it as the change was not as big as I originally thought.

Architecture diagram

Before: User → Frontend → Backend (Digital Ocean) → Cache → API

After: GitHub Action (daily) → Generate JSON → commit to repo → Frontend fetches static data

When this works

This pattern works well when:

• Data changes infrequently

• Results are identical for all/most users

• Real time updates are not needed

Result

Before

• DigitalOcean - $4/month

• $48/year After

• GitHub actions - $0

If your backend only runs once per day, it's not really a backend.

You have two tables:

• orders: a user bought a headphone and we save it in an order.

• addresses: responsible for saving the different addresses of the user.

And we will fetch country_code for our endpoint.

The column country_code is not ready yet and it's filled with NULLs. So we add 'temporary_code' AS country_code to have the filter logic ready and we do: WHERE country_code LIKE '%temporary_code%' .

SELECT  order_id,  
        address_id,  
        size,  
        street,  
        country,  
        'temporary_code' AS country_code  
FROM orders  
    LEFT JOIN addresses ON orders.address_id = addresses.id
WHERE country_code LIKE '%temporary_code%'

But something is not working correctly... Why?

The order in a SQL execution is:

1. FROM

2. WHERE

3. SELECT

So in this case we will be filtering by the country_code column that already exists inside the addresses table and not the one in SELECT, and this will return empty.

Temporarily you could write:

SELECT  order_id,  
        address_id,  
        size,  
        street,  
        country,  
        'temporary_code' AS country_code  
FROM orders  
    LEFT JOIN addresses ON orders.address_id = addresses.id
WHERE 'temporary_code' LIKE '%temporary_code%'

Solution Overview

If you're only interested in the solution, here it is:

# Build the main query
query: Query = session.query(Post)

# Build the subquery
subquery = (
    self.session.query(Reaction)
    .filter(Reaction.user_id == user_id)
    .subquery()
)

# Join with the subquery
query = query.outerjoin(subquery, Post.id == subquery.c.post_id)
alias_reaction = aliased(Reaction, subquery)
query = query.add_entity(alias_reaction)

Background

In many backend systems, we have different models for validation (Pydantic) and for models in database representation (SQLAlchemy). Although some libraries like SQLModel attempt to bridge this gap, they are still in early stages.

Consider a scenario where you have two SQLAlchemy models: Post (P) and Reaction (R). Here, P represents a social network post, and R represents a user's reaction to a post.

Suppose you need to query all P along with the R of a specific User (U). Directly joining all posts with all reactions and filtering by user_id may be inefficient due to the large volume of data. Instead, you can optimize this by filtering reactions first and then performing the join.

The SQL for this optimized approach looks like:

SELECT *
FROM 
  posts p
LEFT JOIN 
  (SELECT * 
   FROM reactions 
   WHERE user_id = {{user_id}}
  ) r
ON 
  r.post_id = p.id

The query itself it's not hard, but what I have found difficult was to found a way of keeping both models when using SQLAlchemy.

First we do a subquery, this would be equivalent to the subquery we mentioned before to speed up the join operation.

# Build the main query
query: Query = session.query(Post)

# Build the subquery
subquery = (
    self.session.query(Reaction)
    .filter(Reaction.user_id == user_id)
    .subquery()
)

This part it's not something hard and we can easily find documentation about it. The tricky part comes now, I had a hard momment finding a way to return both models in one single return. I could have returned the model P and the all the fields from R, but remember, we want a return with only both models, so it's clean and concise.

After some investigation I discovered that you need to alias the subquery, what I mean by this is that you need to specify the model that will be returned and then add the entity to the original query. Below you can see how it's done.

# Join with the subquery
query = query.outerjoin(subquery, Post.id == subquery.c.post_id)
alias_reaction = aliased(Reaction, subquery)
query = query.add_entity(alias_reaction)
return query.all()

Then we just do:

post_models: list[PostModel] = [PostModel.model_validate(post) for post, _ in posts]
reactions_models: list[ReactionModel] = [ReactionModel.model_validate(reaction) for _, reaction in reactions]

It's interesting to understand why this happens, when you do SELECT * behind it deconstructs the model and returns you all the fields. But now in SQLAlchemy you don't want to lose these models as you may want to return them, and that's basically the reason.

It was an interesting investigation that was not easy to find and I wanted to share with all of you.