Pub/sub Sisyfos stupidities

·

2 min read

Table of contents

No heading

No headings in the article.

In a project using Google cloud functions I recently got billed 10 times the normal amount. Turns out the delinquent is the pub/sub system.

Googles pub/sub system is a very handy way to execute background tasks. Since you want the Cloud function to finish asap, the pub/sub system comes in handy for starting a background task. So instead of including the task that is not required to send the response inside the function, you send a message with a topic right before exiting the function. And the subscriber subscribing to that topic gets called, but the cloud function can send its response and exit. You might be thinking that you can send the response, then do background tasks, but then the background tasks are not guaranteed to be executed.

Obiously, there are things to watch out for. In this case for some reason the message was larger than 10 MB, which is weird. But what is even weirder is the way this is handled by the system. There is an error sending a message that large, fine so far. But the system keeps trying to send the same message for seven days. Alot and alot..
Now remember the message is too large, that is the reason for the error.. but yeah, let's keep resending it, hoping that it shrinks? That is not even Sisyfos, the rock stays at the foot of the mountain.

Pub/Sub is an asynchronous and scalable messaging service that decouples services producing messages from services processing those messages.