Shitty Coders

SQS bounced PHP-serialized Messenger envelopes at 256 KiB

What

Symfony Messenger’s default transport serializer is messenger.transport.native_php_serializer. It runs serialize($envelope), then addslashes(), then base64. It has to base64 because serialize() emits null bytes and SQS wants a string. PHP serialize also writes class names, visibility markers, and types. Base64 adds about 33% on top of that dump. Queues were capped at 256 KiB (MaximumMessageSize).

A new integration started shipping large records from the data source. PHP-serialized, those envelopes no longer fit. SendMessage failed. The worker never saw a job.

The same size error was already in producer logs on a very small number of older messages and went unnoticed. CloudWatch SentMessageSize only records what landed. Volume from the new source made a silent miss into an incident.

JSON of the same data has no class metadata, no null bytes, and no base64. That was the fix. Dual-format decode until the queue drains, then drop the PHP serializer.

Why it hurts

The producer thinks it dispatched. SQS never accepted the body. The worker is idle. You debug the consumer. Rejected sends do not appear in CloudWatch.

Raising MaximumMessageSize to 1 MiB would have let the fat envelopes through. It would not have shrunk them. The SQS Extended Client does the same kind of dodge: park the blob in S3, put a pointer on the queue. Both treat infrastructure as a patch for bad serialization. SQS bills per request, and a request is a 64 KiB chunk. Ireland standard: $0.40 per million after the free tier ($0.50 FIFO). A 2 KB trip is about three requests (send, receive, delete). A 200 KB trip is about nine. A 1 MiB trip is thirty-three.

PhpSerializer pushes bodies across those cliffs. You end up with bounced messages or pay extra to transport fat envelopes.

Cost models, comparisons, and IaC setups: SQS payload size.

What to do

  • Measure the SQS body, not the object: Log strlen() before send. Grep producer logs for Message must be shorter than 262144 bytes. CloudWatch will not show the ones that bounced.
  • JSON encode, dual decode, then drop the PHP serializer: JSON bodies start with { or [. PHP serialize starts with type markers (O: object, a: array, C: custom). Decode both while the queue drains. Encode JSON going forward. Then drop the PhpSerializer decode path. Per transport.
  • (Optional) Pointer if the payload, or the fat part of it, is already in the database: After JSON, you can shrink further by putting ids on the queue and hydrating that piece on consume. That is a nice-to-have. But, it adds DB reads on every consume. Not a way to dodge S3, and not a way to avoid raising MaximumMessageSize. Do it only if the extra load is worth a smaller body.
  • Do not raise the cap or reach for the Extended Client to hide the serializer: Raising MaximumMessageSize is one command. Extended Client is S3 plus a pointer. Both stop the bounce without shrinking the envelope. If bodies grow into 1 MiB, you pay 16 send units and 16 receive units. Extended Client is for blobs that belong in object storage, not for PHP envelopes.