{"version":"im7uspS2a43L","topics":["Sharding","Queues","Redis","Postgres"],"cards":[{"id":"walk-through-how-an-sqs-message-is-processed-including-failures","topics":["Queues"],"questionHtml":"Walk through how an SQS message is processed, including failures.","answerPages":["<p>Receive → process → delete; a received message stays hidden from other workers for its visibility timeout (30 s by default).</p><ul><li>If the worker crashes, or is still processing when the timeout expires, the message reappears for another worker.</li><li>If it keeps failing, a redrive policy moves it to a dead-⁠letter queue after <code>maxReceiveCount</code> receives.</li></ul>","<p>Retention is a per-⁠queue setting: <strong>4 days by default</strong>, from 1 minute up to 14 days.</p><ul><li>SQS is regional: it stores messages across several availability zones in one region, with no copy in another region.</li></ul>","<p>Poll efficiently: long-⁠poll in batches, then call again straight away.</p><ul><li>On an empty queue a receive call blocks until a message arrives, for up to 20 s.</li><li>Long polling is opt-⁠in: set a wait time, as the default is 0 s.</li><li>One call returns <strong>at most 10 messages</strong>, a hard limit, and only 1 unless you ask for more.</li></ul>"]},{"id":"what-does-setnx-do-in-redis-and-what-is-it-used-for","topics":["Redis"],"questionHtml":"What does <code>SETNX</code> do in Redis, and what is it used for?","answerPages":["<p>Sets a key <strong>only if it does not exist</strong>: it returns 1 if it set the key, 0 if not. Used for simple locks and for deduplicating work. Prefer <code>SET key value NX PX 30000</code>: it also sets an expiry atomically.</p>"]},{"id":"how-does-a-consistent-hashing-ring-work-from-placing-nodes-to-mo","topics":["Sharding"],"questionHtml":"How does a consistent-⁠hashing ring work, from placing nodes to moving data?","answerPages":["<p>Each node sits at <code>hash(node_id)</code>, for example <code>hash(&quot;10.0.0.5:6379&quot;)</code>, on a ring spanning the hash’s range: 2^32 positions for a 32-⁠bit hash.</p><ul><li>The input is the list of nodes, kept strongly consistent, for example in etcd or ZooKeeper.</li><li>Virtual nodes space the nodes out: each is placed many times, at <code>hash(node_A#001)</code>, <code>hash(node_A#002)</code> and so on. Cassandra defaults to <strong>16 per node</strong>.</li></ul>","<p>To find a key’s node, hash the key with the same function, then binary-⁠search the sorted node positions for the <strong>first node at or after it</strong>, wrapping round to the start.</p><ul><li>Example: nodes <code>a=3, b=55, c=99</code> and <code>hash(value) = 42</code> give node <code>b</code>.</li><li>Clients precompute the positions from the node list and update them only when the list changes.</li></ul>","<p>Moving data when nodes join depends on what the nodes hold.</p><ul><li>Caches: don’t bother; let the new nodes fill up slowly.</li><li>Data stores: dual writes plus an asynchronous data transfer for a while, then flip over to the new layout.</li></ul>"]},{"id":"why-does-postgres-need-vacuum","topics":["Postgres"],"questionHtml":"Why does Postgres need <code>VACUUM</code>?","answerPages":["<p>MVCC never changes a row in place: an <code>UPDATE</code> writes a new row version and leaves the old one as a dead tuple. VACUUM makes that space reusable.</p><ul><li>It also freezes old transaction ids before they wrap around.</li><li>Autovacuum runs it in the background: tune it, never turn it off.</li></ul>"]},{"id":"what-does-redis-do-when-memory-is-full-and-why-jitter-ttls","topics":["Redis"],"questionHtml":"What does Redis do when memory is full, and why jitter TTLs?","answerPages":["<p><strong>Nothing is evicted.</strong> The default policy is <code>noeviction</code>: once <code>maxmemory</code> is reached, writes fail with an OOM error; reads still work.</p><ul><li>On 64-⁠bit, <code>maxmemory</code> defaults to 0 (no limit), so set one.</li><li>For a cache, configure <code>allkeys-⁠lru</code> (or <code>allkeys-⁠lfu</code>).</li><li>ElastiCache and Redis Cloud default to <code>volatile-⁠lru</code>: LRU among keys with a TTL.</li></ul>","<p>Add random jitter to TTLs so that keys written together don’t all expire together.</p><ul><li>Synchronised expiry sends a stampede of cache misses to the database at once.</li><li>A small random offset per key spreads the expiries out.</li></ul>"]},{"id":"how-can-several-workers-take-jobs-from-one-postgres-table-withou","topics":["Postgres","Queues"],"questionHtml":"How can several workers take jobs from one Postgres table without blocking each other?","answerPages":["<p>Each worker locks the row it claims and skips rows that others hold:</p><pre><code>SELECT id FROM jobs\nWHERE status = 'queued'\nORDER BY id\nLIMIT 1\nFOR UPDATE SKIP LOCKED;\n</code></pre><p>Do the job in that transaction, then mark it done.</p>"]}]}