Clock reading should return a range of time + confidence level, instead of point in time.Google assumes 6ms drift for clock synchronized with NTP every 30 secs, 17 secs if synchronized once a day.

Human error is the major cause of network outages.When a timeout occurs, you still don’t know whether the remote node got your request or not, or if is still queued.If you send a request to another node and don’t receive a response, it is impossible to tell why.In a system with thousands of nodes, something is always broken.SQL standard definition of isolation levels is flawed.Consistency is a property of the application, not the database.

High availability, low latency, occasional stale read.

Multi datacenter, offline clients, collaborative editing.

Scalability of read-only replicas requires async replication.

Two operations are concurrent if neither “happens-before” the other.

RPC/location transparency: there is no point to make a remote service look too much like a local object, because it is a fundamentally different thing.

Base64 encoding increases data size by 33%.

Less convenient for simple one-off queries.

Good for evolvability, ease to add new relations and properties.

Data models affect how we think about the problem that we are solving.

Every legacy system is unpleasant on its own way.

Reliability: Systems should work correctly even in the face of adversity, including human error.

Martin also explains some of the book contents his distributed system course. It made me smile that there is one chapter dedicated to the perils of distributed programming, when the fact is that the whole book is a warning after another of all the possible things that can go wrong. But you can’t take averages as gospel.These are my notes on Designing Data-Intensive Applications by Martin Kleppmann. Recently we performed a PHP upgrade and experienced a 30% improvement in average API response time. Of course, decreasing your average response time is a good sign.

That’s why Amazon looks at the top 0.01% of users and what speeds those high LTV users experience – not the average user. Those are the users you need to test with and benchmark against. High value users (especially!) need fast, reliable responses. An athlete with 6 years of training data, hundreds of sessions performed, and millions of pounds lifted is going to require more data processing and transfer than a new user.īut the question isn’t about relative speed, it’s about requirements and acceptable speed. Coaches and athletes with the most training data also experience the slowest API response times. We experience this uneven distribution challenge every day at TrainHeroic. In other words, the customers with the most purchases and highest LTV (life time value). At Amazon, the customers with the slowest page request times are the customers with the most data.

Designingdata intensiveapplications