Build apps that scale to billions

“Our app now handles thousands of users without a sweat. Thanks Julien!” – Igor Subira


Buy Now Get the sample chapter
Image
Python

Why Python?

Python is an easy to learn programming language. Yet, it is powerful, versatile and comes with a luxuriant ecosystem. That makes it a very productive programming language for beginners and experts.

Building substantial applications with Python is totally possible. Like many other technologies, mastering Python requires know-how, techniques, trade-offs to be made, and traps to avoid.

It would be awful to end up like this, right?

“Python is fast enough for our site and allows us to produce maintainable features in record times, with a minimum of developers.”
Cuong Do
Software Architect at YouTube

Python is already trusted by thousands of companies using it a very large scale.


How can this book help me?

Build for Resiliency

Today's applications run on large and distributed infrastructures. They need to handle failure without blinking. In an environment where anything could go wrong, how do you design your code to handle those failures?

Remove Bottlenecks

Learn how to plan an architecture that has no bottleneck. Get straight answers to your conception questions and solutions that work and solve concrete cases. Discover distributed systems that scale properly.

Performance

We'll talk performance: how to write faster Python code, profile it, and optimize it. Read about concrete use-cases on how to enhance slow applications, transforming them into modern applications.

Batteries Included

Use the included Docker image to replay all the examples presented in the book. You get a complete Python environment that you need in order to execute the examples. Use the notebooks to run them in a click from your Web browser.

When you build a scalable, performing and distributed system, there are hundreds of problems to solve

Slow request
Server crash
Network timeout
Race condition
Dead lock
Network failure
Cold cache
High latency

This book offers shortcuts.
Here's what it includes:

What's scaling?

Scaling across CPUs
Distributed Systems
Service-Oriented Architecture

CPU scaling

Using Threads
Using Processes
Using Futures
Advanced Futures Usage
Daemon Processes

Event Loops

Basic Pattern
Using Asyncio
Network Server

Queue-Based Distribution

RQ
Celery
Handling failure
Chaining Tasks
Multiple Queues
Monitoring

Designing for Failure

Naive Retrying
Retrying with Tenacity

Lock Management

Thread Locks
Processes Locks
Multiprocessing Locks
Inter-Process Locks
Using etcd for Distributed Locking
Using Tooz Locking Abstraction

Group Membership

Creating, Joining and Leaving Groups
Using Capabilities
Using Watchers Callbacks
Consistent Hash Rings
Partitioner

Building REST API

The WSGI Protocol
Streaming Data
Using ETag
Asynchronous HTTP API
Fast HTTP Client
Testing REST API

Deploying on PaaS

Heroku
Amazon Beanstalk
Google App Engine
OpenShift
Beyond PaaS

Testing Distributed System

Setting Up Environments with tox
Manage External Services with pifpaf
Using Fixtures with pifpaf

Functional Programming

The Functional Toolkit

Caching

Local Caching
Memoization
Distributed Caching

Performance

Memory and CPU Profiling
Profiling Strategy and a Case
Zero-Copy
Disassembling Code

Learn Straight From the Experts

Scaling Python includes seven interviews from experienced Python developers. They share their know-how and wisdom on the various themes covered in the book.

Mehdi Abaakouk

Mehdi is a free software hacker, working as a Senior Software Engineer at Red Hat. He contributes to OpenStack, the largest open-source project using Python, where he builds a fault-tolerant, scalable and distributed cloud platform.

Chris Dent

Chris Dent is a Staff Engineer at VMware, a member of the OpenStack Technical Committee, and an HTTP connoisseur. Chris has among other things created Gabbi, a fabulous Python testing tool for HTTP, and is responsible for HTTP REST API testing.

Joshua Harlow

Joshua is a highly experienced engineer in distributed systems, working at GoDaddy. Josh is very knowledgeable on principles that make Python application resilient and faults tolerant as he wrote and maintained several of those over the years.

Naoki Inada

Naoki is a Researcher and Architect at KLab. He also happens to be one of the CPython developers. He worked on several significant features in CPython that are used by thousands, such as the asyncio event loop.

Victor Stinner

Victor is a long time CPython developer, working on the language itself for several years now. He is well known in the community for working on making CPython faster and leads several performance-oriented projects. He likes to talk about optimizations, profiling, and performance when using Python, and how to make the right decisions.

Jason Myers

Jason is a Python developer, working as a Software Engineer at Juice Analytics. He wrote an entire book on SQLAlchemy, the well-known Python SQL library (Essential SQLAlchemy (O'Reilly)) He worked on cloud computing platforms, as a Web developer, and as a data engineer.

Alexys Jacob-Monier

Alexys is the CTO of 1000mercis and is part of the open-source software community for a few years now. He regularly gives speeches at Python conferences (PyCon) and talks about how to leverage Python when distributing applications. He writes, maintains and operates highly distributed Python applications.

image

Who's the Author?

Hi, I'm Julien, a Free Software hacker for close to twenty years now. For more than 12 years, I've been developing software using Python.
I spent those last years as a Project Team Leader in OpenStack, a distributed cloud platform. It's the largest existing open-source Python code base (2.5 millions of lines of Python).
Today, I work at Datadog and runs my own company Mergify — where I write Python code everyday.


Julien Danjou

Staff Engineer at Datadog / Founder at Mergify

"A Must Read"

The readers of Scaling Python talk about it better than anyone else.

Ready to build scalable apps?

Starter
$39
  •   The ebook in PDF, HTML, MOBI and EPUB formats
  • 7 experts interviews
Purchase
Old-School
$49
  •   The ebook in PDF, HTML, MOBI and EPUB formats
  •   7 experts interviews
  •   The book printed in paperback format, delivered to your door
Purchase
Best-Seller
Standard
$89
  •   The ebook in PDF, HTML, MOBI and EPUB formats
  •   7 experts interviews
  •   Future editions of the ebook
  •   All code examples ready to run
  •   Docker image with the complete environment to run all the book examples
Purchase
Enterprise
$199
  •   The ebook in PDF, HTML, MOBI and EPUB formats
  •   7 experts interviews
  •   Future editions of the ebook
  •   All code examples ready to run
  •   Docker image with the complete environment to run all the book examples
  •   Web interface to run all the examples as notebook – easy editing and toying
  •   License to share with 10 members of your team
Purchase

Get the sample chapter

Not convinced yet? Get the table of contents and read the beginning of the book for free:

Download Sample Chapter
image

Frequently Asked Questions

I bought the paperback version. How do I get the electronic version?

Lulu is not able to distribute the electronic version along the paperback version. Therefore once you purchased your paperback copy, forward me your receipt and I'll get you the electronic version.

What if the book is not for me?

If you think the book has no value to you, then I don't want your money. Ask me for a refund.

I need a license for more than 10 users.

Let's discuss this over email.

I probably need more than a book. Are you available for consulting?

Let's talk about that over email.

I have another question!

Send me an email.