ZeroMQ

Table of contents

1. ZeroMQ
2. Dedication
3. Preface
   1. ØMQ in a Hundred Words
   2. The Zen of Zero
   3. How This Book Came to Be
   4. Audience
   5. Conventions Used in This Book
   6. Using the Code Examples
   7. Safari® Books Online
   8. How to Contact Us
   9. Acknowledgments
4. I. Learning to Work with ØMQ
   1. 1. Basics
      1. Fixing the World
      2. Audience for This Book
      3. Getting the Examples
      4. Ask and Ye Shall Receive
      5. A Minor Note on Strings
      6. Version Reporting
      7. Getting the Message Out
      8. Divide and Conquer
      9. Programming with ØMQ
         1. Getting the Context Right
         2. Making a Clean Exit
      10. Why We Needed ØMQ
      11. Socket Scalability
      12. Upgrading from ØMQ v2.2 to ØMQ v3.2
      13. Warning: Unstable Paradigms!
   2. 2. Sockets and Patterns
      1. The Socket API
         1. Plugging Sockets into the Topology
         2. Using Sockets to Carry Data
         3. Unicast Transports
         4. ØMQ Is Not a Neutral Carrier
         5. I/O Threads
      2. Messaging Patterns
         1. High-Level Messaging Patterns
         2. Working with Messages
         3. Handling Multiple Sockets
         4. Multipart Messages
         5. Intermediaries and Proxies
         6. The Dynamic Discovery Problem
         7. Shared Queue (DEALER and ROUTER Sockets)
         8. ØMQ’s Built-in Proxy Function
         9. Transport Bridging
      3. Handling Errors and ETERM
      4. Handling Interrupt Signals
      5. Detecting Memory Leaks
      6. Multithreading with ØMQ
      7. Signaling Between Threads (PAIR Sockets)
      8. Node Coordination
      9. Zero-Copy
      10. Pub-Sub Message Envelopes
      11. High-Water Marks
      12. Missing Message Problem Solver
   3. 3. Advanced Request-Reply Patterns
      1. The Request-Reply Mechanisms
         1. The Simple Reply Envelope
         2. The Extended Reply Envelope
         3. What’s This Good For?
         4. Recap of Request-Reply Sockets
      2. Request-Reply Combinations
         1. The REQ to REP Combination
         2. The DEALER to REP Combination
         3. The REQ to ROUTER Combination
         4. The DEALER to ROUTER Combination
         5. The DEALER to DEALER Combination
         6. The ROUTER to ROUTER Combination
         7. Invalid Combinations
      3. Exploring ROUTER Sockets
         1. Identities and Addresses
         2. ROUTER Error Handling
      4. The Load-Balancing Pattern
         1. ROUTER Broker and REQ Workers
         2. ROUTER Broker and DEALER Workers
         3. A Load-Balancing Message Broker
      5. A High-Level API for ØMQ
         1. Features of a Higher-Level API
         2. The CZMQ High-Level API
      6. The Asynchronous Client/Server Pattern
      7. Worked Example: Inter-Broker Routing
         1. Establishing the Details
         2. Architecture of a Single Cluster
         3. Scaling to Multiple Clusters
         4. Federation Versus Peering
         5. The Naming Ceremony
         6. Prototyping the State Flow
         7. Prototyping the Local and Cloud Flows
         8. Putting It All Together
   4. 4. Reliable Request-Reply Patterns
      1. What Is “Reliability”?
      2. Designing Reliability
      3. Client-Side Reliability (Lazy Pirate Pattern)
      4. Basic Reliable Queuing (Simple Pirate Pattern)
      5. Robust Reliable Queuing (Paranoid Pirate Pattern)
      6. Heartbeating
         1. Shrugging It Off
         2. One-Way Heartbeats
         3. Ping-Pong Heartbeats
         4. Heartbeating for Paranoid Pirate
      7. Contracts and Protocols
      8. Service-Oriented Reliable Queuing (Majordomo Pattern)
      9. Asynchronous Majordomo Pattern
      10. Service Discovery
      11. Idempotent Services
      12. Disconnected Reliability (Titanic Pattern)
      13. High-Availability Pair (Binary Star Pattern)
          1. Detailed Requirements
          2. Preventing Split-Brain Syndrome
          3. Binary Star Implementation
          4. Binary Star Reactor
      14. Brokerless Reliability (Freelance Pattern)
          1. Model One: Simple Retry and Failover
          2. Model Two: Brutal Shotgun Massacre
          3. Model Three: Complex and Nasty
      15. Conclusion
   5. 5. Advanced Publish-Subscribe Patterns
      1. Pros and Cons of Publish-Subscribe
      2. Pub-Sub Tracing (Espresso Pattern)
      3. Last Value Caching
      4. Slow Subscriber Detection (Suicidal Snail Pattern)
      5. High-Speed Subscribers (Black Box Pattern)
      6. Reliable Publish-Subscribe (Clone Pattern)
         1. Centralized Versus Decentralized
         2. Representing State as Key-Value Pairs
         3. Getting an Out-of-Band Snapshot
         4. Republishing Updates from Clients
         5. Working with Subtrees
         6. Ephemeral Values
         7. Using a Reactor
         8. Adding the Binary Star Pattern for Reliability
         9. The Clustered Hashmap Protocol
            1. Goals
            2. Architecture
            3. Ports and Connections
            4. State Synchronization
            5. Server-to-Client Updates
            6. Client-to-Server Updates
            7. Reliability
            8. Scalability and Performance
            9. Security
         10. Building a Multithreaded Stack and API
5. II. Software Engineering Using ØMQ
   1. 6. The ØMQ Community
      1. Architecture of the ØMQ Community
      2. How to Make Really Large Architectures
         1. Psychology of Software Architecture
         2. The Contract
         3. The Process
         4. Crazy, Beautiful, and Easy
         5. Stranger, Meet Stranger
         6. Infinite Property
         7. Care and Feeding
      3. The ØMQ Process: C4
         1. Language
         2. Goals
         3. Preliminaries
         4. Licensing and Ownership
         5. Patch Requirements
         6. Development Process
         7. Creating Stable Releases
         8. Evolution of Public Contracts
      4. A Real-Life Example
      5. Git Branches Considered Harmful
         1. Simplicity Versus Complexity
         2. Change Latency
         3. Learning Curve
         4. Cost of Failure
         5. Up-Front Coordination
         6. Scalability
         7. Surprise and Expectations
         8. Economics of Participation
         9. Robustness in Conflict
         10. Guarantees of Isolation
         11. Visibility
         12. Conclusions
      6. Designing for Innovation
         1. The Tale of Two Bridges
         2. How ØMQ Lost Its Road Map
         3. Trash-Oriented Design
         4. Complexity-Oriented Design
         5. Simplicity-Oriented Design
      7. Burnout
      8. Patterns for Success
         1. The Lazy Perfectionist
         2. The Benevolent Tyrant
         3. The Earth and Sky
         4. The Open Door
         5. The Laughing Clown
         6. The Mindful General
         7. The Social Engineer
         8. The Constant Gardener
         9. The Rolling Stone
         10. The Pirate Gang
         11. The Flash Mob
         12. The Canary Watcher
         13. The Hangman
         14. The Historian
         15. The Provocateur
         16. The Mystic
   2. 7. Advanced Architecture Using ØMQ
      1. Message-Oriented Pattern for Elastic Design
         1. Step 1: Internalize the Semantics
         2. Step 2: Draw a Rough Architecture
         3. Step 3: Decide on the Contracts
         4. Step 4: Write a Minimal End-to-End Solution
         5. Step 5: Solve One Problem and Repeat
      2. Unprotocols
         1. Contracts Are Hard
         2. How to Write Unprotocols
         3. Why Use the GPLv3 for Public Specifications?
         4. Using ABNF
         5. The Cheap or Nasty Pattern
            1. Error handling
      3. Serializing Your Data
         1. ØMQ Framing
         2. Serialization Languages
         3. Serialization Libraries
         4. Handwritten Binary Serialization
         5. Code Generation
      4. Transferring Files
      5. State Machines
      6. Authentication Using SASL
      7. Large-Scale File Publishing: FileMQ
         1. Why Make FileMQ?
         2. Initial Design Cut: The API
         3. Initial Design Cut: The Protocol
         4. Building and Trying FileMQ
         5. Internal Architecture
         6. Public API
         7. Design Notes
         8. Configuration
         9. File Stability
         10. Delivery Notifications
         11. Symbolic Links
         12. Recovery and Late Joiners
         13. Test Use Case: The Track Tool
      8. Getting an Official Port Number
   3. 8. A Framework for Distributed Computing
      1. Design for the Real World
      2. The Secret Life of WiFi
         1. Why Mesh Isn’t Here Yet
         2. Some Physics
         3. What’s the Current Status?
         4. Conclusions
      3. Discovery
         1. Preemptive Discovery over Raw Sockets
         2. Cooperative Discovery Using UDP Broadcasts
         3. Multiple Nodes on One Device
         4. Designing the API
         5. More About UDP
      4. Spinning Off a Library Project
      5. Point-to-Point Messaging
         1. UDP Beacon Framing
         2. True Peer Connectivity (Harmony Pattern)
         3. Detecting Disappearances
      6. Group Messaging
      7. Testing and Simulation
         1. On Assertions
         2. On Up-Front Testing
         3. The Zyre Tester
         4. Test Results
         5. Tracing Activity
         6. Dealing with Blocked Peers
      8. Distributed Logging and Monitoring
         1. A Plausible Minimal Implementation
         2. Protocol Assertions
         3. Binary Logging Protocol
      9. Content Distribution
      10. Writing the Unprotocol
      11. Conclusions
   4. 9. Postface
      1. Tales from Out There
         1. Rob Gagnon’s Story
         2. Tom van Leeuwen’s Story
         3. Michael Jakl’s Story
         4. Vadim Shalts’s Story
      2. How This Book Happened
      3. Removing Friction
      4. Licensing
6. Index
7. About the Author
8. Colophon
9. Copyright