Crash course on optimizing WebSocket compression: a look under the hood of Deflate compression and how to configure it for best performance.
igrigorik — 108 posts
A hands-on look at how to configure Nginx to transparently deliver and cache WebP assets via Accept negotiation.
Deploying New Image Formats on the Web igvita.com
If we really want to make an impact on web performance, then image formats is the place to do it. There is absolutely no reason why we shouldn't have dozens of specialized formats, each tailored for a specific case and type of image. But before we get there, we need to iron out some kinks...
If you are using Google Analytics, then you have a powerful anomaly detection engine at your disposal... and it can be easily configured to help you monitor the performance of your site: server response times, DNS, page loading times, and more.
Wait, Chrome DevTools could do THAT? igvita.com
mod_pagespeed is a just in time (JIT) performance compiler for the web. This free and open-source Apache module automates all of the most popular web-performance best practices by dynamically rewriting and optimizing your website assets. A look under the hood and the architecture of the module within Apache...
Web Fonts Performance: Making Pretty, Fast! igvita.com
An in-depth look at the performance and optimizations behind web fonts, and Google Web Fonts in particular.. Web fonts are here to say, and that's a good thing - yes, even for performance!
Hand's on look at the HTTP Archive data format, which allows us to export, analyze, and visualize network performance data from the network timeline... Learn how to build a performance dashboard in three easy steps, with free and open-source tools!
Latency: The New Web Performance Bottleneck igvita.com
'High speed' connectivity is not all about bandwidth, latency is the new bottleneck for most web browsing applications - especially, for the mobile web. An overview of what latency is, how it affects us, and the current latency numbers for wired and the mobile web.
Network latency is anything but free. To address this, Chrome learns the network topology as you use it via a number of predictor heuristics. Let's take a peek under the hood of the Chrome networking stack and see how this can be applied to building faster web apps.
Chrome's remote debugging allows you to easily drive the browser via a WebSocket: interact and modify the DOM, listen to network events, instrument the V8 VM, and much more! A hands on example of driving Chrome via Ruby + WebSockets.
Measuring Site Speed with Navigation Timing igvita.com
If your job is to think about web performance, then you need to approach it from a users perspective: use Navigation Timing to measure true latency, leverage Site Speed reports in Google Analytics, and focus on the shape (distribution) of the performance data! A hand's on look at the spec, and sample GA reports.
JDK7's Fork/Join combines a double ended queue (deque) and a recursive job partitioning step to minimize synchronization - a great design pattern to keep in mind for single host and distributed cases! A look under the hood of the Fork/Join framework, and a few JRuby examples which will light up all of your available cores.
WebSockets, SPDY, SSL, and persistent connections are in, except that our infrastructure can't support most of these use cases. To enable the modern, real-time web, we need to drag our 'back office' architectures into this century.
Web-VPN: Secure Proxies with SPDY & Chrome igvita.com
Is it possible to securely route SSL sessions via an HTTP Proxy? A look at the existing HTTP specification, and what the SPDY spec and Google Chrome team bring to the table - hint, the answer is yes. As of January 2012, over 50% of all internet sessions will be SPDY capable, and Web-VPN is one of the many great features it brings.
Do use reuse HTTP connections in your code? Does your app server support pipelining? You can speed up your code and your apps by orders of magnitude if you answer those questions correctly. A look at the HTTP internals, and Ruby libraries and solutions.
Server-Sent Events are an HTML5 feature which allows you to easily push real-time notifications from the server to the client! Why not use a websocket? Turns out SSE offers a much simpler API optimized for one way push. A quick look at an SSE API and a Ruby/Goliath server implementation.
Rails 3 Asset Pipeline & Google Closure igvita.com
Rails 3.1.0 is on the horizon and Asset Pipeline is the king of the show. A hands on look at some of the internals + example of extending the Asset Pipeline to support Google's Closure library.
Protocol Buffers, Avro, Thrift & MessagePack igvita.com
Which is the best serialization format? Protocol Buffers from Google, Facebook's Thrift, MessagePack, or maybe Avro? A look at the use cases and the historical context in which each was developed is instrumental in helping us answer this question.
HTML5 Visibility API & Page Pre-Rendering igvita.com
0-60: Deploying Goliath on Heroku Cedar igvita.com
Heroku's new Cedar stack allows us to deploy any ruby app to their cloud! Which means, we can now deploy async Goliath Ruby apps with near minimal effort: streaming API's, async API endpoints, etc, all without any of the callback mess!
Stream(SQL) Event Processing with Esper igvita.com
Hadoop batch-processing is not the panacea to every problem. StreamSQL allows us to easily filter, aggregate, and even merge multiple realtime streams to detect correlations, run custom calculations, and much more - all without extra code! A quick intro to StreamSQL, the Esper engine, and a JRuby example to apply it to a real-time Twitter stream>.
State of art for reverse proxies (nginx, haproxy, etc), is terrible: need to add a new appserver? Modify the config, reload - ugh. An experiment with Goliath, SPDY and 0MQ to build a zero-config reverse proxy.
Intuition & Data-Driven Machine Learning igvita.com
Machine learning is hard, right? It doesn't have to be. Focus on developing intuitive insights, get more data, and get started! A video and Ruby code examples from a talk at GoGaRuco 2010.
VMware CloudFoundry: Ruby powered PaaS igvita.com
VMware announced their CloudFoundry project earlier this week, which is an open source PaaS platform - run your own "mini Heroku" or "EY cloud" on your own servers! The entire platform is powered by a collection of distributed Ruby daemons and services - great case study of building a distributed Ruby system.
Life beyond HTTP 1.1: Google’s SPDY igvita.com
If you are using Google's Chrome broweer, and you are using Google web services today, chances are, you are not running over HTTP! More likely, you're running over SPDY. A look under the hood of SPDY and a Ruby parser for SPDY.
Mneme is an HTTP web-service for recording and identifying previously seen records - aka, duplicate detection. It is powered by Goliath, Redis, and a collection of bloomfilters under the hood. Check out the source on github.
Goliath: Non-blocking, Ruby 1.9 Web Server igvita.com
Goliath is an open source version of the non-blocking (asynchronous) Ruby web server framework powering PostRank. It is a lightweight framework designed to meet the following goals: bare metal performance, Rack API and middleware support, simple configuration, fully asynchronous processing, and readable and maintainable code (read: no callbacks).
Quick introduction to the features, API, and why it exists!
Quick introduction to the features, API, and why it exists!
Concurrency with Actors, Goroutines & Ruby igvita.com
Should we remove threads in Ruby? What are some "advanced concurrency models"? A look at Actors vs CSP/pi-calculus and Agent gem which models Go-concurrency in Ruby.
Routing with Ruby & ZeroMQ Devices igvita.com
ZeroMQ allows the programmer to assemble high-performance, in process fanouts, queues, and other messaging patterns required for building high-performance applications. ZDevice is a new Ruby library & DSL which simplifies this process.
ZeroMQ: Modern & Fast Networking Stack igvita.com
ZeroMQ is a network library which provides a much needed layer of abstraction on top of the traditional BSD socket API: transport agnostic, connection management, routing. With language bindings for a dozen languages (and Ruby, of course), it is a fast, modern API which makes developing high-performance network apps fun again.
Multi-core, Threads & Message Passing igvita.com
It's not a question of whether threads, events or message-passing is a better model - the hardware trends require that we use all of the above. Either the VM (like Ruby) has to abstract it all for us, or we need to build frameworks to match the capabilities of the hardware.
Speed Tracer Server-side Tracing with Rack igvita.com
Google's Speed Tracer instruments the browser and the V8 VM to show you what the browser is doing: GC, reflow, etc, such that you can optimize the performance of your code. A recent feature is the ability to also bring in server-side traces! Rack-speedtracer is a middleware which allows any Rack compatible app to surface its runtime information in Speed Tracer.
Scalable Work Queues with Beanstalk igvita.com
Beanstalk is a fast, in-memory work queue system - a memcached of work queues. A look at the features, advanced recipes, and its use at PostRank, with Ruby examples, of course.
Zookeeper is a distributed lock and metadata store originally incubated within the Hadoop umbrella of services. However, it is also generally useful for solving distributed problems, and is designed to be highly available and scalable -- a quick look at the architecture, API's, and working with it in Ruby.
Non-blocking ActiveRecord & Rails igvita.com
The mysql gem is one of the worst offenders when it comes to performance of Rails. A look under the covers of the driver architecture & available alternatives.. Followed by a demo of an async ActiveRecord driver! (MySQLConf presentation)
Event driven programming does not have to be complicated. With a little help from Ruby 1.9 Fibers, and new library em-synchrony, much of the complexity is easily abstracted, which means that we can have all the benefits of event-driven IO, without any of the overhead of a thread scheduler or complicated code.
Distributed Ruby with the MagLev VM igvita.com
MagLev Ruby VM has a unique persistence model: it is a database and a ruby runtime in one. Based on the smalltalk VM, it offers a JIT, transparent object persistence, distributed layer, and many other goodies. A on overview of the state, architecture and examples of use.
Flow Analysis & Time-based Bloom Filters igvita.com
Working with large streams of data is becoming increasingly widespread, be it for log, user behavior, or raw firehose analysis of user generated content. Time-based bloom filters are just the tool for the job & the bloomfilter gem now implements a Redis-backed, counting, time-based bloom filter!
Ruby & WebSockets: TCP for the Browser igvita.com
WebSockets are one of the most underappreciated innovations in HTML5 - bi-directional, fully asynchronous and data agnostic data exchange. Dev builds of Google Chrome, Firefox and Safari now all support WebSockets, which means that as developers we start taking advantage of the new architectures. A hands on look at the API, and implementation of Ruby WebSocket server and clients.
Future of RDBMS is RAM Clouds & SSD igvita.com
Rumors of the demise of relational database systems are greatly exaggerated. While the new NoSQL storage engines are exciting to see, it is also important to recognize that relational databases still have a bright future ahead - RDBMS systems are headed into main memory, which changes the playing field all together.
State of Ruby VMs: Ruby Renaissance igvita.com
There are 8 alternative Ruby VM's and 4 of them will hit 1.0 status in the upcoming year. A detailed look at the past year and where the community is heading (hint: it's an exciting time to be a Rubyist).
Consuming XMPP PubSub in Ruby igvita.com
A how-to for using blather and switchboard gems in Ruby to work with XMPP PubSub streams.
Nginx & Comet: Low Latency Server Push igvita.com
Nginx can be converted into a fully capable long-polling Comet server with a single plugin. Best of all, it is fully asynchronous, supports message queuing, memory limits, and is easy to get started with. A look at the configuration and a sample ruby client.
Advanced Messaging & Routing with AMQP igvita.com
AMQP & RabbitMQ are industrial grade message routers with support for failover, load balancing, pubsub, you name it! A look at the available brokers, features, and integration with Ruby.
Collaborative Filtering with Ensembles igvita.com
Ensemble methods are proving to be very effective for doing collaborative filtering (product recommendations, etc) as the results from both the Netflix and the recent GitHub challenge clearly show. A look at what ensembles are, and how they can be used (in Ruby, of course).
Masking Latency & Failures with Squid igvita.com
Squid cache server can help you mask slow application servers and intermittent downtime via stale-while-revalidate and stale-if-error extensions. A hands on look at configuring Squid and connecting it with a simple Rack application.
Extending Tokyo Cabinet DB with Lua igvita.com
Tokyo Cabinet/Tyrant database can be scripted via Lua extensions: inject new functionality, application or data logic, etc. A look at the available extensions (auto expiry, working with sets, wordcount map-reduce example) and how to get started (with Ruby).
A look at the emerging Webhooks & PubSubHubbub HTTP callback implementations and how they enable easy, distributed, web-scale PubSub.. with Ruby code samples and example of new PubSubHubbub Ruby gem.
Profiling Ruby With Google's Perftools igvita.com
Using Google Perftools to profile and visualize the codepaths of any Ruby application.
Fibers & Cooperative Scheduling in Ruby igvita.com
Continuations (known as Fibers) in Ruby 1.9 give us the ability to do cooperative scheduling, and implement synchronous API's on top of async libraries, which are both much more efficient and avoid the complexity of threading. A hands on look at the theory, code, and applications.
Building a high-performance Ruby proxy server is both remarkably simple, and a very powerful technique: intercepting arbitrary data, extending protocols & injecting new functionality, ... It's a wonderful hammer, learn how to use it!
A hands-on look at sorting performance, and Trie, Priority Queue and Heap data structures implementations and use cases in Ruby - drop in replacements for performance and better memory usage!
Ruby Polyglot: Talking with Erlang igvita.com
How-to for transparently connecting Ruby and Erlang VM's with erlectricity gem. Get the best of Ruby, and the functional programming of Erlang without sacrificing either.
Scrooge is an automated application layer query optimizer, which tracks usage of your attributes and then dynamically rewrites the ActiveRecord queries to only select those columns.
A hands on look at the Tokyo Cabinet database project by Mikio Hirabayashi (powering mixi.co.jp) - it is an incredibly fast, and feature rich database library, with Ruby libraries, and memcached and RESTful protocol support.
Ruby 1.9 Internals: Ordered Hash igvita.com
A look at the internals of the Ruby 1.9 Hash: how it works, how it compares to 1.8, and alternatives.
When you're working with large datasets it's always nice to have a few algorithmic tricks up your sleeve, and Bloom Filters are exactly that - often overlooked, but an extremely powerful tool when used in the right context. Learn the theory and get started with Bloom Filters in Ruby.
Ruby AST for fun and profit igvita.com
An AST to a program is what the DOM is to a web-page - a hands on look at working with the Ruby's Abstract Syntax Tree and the projects that are using it. Ever wanted to translate Ruby to Lolcode?
Scaling ActiveRecord with MySQLPlus and ConnectionPool: A hands on look at the new ConnectionPool interface in Rails 2.2RC1, and the MySQLPlus gem which promises asynchronous processing.
Performance & Load testing with Log Replay igvita.com
A hands on look at the available tools, and techniques to do log replay to simulate a collection of users + release of a Ruby driver to automate these tests.
Asynchronous Database Access in Ruby igvita.com
Benchmarks of the latest MySQLPlus, EM/MySQL and DBSlayer libraries for asynchronous database access in Ruby - with surprising and promising results!
Can a Rails app serve 3 million dynamic pageviews a day? Absolutely, causes.com is doing it on a daily basis.
Make use of UNIX signals to easily toggle debug mode on any process.
Hands on tutorial / look at Ruby EventMachine (the speed demon).
Hands on how-to for optimizing response times and coordinating traffic between multiple clusters of app. servers, with the help of HAProxy.