Rate Limiting HTTP Requests (via http.HandlerFunc middleware)

go, http

Solution

The rate limiting example you've linked to is a general one. It uses range because it gets requests over a channel.

It's a different story with HTTP requests, but there's nothing really complicated here. Note that you don't iterate over a channel of requests, or anything -- your HandlerFunc is called for every incoming request separately.

func rateLimit(h http.HandlerFunc) http.HandlerFunc {
    return func(w http.ResponseWriter, r *http.Request) {
        remoteIP := r.Header.Get("REMOTE_ADDR")
        if exceededTheLimit(remoteIP) {
            w.WriteHeader(429)
            // it then returns, not passing the request down the chain
        } else {
            h.ServeHTTP(w, r);
        }
    }       
}

Now, choosing the place to store the rate limit counters is up to you. One solution would be to simply use a global map (don't forget safe concurrent access) that would map IPs to their request counters. However, you would have to be aware of how long ago the requests were made.

Sergio suggested using Redis. Its key-value nature is a perfect fit for simple structures like this and you get expiration for free.

Problem

I'm looking to write a small piece of rate-limiting middleware that: - Allows me to set a sensible rate (say, 10 req/s) per remote IP - Possibly (but it doesn't have to) allow for bursts - Drops (closes?) connections that exceed the rate and returns a HTTP 429 I can then wrap this around authentication routes or other routes that might be vulnerable to brute-force attacks (i.e. password reset URLs using a token that expires, etc.). The chances of someone brute forcing a 16 or 24 byte token are really low, but it doesn't hurt to go that extra step. I've had a look at https://code.google.com/p/go-wiki/wiki/RateLimiting but am not sure how to reconcile it with http.Request(s). Further, I'm not sure how we'd "track" requests from a given IP over any period of time. Ideally I'd end up with something like this, noting that I'm behind a reverse proxy (nginx) so we're checking for the `REMOTE_ADDR` HTTP header rather than using `r.RemoteAddr`: ``` // Rate-limiting middleware func rateLimit(h http.HandlerFunc) http.HandlerFunc { return func(w http.ResponseWriter, r *http.Request) { remoteIP := r.Header.Get("REMOTE_ADDR") for req := range (what here?) { // what here? // w.WriteHeader(429) and close the request if it exceeds the limit // else pass to the next handler in the chain h.ServeHTTP(w, r) } } // Example routes r.HandleFunc("/login", use(loginForm, rateLimit, csrf) r.HandleFunc("/form", use(editHandler, rateLimit, csrf) // Middleware wrapper, for context func use(h http.HandlerFunc, middleware ...func(http.HandlerFunc) http.HandlerFunc) http.HandlerFunc { for _, m := range middleware { h = m(h) } return h } ``` I'd appreciate some guidance here.

Original source