JochemKuijpers.nl Personal blog and portfolio

A quick and simple Java rate limiter

java, telegram

For my current project (a Telegram Bot API framework) I had to figure out a way to limit the amount of outgoing API calls that bots would send. I did not want the bot programmer to have to deal with the error message Telegram sends you when you've passed your requests per second limit, nor with the timing of the API calls.

The API limits work as follows: bots can send up to 30 messages per second, with an additional limitation that they can only send 20 messages per minute for a specific group chat. This RateLimiter will be the basis of such a system.

The way my framework works is that every update is handled as a task in some thread pool. Whenever the bot sends a message, this thread is blocked until the API responds. To limit the outgoing requests, I will block the thread some more before sending, if necessary. This way, incoming updates can be handled without having to deal with asynchronous requests, callbacks, etc. Instead, the API calls just block and wait, then return the proper value.

Here's what I was able to come up with in about an hour. In terms of the finished lines of code, this is probably worth 10 minutes of programming, but could be useful to anyone facing the same problem.

package nl.jochemkuijpers.ratelimiter;

import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.ArrayBlockingQueue;
import java.util.concurrent.BlockingQueue;

/**
 * A simple BlockingQueue based rate limiter.
 * Usage: call limit() to throttle the current thread (blocks)
 * and call tick() at regular intervals from a separate thread.
 */
public class RateLimiter {
	private final long fillPeriod;
	private final BlockingQueue<Object> queue;
	private long timer;

	/**
	 * Create a simple blocking queue based rate limiter with a
	 * certain capacity and fill rate. Be careful when handling
	 * lots of requests with a high capacity as memory usage 
	 * scales with capacity.
	 * 
	 * @param capacity
	 *            capacity before rate limiting kicks in
	 * @param rate
	 *            rate limit in allowed calls per second
	 */
	public RateLimiter(int capacity, double rate) {
		if (rate <= 0) {
			this.fillPeriod = Long.MAX_VALUE;
		} else {
			this.fillPeriod = (long) (1000000000L / rate);
		}
		this.queue = new ArrayBlockingQueue<Object>(capacity);
		this.timer = System.nanoTime();
	}

	/**
	 * Tick the rate limiter, advancing the timer and possibly
	 * unblocking calls to limit()
	 */
	public synchronized void tick() {
		long elapsedTime = System.nanoTime() - timer;
		int numToRemove = (int) (elapsedTime / fillPeriod);

		// advance timer
		timer += fillPeriod * numToRemove;

		List<Object> discardedObjects = new ArrayList<Object>(numToRemove);
		queue.drainTo(discardedObjects, numToRemove);
	}

	/**
	 * A call to this method blocks when it is called too often
	 * (depleted capacity).
	 * 
	 * @return false when interrupted, otherwise true
	 */
	public boolean limit() {
		try {
			queue.put(new Object());
		} catch (InterruptedException e) {
			return false;
		}
		return true;
	}
}

And here is an example on how to use it:

import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;

public class Example {
	public static void main(String[] args) {
		// capacity of 10 and a rate of 1/second
		RateLimiter limiter = new RateLimiter(10, 1);

		// schedule rate limiter ticks every 100 milliseconds
		ScheduledExecutorService scheduler = Executors
				.newSingleThreadScheduledExecutor();
		
		scheduler.scheduleAtFixedRate(new Runnable() {
			@Override
			public void run() {
				limiter.tick();
			};
		}, 0, 100, TimeUnit.MILLISECONDS);
		
		// bark 100 times, but limit using the rate limiter
		for (int n = 0; n < 100; n += 1) {
			limiter.limit();
			System.out.println("bark #" + n);
		}
	}
}

It works by utilising Java's standard library's BlockingQueue, which blocks when you try to add an object when it is full (only works with bounded implementations such as the ArrayBlockingQueue). It requires the tick method to be called at regular intervals. This should be done at a frequency in the same order of magnitude as the fill rate passed to the constructor. Example: If the fill rate is 10, you probably want to call tick every 100 milliseconds (or 50, or 150, whatever).

The drawback of this implementation is that it will store unused objects that literaly serve no purpose other than to have the BlockingQueue fill up and block when it reaches its capacity. These objects need to be disposed of by the garbage collector, and take up a bit of valuable memory space.

Since I will be using a small amount of RateLimiter objects combined with small capacities (less than a hundred), I think the current solution is fine for now. I'm posting this mainly because I think it's an interesting way to implement a rate limiter as it automatically handles the order of unblocking blocked threads.

New website! Woo!

meta

This is the first blog post on my new website. It's just here to show you this website exists.

I still need to finish the control panel so I can more easily add posts, add a comment system and implement a way to share my posts on social media (by which I mean Twitter).

About me

I'm a 21-year-old Computer Science student at Eindhoven University of Technology.

More on the about page.

Archive

Tags

telegram, java, meta

Other stuff