all work
shipped[2024]22

Algocode

Distributed online judge for C++. Microservices and kernel isolation.

Problem

Running untrusted user code at scale without a shared blast radius between submissions.

Built

3 independently deployable services (Auth, Code Manager, RCE Engine) coordinated through RabbitMQ. Each submission spawns an ephemeral sibling Docker container with Linux namespaces, cgroups, and seccomp enforcing strict CPU/memory/time limits.

Clientpostman / apiAuth Servicedjango · postgresqlCode Managerdjango rest frameworksubmit · poll resultRabbitMQcpp_submit_queueRCE Engineconsumer · schedulerC++ Judgesibling docker · sandboxedAC · WA · TLE · MLERabbitMQresult_queueMongoDB · Redisresult store · cache

3

independent services

AC/WA/TLE/MLE/SegFault

verdicts

kernel-level

isolation

token-bucket

rate limiting

The build

[Idea]

Algocode was the project where I had to think about what 'securely running untrusted code' actually means. The naive version is a Docker container per submission, and it sounds fine until you realize that 'shared blast radius' is the entire problem. One user submits a fork bomb, the whole node dies.

[Framing]

The fix was Linux isolation primitives. Namespaces for filesystem and process isolation. cgroups for CPU and memory caps. seccomp to whitelist the syscalls a C++ judge actually needs, since the syscall surface for a C++ binary is small and predictable. Each submission gets a sibling container that lives only as long as the submission, with hard wall-clock limits and a hard syscall allowlist.

[Build]

The system splits into three independently deployable services. Auth handles user management and tokens. Code Manager is the public-facing API. It validates the user, writes the submission to its own database, and publishes an event to RabbitMQ tagged by language. RCE Engine consumes from its language-specific queue and runs the judge in the hard-isolated sibling container. Results publish back via a unified RabbitMQ result queue, and Code Manager caches in Redis and persists to Mongo.

[Deploy]

The whole thing runs on free-tier AWS EC2 plus an Azure VM (1GB RAM each), which is why the rate-limit middleware exists. Three submissions per minute per user, via Token Bucket. I built the limiter in Django Middleware rather than reaching for an API gateway, because adding cloud spend for one feature felt wrong. The split into three repos (algocode-auth, code-manager, rcee) was deliberate. Independent deploys and independent failures.

[What's next]

What's next, if I pick this back up. Skip Mongo for results and use Postgres JSONB, one less service to operate. Build a frontend. Add Java and Python judges (the RCE Engine stubs are in place). The rate-limit logic deserves a small dashboard so I can see who's hitting it.

Stack

9 techs · grouped by layer

Backend

  • Django
  • DRF
  • RabbitMQ
  • JWT/OAuth2

Infrastructure

  • Docker
  • AWS

Data layer

  • PostgreSQL
  • MongoDB
  • Redis

Also used

Docker (sibling)AWS EC2Azure VMLinux namespacescgroupsseccomp

Links

Related writing

8 posts reference Algocode

Related stack

techs used in Algocode