⚽️ RoboFootball – CL-RIE Festival of AI 2025

Project Overview

This 2025 project is an interactive RoboFootball game built for the CL-RIE Festival of AI. It features a hybrid human-AI control scheme:

• Two human players control four ESP32 robots (two each) using mobile phone interfaces.

• An AI opponent controls two additional robots.

• A large TV display serves as a Heads-Up Display (HUD).

• An overhead camera tracks the game in real-time.

• All robots are controlled wirelessly via a FastAPI backend using WebSocket communication.

---

System Architecture

Backend (/api) – FastAPI

• Acts as the central WebSocket hub.

• Handles:

  • /ws/mobile: mobile controller connections

  • /ws/tv/onloading: TV HUD status updates

  • /ws/robotcontrol: ESP32 robot receivers

  • /ws/mobilecontrol: simulator-to-robot command routing

📦 Dependencies: See /api/requirements.txt

---

Frontend (/ui) – Nuxt.js + Vue.js

• Web-based UI for:

  • Mobile controllers

  • TV HUD display

• Real-time interactions over WebSocket.

Configured via /ui/package.json

---

Simulation (/simulation)

A modular Python simulation environment that includes:

• Camera calibration and perspective transforms

• Real-time detection of:

  • Robots (via AprilTags)

  • Ball (via HSV color filtering)

• Physics engine and game state logic

• AI opponent logic via:

  • Behaviour Tree (open-source)

  • DQN agent (FootballFX 8 – proprietary)

• WebSocket client to send commands to physical robots

Key Files:

File	Purpose
setup.py	Camera calibration; generates .npz files
hsv_tuner.py	Interactive HSV tuning for ball detection
main.py	Game loop, AI logic, WebSocket relay
xform_sandbox.py	Translates AI world-targets into joystick input
ws_client.py	Sends joystick commands via WebSocket
KILL.py	Emergency stop for all robots

---

Robot Firmware (/esp32)

• Arduino C++ code for omni-directional ESP32 robots.

• Connects to WiFi and establishes a WebSocket to /ws/robotcontrol.

• Interprets incoming joystick/rotation commands.

To configure:

const char* ssid = "YOUR_WIFI";
const char* password = "YOUR_PASSWORD";
const char* websockets_server_host = "API_SERVER_IP";

---

Line Camera (/linecam)

• Experimental prototype for a CCTV-style sideline view.

• Mimics a football broadcast for large screen projection.

• Scripts: cctv_fest24.py, test_cctv.py.

---

Scripts & Utilities (/scripts)

• Deprecated pathfinding tools (raj_bfs.py, oze_astar.py)

• Hardware test firmware (*.ino)

• See /scripts/NOTICE.txt for legacy context.

---

Running the System

Basic Development Setup

# 1. Calibrate overhead camera
python setup.py

# 2. Tune HSV for yellow ball detection
python hsv_tuner.py

# 3. Paste HSV thresholds into main.py

# 4. Start backend and frontend (dev mode)
npm run dev --prefix ui & uvicorn api.main:app --reload

# 5. Flash and configure ESP32 robots

# 6. Run simulation
cd simulation/
python main.py

# 7. Emergency stop (all robots)
python KILL.py

---

AI Module: FootballFX 8 (DQN Agent)

While the public system uses a Behaviour Tree (BT) for AI logic, a proprietary deep reinforcement learning model – FootballFX 8 – was used during the actual event.

Highlights:

• Based on DQN, augmented with:

  • Transformers

  • Dense blocks

  • CNN layers

• Inputs: game state, vision data (AprilTags, ball position)

• Outputs: world-space movement targets

• Trained to exhibit high-level football tactics

Placeholder API: dqn_football_fx8_api.py

• Provides the interface structure for FootballFX 8

• Implements a dummy get_action() method

• Checks for the presence of footballfx8.bin (does not use it)

• Allows future replacement of BT with RL agents

Integration Example (in main.py):

from dqn_football_fx8_api import FootballFX8_DQN

# In Player.__init__
self.dqn_agent = FootballFX8_DQN(model_weights_path=".")
self.is_ai_driven_by_dqn = True

# In Player.update_ai
if self.is_ai_driven_by_dqn:
    target = self.dqn_agent.get_action(game_state_for_bt, self)
    self.primary_steering_target_m = target

---

Docker Support

To containerise and run the backend:

docker-compose up --build

Uses:

• docker-compose.yml

• Dockerfile.backend

---

Directory Structure 

/
├── api/              # FastAPI backend
├── ui/               # Nuxt.js frontend
├── esp32/            # Arduino robot firmware
├── simulation/       # Simulation engine & AI
├── linecam/          # CCTV-style side camera
├── scripts/          # Test tools and legacy code
├── docker-compose.yml
└── README.md

Acknowledgements

Initial FastAPI + Vue structure was inspired by work from Diogo Miguel:

• Github Repo

• EnhanceFX™