1kW tank platform pulling a loaded metal cart around the homestead.

Text your robot from your phone. It understands natural language, moves autonomously, and reports back.

Your Phone (iMessage)
    |
    v
Mac Mini (message relay)
    |
    v
Jetson Orin Nano (robot-brain/brain.py)
    ├── Claude API parses natural language
    ├── Plans movement from sensor data
    ├── LIDAR + ultrasonic obstacle avoidance
    └── Sends motor commands via WebSocket
            |
            v
      VPS Server (command routing)
            |
            v
      ESP32 -> Teensy 4.1 -> Motors

The brain is also controllable from the web UI with a BRAIN panel:

• Quick buttons: FWD, BACK, LEFT, RIGHT, EXPLORE, HOME, STOP
• Text input for natural language commands
• Live status: position, heading, obstacles, battery

# Start the brain on Jetson
cd robot-brain && bash start.sh

# HTTP endpoints (port 5000)
curl -X POST localhost:5000/command -d '{"action":"forward","value":3}'
curl -X POST localhost:5000/text -d '{"text":"go forward 6 feet"}'
curl localhost:5000/status

Real-time photorealistic 3D environment mapping using sensor fusion

PTZ Cameras (2x)         RPLidar A1M8 (360°)
      |                        |
      v                        v
Depth Anything V2        Laser Point Cloud
(Mac M1 GPU)             (8000 pts/sec)
      |                        |
      +--------+-------+------+
               |
        Point Cloud Fusion
        • Voxel grid (10cm cells)
        • Dynamic vs static classification
        • Wall confirmation (3+ observations)
        • 1M point photorealistic output
               |
               v
        Three.js 3D Visualization
        (live in browser)

601-class real-time object detection on Jetson Orin Nano

• YOLOv8n with TensorRT GPU acceleration (~11ms per frame)
• Indoor mode: furniture, appliances, household items
• Outdoor mode: vehicles, wildlife, landscape
• Living mode: people, animals, threats (triggers safety stops)
• Detections overlay on live camera feeds in browser

┌──────────────────────────────────────────────────────────────┐
│                                                                │
│   Phone ──► Mac ──► Jetson (Robot Brain)                      │
│                        ├── YOLOv8 Detection                   │
│                        ├── LIDAR Streaming                    │
│                        └── Autonomous Navigation              │
│                              |                                 │
│   Browser ◄──► VPS Server (WebSocket Hub) ◄──► ESP32          │
│                     ├── Command routing          |             │
│                     ├── Frame relay          Teensy 4.1        │
│                     └── Brain control         ├── Modbus       │
│                              |                ├── Sensors      │
│                         Mac Mini M1           └── Motors       │
│                         └── Depth Anything V2                  │
│                         └── 3D Mapping                         │
│                                                                │
└──────────────────────────────────────────────────────────────┘

CemaniHomesteadRobot/
├── robot-brain/               # Autonomous movement + text control + Claude AI
├── teensy-robot/              # Motor control, Modbus, sensors, safety
├── esp32-robot-controller/    # WiFi/BT bridge, Xbox controller
├── vps-server/                # Web UI, WebSocket hub, command routing
├── jetson-object-detection/   # YOLOv8 TensorRT + autonomous navigation
├── jetson-lidar/              # RPLidar A1 streaming
├── mac-visualizer/            # Depth Anything V2, 3D mapping pipeline
├── mac-camera-relay/          # PTZ camera control relay
└── docs/                      # Diagrams, screenshots

This repo was scrubbed of secrets before being made public. If you clone it and try to run it as-is, you'll see placeholder values where real network addresses and credentials used to be. You need to replace them with your own before anything actually connects to anything.

launch_map1.sh now reads VPS_HOST and JETSON_HOST from environment variables. Set them before running:

export VPS_HOST="root@1.2.3.4"
export JETSON_HOST="jetson@192.168.1.31"
bash launch_map1.sh

And set up SSH key authentication for both hosts — the original script shipped with sshpass -p 'jetson' (NVIDIA's default Jetson password), which is exactly the kind of pattern that should never be in a public repo. Generate SSH keys and ssh-copy-id them to both hosts before running anything.

The .gitignore already excludes real secret files — you'll see *.example stubs for each of these but never the real thing:

• **/credentials.h (ESP32 WiFi credentials)
• mac-camera-relay/config.json, jetson-camera-relay/config.json, jetson-lidar/config.json, jetson-object-detection/config.json (camera passwords)
• vps-server/auth.json (VPS login)
• .env, .env.*
• *.pem, *.key, id_rsa* (SSH keys)
• private/, secrets/

If you add new secrets while working on your fork, put them in a file that matches one of those patterns — or add a new line to .gitignore — so they stay out of git.

• Tank drive with Xbox controller + web joystick
• Dual PTZ camera streaming with depth overlay
• 601-class AI object detection (YOLOv8 + TensorRT)
• 360° LIDAR with 3D point cloud visualization
• Hybrid 3D mapping (LIDAR + monocular depth fusion)
• Encoder-based dead reckoning odometry
• Autonomous mapping with obstacle avoidance
• Robot Brain with text message control from phone
• Claude AI natural language command parsing
• Distance-based movement (go forward X feet)
• Web command center with brain control panel
• Indoor SLAM occupancy grid mapping
• Semantic map (named zones, object tracking)

• IMU upgrade (BNO085 - accelerometer + gyroscope)
• Intel RealSense D455 depth camera
• A* path planning with obstacle map
• SLAM loop closure
• Auto-docking charging station
• Predator detection alerts (text when bear/coyote seen)
• Patrol route scheduling (time-based rounds)

• Robotic arm (scoop poop, pour chicken feed, grab objects)
• Dog walking with leash tension control
• Chicken coop automation (feeding, door open/close)
• Voice commands via phone
• Multi-robot coordination
• Train engine body shell for giving kids rides

MIT