Project 02 · June to August 2026
5-DOF Autonomous Robotic Arm
A robot built to pick up an object at any location in a given 3D space, with a 100% ground pick rate and an 80% elevated pick rate, running off MATLAB and Python and driving to the serial bus driver.
- Ground pick success
- 100%
- Elevated pick success (8/10)
- 80%
- Picks anywhere it can reach
- 3D
- Degrees of freedom
- 5
- Goal
- The inspiration for this project stemmed from when I had seen someone run a robot via machine learning and manual training in tandem. I was interested to see if I could capture the same results, running purely off of model data and feedback loop simulation.
- How
- The two pose system solves for a target anywhere inside the arm's reachable volume, at any height, without being told how high the object is sitting.
- Result
- We brought our held-out error down from the CAD model's 73.1 mm to 6 mm, which at the ground pick level left us with still a 100% success rate.
- Open
- Hand-eye calibration never came together. Every solve we ran put the camera 80 to 100 mm from the wrist, against the 24 mm we measured by hand.
Results
The three clips below are sample points and not the limit of where the arm can reach.
Ground-level pick
Physical arm (iPhone)
Medium-height pick
Physical arm (iPhone)
OpenCV detection + sim overlay
High pick
Physical arm (iPhone)
OpenCV detection + sim overlay
The descent moves in slow jogs on purpose. Clips are sped up 9× for viewing.
What fires on every step of the descent
The descent is not one command. It is a loop that re-measures, re-solves and re-commands about every eight millimetres of travel.
Camera
OpenCV brick detection, 2 pose vectors
MATLAB / Simscape
Inverse kinematics over the surveyed chain
Serial bus
Paced, safety-capped servo commands
Physical arm
Closed-loop visual servo descent
- 01The camera finds the brick and we shorten the error vector
- 02The pixel step becomes a Cartesian target in the base frame
- 03Python hands the target to MATLAB and MATLAB solves the inverse kinematics
- 04Joint angles turn into paced servo commands
The 2 pose system, and why we never have to know the height
One camera looking at one frame cannot get depth. Two rays from two known camera positions intersect at one point in space, and that intersection is the brick's actual 3-D position with the height included. The arm ends up being its own stereo pair, using its own forward kinematics to know how far apart the two viewpoints are.
What the rebuild bought
One jog moved the claw 30 mm where the model had predicted 13.8, off by a factor of 2.17. We instead decided to opt for a physical test at different points, measuring joint offsets with a ruler.
Forward kinematics accuracy, checked on held-out touch points
held-out error
held-out error, zero fitting
That is a 12× reduction in positioning error, and it came out of ten ruler measurements with no curve fitting and no reliance on the CAD geometry.
Simulation validation
Everything was solved in MATLAB and checked against a mesh view of the arm before a single part got printed.
Simscape Multibody, early IK validation, 9× speed
Fabrication & build
I modelled the arm in SolidWorks around the servos themselves.
Literature referenced
We did not improvise the hand-eye calibration or the kinematic identification work.
- 01H. H. Chen, “A screw motion approach to uniqueness analysis of head-eye geometry,” CVPR, 1991.
- 02Y. Sun and J. M. Hollerbach, “Observability index selection for robot calibration,” IEEE ICRA, 2008.
- 03F. C. Park and B. J. Martin, “Robot sensor calibration: solving AX = XB on the Euclidean group,” IEEE Transactions on Robotics and Automation, 10(5), 1994.
- 04V. V. Fedorov, Theory of Optimal Experiments, Academic Press, 1972 (Fedorov exchange algorithm).
- 05H. Zhuang, S. H. Motaghedi, and Z. S. Roth, “Robot calibration with planar constraints,” IEEE ICRA, 1999.
- 06R. Y. Tsai and R. K. Lenz, “A new technique for fully autonomous and efficient 3D robotics hand/eye calibration,” IEEE Transactions on Robotics and Automation, 5(3), 1989.
What I took away
This project taught me a lot about the importance of not only having the end to end vision early on, but to also be flexible to work within that vision. I used Claude Code to scaffold both the MATLAB framework and the detection framework, and this project really taught us how to work effectively with AI where it was needed, and work things from scratch without the reliance of AI when we had to.
Built with Krish Vatsa, who handled the OpenCV work we had outlined and found the literature behind the vector-shortening model, while I cleaned up the MATLAB. Calibration, fabrication, and closed-loop control were done together, and integrating the three systems was the hardest and longest part of the project.