Julia Sakalus
May 2020
Mr.Robot
Breakdown of content
-
Robot build
-
SerialLink/cloud communication
-
Serial communication
-
Machine Learning
-
Reliability
-
Future Improvements
-
Full Code
Scissor Lift
Securely brings ball up to grabber arms through a series of gears to provide maximum torque
Robot Build
Ball Grabber Arms
Color sensor in the back detects presence of red ball, and grabber arms close. Grabber arms have silicone ends on the end that allow them to deflect and conform better to the shape of ball for a more secure grip. The robotic arm itself has three degrees of freedom, as it can grab the ball, rotate on its axis, and flip the grabber arms 180 degrees.
Power Dial
Allows user to control the speed at which the conveyor belt travels. Has haptic feedback so power dial resists change and comes back to zero after user has made selection
Weighted Ball
A mini surgery had to be performed on the red ball. The plastic ball on its own was not heavy enough to trigger the touch sensor under the cup. A small incision was made, a syringe was used to squirt water inside, and wax was used to seal the wound.
Soldering-from-home
This unexpected bonus resulted from the need to have the the two EV3 bricks communicate with each other over serial, and the blue and yellow wires needed to be flipped to allow this. Trust me I'm an engineer?
SystemLink/Cloud Communication
As part of a virtual Rube Goldberg machine, Mr.Robot communicated with the cloud (SystemLink) to know when is his turn to perform his stunts. He politely waits for "Start09", his personal tag, to be flipped to true before he proceeds to lift the ball and begin his motions. Once the tag is flipped to true, the robot lifts the ball to the grabbers, turns it, twists it, waits for user to give desired power, and then drops it in the cup. If the ball makes it into the cup, the touch sensor on the button is engaged and the next SystemLink tag is turned to true.
Serial Communication
Two EV3 bricks were used to control the motions. One controlled the scissor lift, grabber/robotic arm, and color sensor, and the other controlled the conveyor belt and the two touch sensors. Serial communication was simplified by encoding and decoding the messages through the use of functions. The challenge with serial communication in this project was the continuous two-way communication that needed to take place. Since a lot of the motor code is run in a while loop, I had to make sure that when one EV3 was sending information over serial, the other would read it before it got sent new information and vice versa. I accomplished this by alternating sending and receiving commands for each EV3 and added a wait command in the decode functions to make sure they stayed in sync and the code did not proceed until information was received and read.
Machine Learning
The training code was written to be able to almost entirely train itself. The conveyor belt starts running at its lowest power, and the robot drops the ball at a set initial angle. If the ball doesn’t land in the cup(which would trigger the touch sensor underneath), the conveyor belt runs again at the same power and the robot adjusts the angle at which it drops the ball. The angle at which it drops the ball corresponds to the angle of the motor that is driving the conveyor belt. The robot continues adjusting this angle until the ball lands inside of the cup, and once it does, the power at which the conveyor belt is running and the successful drop angle are recorded and stored in a text file on board the EV3. This data is later used to create a fit which allows the robot to make it into the cup at any range of speeds that the user inputs using the power dial. Once the data is stored, the conveyor belt then increases its power by an increment of ten, and the program continues running. This training is completely autonomous except for having to pass the robot the ball.
Model / Reliability
The good news is the robot is able to reliably pick up the ball off of the scissor lift, turn it around, flip it upside down, receive a power from the user, and run the conveyor belt at that power. This is demonstrated in the video on the bottom right.
To the right is the model created from the gathered training data, and based off of which the robot in the timelapse is making its decisions on. The robot was actively able to adjust the angle at which it dropped the ball but there is no sugarcoating here: this model proved to be 0% reliable across 15 tries to drop the ball in the cup (or 100% reliable at not making it into the cup :D ). This can, however, be explained.
Some of the reasons include slack in the motor and imprecise zeroing. When training the robot, the conveyor belt would push back very far against the push button, causing its starting angle to be lower than it was for the 'real run.' I tried to match the code but even across multiple runs, the angle of release would change and the conveyor belt would keep pushing farther and farther back against the push button (despite zeroing it). The drop angle ended up being extremely inconsistent across multiple trials even in training, despite restarting code between each trial and forcing it to drop at a specific angle. Another factor are the grabber arms themselves, as I had to adjust their initial starting angle manually if the program terminated early, and that caused the motor to run inconsistent angles (if it grabbed the ball tighter it would take them longer to open). Other issues arose with the rigidity of the cup running on the conveyor belt (it was pretty floppy), and the battery of the EV3's, which caused the speed of the conveyor belt to fluctuate (and I only had one power cable).
So is it doomed to never work? Not exactly. There are some things I could do better to make this design stronger, and future iterations more uselful.
Future Improvements
For the design:
Due to somewhat limited resources, the current robot is not as robust as it could be. Instead of a conveyor belt, perhaps a linear bearing attached to a motor belt would be more reliable in achieving desired motion and would not be as wobbly. Some changes in the code could also be made to better control the motors.
Ideally more sensors would be added to detect position of motors, or stepper or servo motors could be used for more precise motion.
For the machine learning:
More tests and trials. Each trial took a while to perform since the robot would start at one angle and slowly increase the angle until it was able to make it into the cup. A more efficient way to do this would be to (after having gathered some initial training data) have the robot continuously learn by reading the information it stores in the text file into the program, guessing the needed angle based on the data stored, and appending to the file with successful trials. The model could also be turned into more of a reinforcement learning model to speed up the process. For this iteration just having more test data to begin with would allow the robot to make better predictions despite the varying conditions.
Full code
Click on suitcase to view files!
There are four folders, two training folders and two regular-run folders. One of the training folders is the for the robotic arm (RubeTraining) and the other for the conveyor belt (conveyorTraining). These are only meant to be used with each other. Note that the robot trained already in position to drop the ball (instead of starting at the scissor lift the whole time).