Reinforcement Learning Hexapod

[blegas78]

Well I haven't been able to get my old video camera charged, so it doesn't look like i will be able to get a video anytime soon. I also haven't given any updates, so I have a few words to say.

Fortunately the project was successful. I was able to implement a learning algorithm with off-board processing, and the hexapod did learn to walk. I did have to constrain each leg to a mere three states/positions per leg though, and it took about four days for learning to almost complete. Unfortunately, I burnt out the pololu servo controller after resetting it by unplugging the power cable and accidentally plugging it in shifted by a pin. This means that I couldn't continue on with the learning, but I built my reward matrix to 99.65% complete, which fortunately was enough to get an efficient walking gait.

It was successful, and it was also the most impressive project I think, which landed me an A in the class and a job in the professor's robotics laboratory. The nice part is that I can continue working on components of the hexapod that were only partially completed in my eyes (I dislike only three states per leg, so I might explore function approximation learning instead). Also, I will be taking another robotics class from him next semester: Intro to Robotics (Maybe I should have had this one first? ). My professor had actually ordered a Pico-ITX board as mentioned in this thread, so I had the chance to install OpenCV on it and try out some vision processing. I am a little disappointed in vision processing which I sort of expecting from the 1.0GHz processor, but I am even more disappointed in the power consumption and heat dissipation of the device. Not very good for a walking mobile robot.

Right before I was about to order one for myself anyway, I found a fit-PC 2 advertised on the web. These devices are actually designed to be "green" which is great since it only uses about 6 watts of power. Also, it sports a 1.6GHz atom processor as well. Though the size of the motherboard is a bit bigger, it includes WiFi and a very slim case, which is much better than the case my professor purchased for the Pico-ITX. Here is a good website showing the size of the fit-PC:

http://www.pcper.com/article.php?aid=718

I also replaced another leg motor with an RX-28, and hope to have all femur servos to be RX-28 motors by the end of summer. Another part in the works is to order a custom circuit board to mount flush on the hexagonal connector in the center of the robot. I do not have much experience though with ordering good-looking circuit boards, I have a schematic and board layout in Eagle, but I am not sure where I should get it printed. Can anyone give me any tips for this, or nay recommendations of who to get it printed from?

[Matt Denton]

Hi,

Sounds like it was a really interesting result, I would love to see it walking if you ever replace the servo driver , I'm soon to embark on a hexapod platform using RX-10 and RX-28 servos, should be fun!

I get my PCB's done at www.pcbcart.com they are in China, and I have had very good results with them so far.

[blegas78]

Get yourself a VIA EPIA PX10000G, 10cm x 7cm - takes a single SO-DIMM upto 1GB.

I built a neural net bot a few years back for World of Warcraft, it used to run at about 2.5gb of RAM with C#, and optimisations for speed rather than memory usage. Making something that just runs a bot on 1GB of ram should be pretty easy, especially with C.

Since I have been working in the robotics lab, I have been playing around with one of the Pico-ITX boards, loaded with Ubuntu 8.10. Considering the size, I must say that I am impressed with the performance after some optimizations of my optic flow code. Unfortunately, the processor only clocks in at 800MHz, so performance is still a little sluggish for machine vision processing.

I have since ordered and received a Fit-PC2 from Compulab. The size is a little bigger, and actually has built in wireless-N card (except that they didn't have room for the second antenna, so it is only G compatible as of now). The input voltage can be between 8-15V too, so no inefficient regulators are needed.

I am currently optimizing the optic flow code to run more efficiently on the Fit-PC2, and I have noticed that the code runs about twice as fast on this little machine than it does on the Pico-ITX. Currently I am configuring the code for the PIC to run computer communication under USB instead of Bluetooth. The next step is to port the Matlab code to C++ so I can run everything on board.

[blegas78]

Hi,

Sounds like it was a really interesting result, I would love to see it walking if you ever replace the servo driver , I'm soon to embark on a hexapod platform using RX-10 and RX-28 servos, should be fun!

I get my PCB's done at http://www.pcbcart.com they are in China, and I have had very good results with them so far.

I look forward to seeing what you can do with the Robotis motors! Will you be using the RX-28 motors for both the femur and tibia on each leg? Thaks for the PCB website.

I think I am going to order shorter tubes for Lynxmotion so that the hexapod isn't radially symmetric. I kind of prefer the more biologically inspired placement of the legs.

[blegas78]

Lately at my job, I have been updating the public website. The current website is pretty awful, and it doesn't even include what the current projects are. I have been adding information including pictures and videos of our current robots to a re-designed site, and a grad student wanted me to include my hexapod. The website isn't finished yet, but I have uploaded a simulation of the hexapod incorporating my opencv based simulation of the hexapod running a walking gait with dynamic foot placement and simple balance gestures.

When I get the fit-pc2 running with all robotis motors, I will bring the hexapod in and record it on video.

Hexapod Simulation

[Matt Denton]

Great simulation! You mention OpenCV, your using OpenCV for the simulation??

Also, what is you job? sounds interesting

[Matt Denton]

I pondered about getting an eddy ARM CPU module: http://www.embeddedmodule.com/

I have started to port the full version of my HexEngine to run on an embedded Linux system. I am a complete beginner to Linux, but did manage to get Debian installed on an old Laptop (Target), and setup Eclipse within Ubuntu on my dev PC. Got an initial version of the Engine ported and working, but I would like to cut down the target system to the bare minimum so it will boot fast... I'm thinking one of these Eddy boards could be ideal, did you end up trying one?

[blegas78]

I am using OpenCV for the simulation. Since I am using it for the optic flow for the camera, it seemed natural to build a simulation out of it. All I do is draw lines on a black image. Since I integrated body roll into the inverse kinematics a couple weeks ago, I essentially copied the rotation matrices to rotate the key points of the simulation. I then ignore one of the axis (the z-axis) and plot a 2D line between the key points.

I am much happier with this compare to using Matlab, as I can run the simulation at 160fps, opposed to 3fps in Matlab. This way I don't need to disable the simulation, as the simulation in OpenCV will not slow me down significantly.

Here is the website where I work. Again, the website is being updated, and should be much better than the current one.

[jakkspar]

One thing I forgot to mention in the above post about the AX-12 motors is the range. I really like the 300 degree range, and also that they can be easily electronically converted into continuous rotation, simply by setting the position limit registers to zero or something. Here is a shot of the mechanical part of the hexapod complete. There's a real fun mess of wires coming out through the center:



Here's another shot of the ugly prototype electronics I very poorly mounted. The next step is to rebuild part of the AX-12 - Matlab PIC interface to work with the Pololu serial controller, then power up it up and get it walking.

i think this kind of prototypes are not finish yet



________________
Rapid prototyping

[blegas78]

One of the greatest things about working in the robotics lab is the ability to use some of their tools. I must say that I have many more tools than the lab since the lab is a little small, but they have a Dimension 3D printer, so they really don't need much. Since I have been upgrading the motors to be purely robotis dynamixel motors, I have felt the need for a redesign of the hexapod, since some components are weak and others don't fit well.

I now have nine AX-12, and six RX-28 motors. I will purchase six RX-10 motor this week ( only $600 to go! ).

Here is a redesign of the hexapod, it is incomplete, but I will print the top plate, bottom plate, femurs, and tibia brackets. All I need is to maybe design some coxa brackets: