Scribblybot the Full Story

OK, in my earlier post I promised a full write-up of how scribblybot came to be. What a journey and what hard work!

The Conception

I first conceived of the idea way back in 1999. I was setting up some robotic training cells as a contractor for Ford Motor Company and I thought how cool it would be to get a robot to write some user entered text to a whiteboard. As any robot programmer will tell you, it is trivial to get an industrial robot to write and draw on a whiteboard, in fact it is a very common exercise when learning to teach a robot positional data. To get it to write a string of text however is a just a bit more tricky. For a start you need a way of getting the text into the robot and secondly you need to find a way of dealing with actually writing it to the board. So the idea sat in the back of my mind for about a decade or so during which time happened the whole web 2.0 /social media evolution. I met a lot of stalwarts of the whole social media scene when I moved to Oxford and a friend suggested I went to Oxford Geek Nights, a local meetup of web/IT specialists combined with a jolly good piss-up at the same time! Then the idea came rushing back to me. My idea about a message writing industrial robot would make an excellent user of Twitter! In this way, you are globalising the whole message entry system and Twitter are even kind enough to provide an API to bring you the entire functionality of their service to your own applications.

To set the scene, I have the idea, I’m working at BMW and my friend has a workshop full of robots destined to be used as replacements for faulty units on the line. But it would be pointless to develop something this big without a forum to display it right? Thats when BMW 2010 open day came along! I developed a prototype of just a few letters in my own time and also a software interface to actually send some letters to the robot and showed it to guy at BMW who loved the concept. That was the beginning of scribblybot!

The Challenges

Sending the Text

The first hurdle to overcome in a system like this is how to actually get the text into the robot controller. Various options are open to you…

• Connect an input device via a bus system to the robot controller
• Enter the text via the robot teaching device (teach pendant)
• Have the text sent to the robot via a controlling device such as a PLC

Writing the text

Once you’ve figured out a way to actually get text to the robot the next hurdle to think about is how it should offset each character of the string. It would be possible of course to program each character of the alphabet at every possible position on the board and just send the robot a row, column and character. This approach is way too time consuming given that each character probably takes on average 5-10 minutes to program. Multiply that by the number of printable ascii characters and then multiply again by the number of characters on the board. You’re going to have to trust me (and Mark Lennon) when I tell you that programming the characters once, is completely soul destroying let alone hundreds of times! We ended up using pair programming just to break the monotony and errors which creep in when doing something repetitive like this!

The best way is to program one set of characters and then offset them to the relevant position on the board. This method ensures consistent lettering and reduces the programming time by a factor of the number of letters achievable on the board!

There is also a mechanical difficulty to overcome here. I order to write on a board a pen must apply enough pressure for the ink to flow but not so much as to damage the nib. You can’t have even a small payload industrial robot bearing down on a whiteboard with a pen without pretty much destroying both! Thus a way for the robot to actually hold the pen at a consistent pressure to the board is needed.

Erasing the previous message

Similar to the challenge of holding the pen is how and when to erase the message that is already present on the board. Do you wipe the board with the robot which involves adding a board erase to the pen holder or do you use another method such as feeding the board fabric through an eraser? Should it be a dry wipe or wet wipe which may prevent the pen writing on the board?

Getting messages from twitter

Purely a software challenge this. And to be honest one of the more trivial challenges of the system given the amount of twitter clients out there and the sophisticated twitter API.

The Design

I take the liberty of calling it a design here, it was actually a hack, a long series of trial and error experiments against the constraints of the system!

I had lots of different robots to choose from, Kuka KRC1, KRC2, ABBs etc.. But the robot that I chose to work with was a brand new, small payload ABB IRC5 fitted with a very capable industrial ethernet interface (Profinet to be exact) and a secondary NIC for remote programming and archiving etc. This second NIC (called the service port) also has DHCP and will assign an IP address to any device connected to it. Another nice feature of the new ABB IRC5 range is the number of libraries and functions built into the robot. There is even a ready made socket library for TCP/IP.

I knew all along that I would need a way of manually sending the letters to the robot. It is essential for testing of the various offsets and characters. So to begin with I knocked up a client/server model where my laptop was connected to the robot’s service port via ethernet.

I programmed into the robot a simple server with a protocol which would accept a TCP/IP telegram consisting of the longitudinal and latitudinal offsets and the ascii code for the character to be printed, separated with commas. The robot would wait for a telegram, parse it into the required data-types, offset the robot and write the character to the board. When it was finished it would echo the telegram back to the sender and wait for the next one.

For the laptop test client I wrote a GUI with all of the major parameters in text boxes and with a send functionality to actually send the telegram. The parameters available to experiment with were as follows…

• width and height of each character
• number of characters per line (columns)
• number of lines (rows)
• message to send

This allowed me to experiment with the robot without having to dive back into the code all the time. Eventually of course the GUI disappeared altogether for the final client.

So now I had a computer connected which could control the robot over ethernet, the next step was the actual programming of the characters.

The Mechanical Pen Holder

Originally I had a pen holder which I knocked up out of old bits of pipe I had lying around. This worked OK for the prototype but looked utter garbage and was pretty unreliable. I tried various options for the pen holder before Sergio Ayres stepped up and offered to build a real solution. Sergio is one of those guys who can make absolutely anything, perfectly. Within a day he had made about 5 different versions of the holder and finally settled on the one which we used on the day.

The Robot Positional Data

The first step was to define a work object which described the position of the whiteboard in space relative to the robot. By programming a work object you are basically telling the robot “forget about what direction you think your x,y and z coordinates are, I’ll tell you what they are”. It then means that by offsetting the coordinates in the x and y direction you are following exactly perpendicular to the whiteboard. Without this functionality even with the board mounted exactly in front of the robot you would get discrepancies from one end of the board to the other. This topic would consume an entire book on it’s own so if you want more information about work objects then have a look here.

The robot also has to have a tool centre point (TCP) which tells it where the tip of the pen is in space. Again this is quite a detailed topic and would detract from this writeup, have a look here if you want an explanation.

So I set up the coordinate system for the work object to have it’s origin at the top left of the board with positive x going right and positive y going down. I did this so that it would match a computer screen coordinate system. I actually had a few issues with the work object. The whiteboard was quite large and unless the work object coordinate frame was exactly right the letters tended to slant downwards towards the end of a line of text. This was ultimately solved by Mark Lennon who set the coordinates up with a lazer level.

Once you have the coordinates set up for the work object and TCP all you have to do is figure out a a best fit size for the letters on the board. In our case that was 90mm by 90mm. All the characters are then programmed into a 90mm by 90mm space from the origin of the work object. The client running on the computer calculates all the offsets when it sends each letter based on this 90×90 dimension. so the first letter would be sent with a 0,0 offset the next with 90,0 and so on. All the characters where programmed using linear motions so that we could be sure that the robot would travel one to the other without fouling the whiteboard.

I’ve got to say again that programming each character of printable ascii is an absolute pain. Although each letter doesn’t take long the sheer monotony of doing this makes it seem like an eternity. Mark and I together spent about 2 to 3 days just on this task alone!

The Final Presentation

Once we had the test client running and the robot writing messages, it was time to clean up the whole interface to the thing. The system was getting it’s first outing at BMW open day so we needed to keep the manual input for visitors to write messages on the day. I had a Siemens touch screen PC lying around which I rigged up to robot in place of my laptop test client. At the last minute after a recommendation from a friend (@RadioKate), I decided to moderate all incoming twitter messages. The Siemens touch panel had a dual NIC so I used the second one with an ad-hoc connection to a netbook for moderating the twitter feed. I wrote an application in Java which pulled in messages via twidge and asked for a simple yes/no before sending them on to the robot. I also had it retweet when the message had been written. Incoming messages from both the manual interface and twitter would be queued and written when the robot had finished it’s current message. We also added a webcam to automatically photograph the resulting output from the robot.

The First Outing

On Saturday night, we moved the bot into position at the exhibit and bolted it down. We had to reprogram the work object to overcome inconsistancies which occurred after we had moved it.

About 12000 people came to the Open Day and the robot worked all day without stopping. I didn’t really expect that many people so unfortunately the twitter messages had to take a back seat to those messages put in on the day.

Improvements

The following are things which could be improved given the more time and inclination…

• The message doesn’t wrap properly a word which comes at the end of a line is split in half
• It only does upper-case ascii characters, no unicode or non capital letters
• It doesn’t automatically upload the resulting photo of the message
• When it erases it cleans the whole board, not just what has been previously written

The End

What now for scribblybot? We are going to organise a twitter only day for the bot so that everyone who missed out sending a message on the day can have a go! Hopefully we’ll sort out the webcam to auto upload and tweet the photo back to the message sender. After that scribblybot is going to left waiting in the workshop until he is needed to replace a faulty unit out on the the production line. He’s had his five minutes of fame now!

Thanks to all those people who helped and supported me during this project, most of all…

• My wife and children
• My parents
• @RadioKate for her social media advice
• Mark Lennon for the programming
• Paul Baker for his robot experience and advice

Thank guys!

Advertisement