Scribblybot the Full Story

OK, in my earlier post I promised a full write-up of how scribblybot came to be. What a journey and what hard work!

The Conception

I first conceived of the idea way back in 1999. I was setting up some robotic training cells as a contractor for Ford Motor Company and I thought how cool it would be to get a robot to write some user entered text to a whiteboard. As any robot programmer will tell you, it is trivial to get an industrial robot to write and draw on a whiteboard, in fact it is a very common exercise when learning to teach a robot positional data. To get it to write a string of text however is a just a bit more tricky. For a start you need a way of getting the text into the robot and secondly you need to find a way of dealing with actually writing it to the board. So the idea sat in the back of my mind for about a decade or so during which time happened the whole web 2.0 /social media evolution. I met a lot of stalwarts of the whole social media scene when I moved to Oxford and a friend suggested I went to Oxford Geek Nights, a local meetup of web/IT specialists combined with a jolly good piss-up at the same time! Then the idea came rushing back to me. My idea about a message writing industrial robot would make an excellent user of Twitter! In this way, you are globalising the whole message entry system and Twitter are even kind enough to provide an API to bring you the entire functionality of their service to your own applications.

To set the scene, I have the idea, I’m working at BMW and my friend has a workshop full of robots destined to be used as replacements for faulty units on the line. But it would be pointless to develop something this big without a forum to display it right? Thats when BMW 2010 open day came along! I developed a prototype of just a few letters in my own time and also a software interface to actually send some letters to the robot and showed it to guy at BMW who loved the concept. That was the beginning of scribblybot!

The Challenges

Sending the Text

The first hurdle to overcome in a system like this is how to actually get the text into the robot controller. Various options are open to you…

• Connect an input device via a bus system to the robot controller
• Enter the text via the robot teaching device (teach pendant)
• Have the text sent to the robot via a controlling device such as a PLC

Writing the text

Once you’ve figured out a way to actually get text to the robot the next hurdle to think about is how it should offset each character of the string. It would be possible of course to program each character of the alphabet at every possible position on the board and just send the robot a row, column and character. This approach is way too time consuming given that each character probably takes on average 5-10 minutes to program. Multiply that by the number of printable ascii characters and then multiply again by the number of characters on the board. You’re going to have to trust me (and Mark Lennon) when I tell you that programming the characters once, is completely soul destroying let alone hundreds of times! We ended up using pair programming just to break the monotony and errors which creep in when doing something repetitive like this!

The best way is to program one set of characters and then offset them to the relevant position on the board. This method ensures consistent lettering and reduces the programming time by a factor of the number of letters achievable on the board!

There is also a mechanical difficulty to overcome here. I order to write on a board a pen must apply enough pressure for the ink to flow but not so much as to damage the nib. You can’t have even a small payload industrial robot bearing down on a whiteboard with a pen without pretty much destroying both! Thus a way for the robot to actually hold the pen at a consistent pressure to the board is needed.

Erasing the previous message

Similar to the challenge of holding the pen is how and when to erase the message that is already present on the board. Do you wipe the board with the robot which involves adding a board erase to the pen holder or do you use another method such as feeding the board fabric through an eraser? Should it be a dry wipe or wet wipe which may prevent the pen writing on the board?

Getting messages from twitter

Purely a software challenge this. And to be honest one of the more trivial challenges of the system given the amount of twitter clients out there and the sophisticated twitter API.

The Design

I take the liberty of calling it a design here, it was actually a hack, a long series of trial and error experiments against the constraints of the system!

I had lots of different robots to choose from, Kuka KRC1, KRC2, ABBs etc.. But the robot that I chose to work with was a brand new, small payload ABB IRC5 fitted with a very capable industrial ethernet interface (Profinet to be exact) and a secondary NIC for remote programming and archiving etc. This second NIC (called the service port) also has DHCP and will assign an IP address to any device connected to it. Another nice feature of the new ABB IRC5 range is the number of libraries and functions built into the robot. There is even a ready made socket library for TCP/IP.

I knew all along that I would need a way of manually sending the letters to the robot. It is essential for testing of the various offsets and characters. So to begin with I knocked up a client/server model where my laptop was connected to the robot’s service port via ethernet.

I programmed into the robot a simple server with a protocol which would accept a TCP/IP telegram consisting of the longitudinal and latitudinal offsets and the ascii code for the character to be printed, separated with commas. The robot would wait for a telegram, parse it into the required data-types, offset the robot and write the character to the board. When it was finished it would echo the telegram back to the sender and wait for the next one.

For the laptop test client I wrote a GUI with all of the major parameters in text boxes and with a send functionality to actually send the telegram. The parameters available to experiment with were as follows…

• width and height of each character
• number of characters per line (columns)
• number of lines (rows)
• message to send

This allowed me to experiment with the robot without having to dive back into the code all the time. Eventually of course the GUI disappeared altogether for the final client.

So now I had a computer connected which could control the robot over ethernet, the next step was the actual programming of the characters.

The Mechanical Pen Holder

Originally I had a pen holder which I knocked up out of old bits of pipe I had lying around. This worked OK for the prototype but looked utter garbage and was pretty unreliable. I tried various options for the pen holder before Sergio Ayres stepped up and offered to build a real solution. Sergio is one of those guys who can make absolutely anything, perfectly. Within a day he had made about 5 different versions of the holder and finally settled on the one which we used on the day.

The Robot Positional Data

The first step was to define a work object which described the position of the whiteboard in space relative to the robot. By programming a work object you are basically telling the robot “forget about what direction you think your x,y and z coordinates are, I’ll tell you what they are”. It then means that by offsetting the coordinates in the x and y direction you are following exactly perpendicular to the whiteboard. Without this functionality even with the board mounted exactly in front of the robot you would get discrepancies from one end of the board to the other. This topic would consume an entire book on it’s own so if you want more information about work objects then have a look here.

The robot also has to have a tool centre point (TCP) which tells it where the tip of the pen is in space. Again this is quite a detailed topic and would detract from this writeup, have a look here if you want an explanation.

So I set up the coordinate system for the work object to have it’s origin at the top left of the board with positive x going right and positive y going down. I did this so that it would match a computer screen coordinate system. I actually had a few issues with the work object. The whiteboard was quite large and unless the work object coordinate frame was exactly right the letters tended to slant downwards towards the end of a line of text. This was ultimately solved by Mark Lennon who set the coordinates up with a lazer level.

Once you have the coordinates set up for the work object and TCP all you have to do is figure out a a best fit size for the letters on the board. In our case that was 90mm by 90mm. All the characters are then programmed into a 90mm by 90mm space from the origin of the work object. The client running on the computer calculates all the offsets when it sends each letter based on this 90×90 dimension. so the first letter would be sent with a 0,0 offset the next with 90,0 and so on. All the characters where programmed using linear motions so that we could be sure that the robot would travel one to the other without fouling the whiteboard.

I’ve got to say again that programming each character of printable ascii is an absolute pain. Although each letter doesn’t take long the sheer monotony of doing this makes it seem like an eternity. Mark and I together spent about 2 to 3 days just on this task alone!

The Final Presentation

Once we had the test client running and the robot writing messages, it was time to clean up the whole interface to the thing. The system was getting it’s first outing at BMW open day so we needed to keep the manual input for visitors to write messages on the day. I had a Siemens touch screen PC lying around which I rigged up to robot in place of my laptop test client. At the last minute after a recommendation from a friend (@RadioKate), I decided to moderate all incoming twitter messages. The Siemens touch panel had a dual NIC so I used the second one with an ad-hoc connection to a netbook for moderating the twitter feed. I wrote an application in Java which pulled in messages via twidge and asked for a simple yes/no before sending them on to the robot. I also had it retweet when the message had been written. Incoming messages from both the manual interface and twitter would be queued and written when the robot had finished it’s current message. We also added a webcam to automatically photograph the resulting output from the robot.

The First Outing

On Saturday night, we moved the bot into position at the exhibit and bolted it down. We had to reprogram the work object to overcome inconsistancies which occurred after we had moved it.

About 12000 people came to the Open Day and the robot worked all day without stopping. I didn’t really expect that many people so unfortunately the twitter messages had to take a back seat to those messages put in on the day.

Improvements

The following are things which could be improved given the more time and inclination…

• The message doesn’t wrap properly a word which comes at the end of a line is split in half
• It only does upper-case ascii characters, no unicode or non capital letters
• It doesn’t automatically upload the resulting photo of the message
• When it erases it cleans the whole board, not just what has been previously written

The End

What now for scribblybot? We are going to organise a twitter only day for the bot so that everyone who missed out sending a message on the day can have a go! Hopefully we’ll sort out the webcam to auto upload and tweet the photo back to the message sender. After that scribblybot is going to left waiting in the workshop until he is needed to replace a faulty unit out on the the production line. He’s had his five minutes of fame now!

Thanks to all those people who helped and supported me during this project, most of all…

• My wife and children
• My parents
• @RadioKate for her social media advice
• Mark Lennon for the programming
• Paul Baker for his robot experience and advice

Thank guys!

Three days and counting for scribblybot

Just a quick update on my earlier post about my industrial robot which writes twiiter messages on a whiteboard, then you may.

scribblybot is all packed up now ready to be set up for the open day on Sunday! Remember, you can DM it on the day between 10.30am and 4pm GMT and it will write your message onto a whiteboard At BMW Plant Oxford open day!

In the meantime here are some reactions…

Hack A Day – http://hackaday.com/2010/09/07/re-purpose-industrial-robotic-arms/

Tar Heel Mini Club – http://www.tarheelminis.org/showthread.php?4870-The-robot-that-painted-your-car-MINI-retired-him-into-a-twitter-robot.-scribblybot

Mini Forum – http://www.miniforum.com/forums/showthread.php?t=922

Motoring File – http://www.motoringfile.com/2010/09/09/twitter-robot-tweets-at-bmw-plant-oxford/

Robotistas blog - http://robotistas.blogspot.com/2010_09_01_archive.html

Industrial Twitter Robot

On Sunday 12th September 2010 for one day only (between 10am and 4pm) any Twitter direct messages sent to scribblybot will be read by an industrial robot and written out freestyle on a whiteboard in front of hundreds of visitors to BMW’s Mini Factory in Oxford!

I know that this is a completely useless and redundant thing to spend time developing but it’s my way of bringing Twitter to real life! For once you can post on Twitter and know that people will be watching!

Any messages that are written on the whiteboard will then be re-tweeted to all followers of scribblybot.

A full technical write up will appear here after the event but until then, here is a couple of sneak photo’s of the development! There’s more over at flickr http://flickr.com/gp/mattythorne/f9kY3a.

Google App Engine (for Java) Rundown

Google App Engine

I’ve recently got myself a Java app engine account from google but have held out about reviewing the various features until I had tested it in anger. Well, now I have!

Background Details

I have two applications running on app engine, one is a jsp based website www.oxfordgeekjam.net, which is no more than a static site with a bit of generated markup and one Java servlet application which makes use of several other google APIs. All my development work is done using the opensolaris operating system from Sun (now part of oracle).

What You Get

When you register for a google app engine account (python or java) you get space to host ten applications. Once they are gone you’re on your own! Each one of your apps has a quota which is reset every 24 hours, this quota includes factors such as CPU load, bandwidth etc. Once you breach these, you are on your own again unless you have signed up for googles pay as you go type service, which automatically charges for anything above the standard quotas.

When you register one of your ten applications, you must give it a unique name which becomes it’s application ID. To access your app you can use the URI [applicationID].appspot.com and also your own domain as well if you choose (see below). Its worth noting that once an app has been registered it can not be deleted to make room for another app!

Your application also comes with a “dashboard” which is kind of like a control panel for your app. It gives you information about how much of your quota you are using as well as providing an interface to your server logs. This second point is crucial because one thing you don’t get with the app engine is direct access to the file system space of your application (see deployment below).

Using Your Own Domain Name

The first site is hosted using its own domain name (rather than the default *.appspot.com).  To use my own registered domain name, which I bought through a third party vendor rather than google, was a bit fiddly to begin with but now I know the process should be a cinch. To use your own domain you must have the ability to change your own DNS settings, in particular the CNAME records. Most domain vendors will allow you to do this as did eurodns with whom mine was registered with. You also have to register with google apps (yes, another google service to register with) to prove domain ownership and link your domain to your google “web master tools”. After that you simply need to click on versions in you app dashboard and link the domain to your app.

One downside of the process, and one that is causing quite a stir with other app engine developers, is that you cannot have a naked domain resolve to your application. That is to say, if I registered thebestdomainever.com the URI as it stands can NOT resolve to your app! www.thebestdomainever.com or anything.thebestdomainever.com would be fine though! Of course the way to get around this is to redirect the naked domain to the google acceptable domain, but I understand some registrars don’t allow this, again I was lucky, mine did!

Coding the Application

For vanilla jsp applications this is no more difficult than coding the pages and bundling them with the xml deployment descriptor and a couple of settings files in standard WAR format. The process is somewhat more difficult when it comes to writing servlets and using regular java classes.

There is a plugin for the eclipse IDE which is supposed to mange the coding and build process for you but I haven’t used it therefore cannot comment! I used netbeans (as I always do) and managed to configure the IDE relatively easily with a bit of  help from the documentation, and using apache ant to manage the build. I won’t go into too many details unless somebody asks for it but it basically involves downloading the app engine SDK in zip format (which rolls in at a hefty 20Mb)  and importing the com.google.appengine libraries. App engine applications use the Java Servlet Standard so most developers should feel right at home.

The only thing to watch out while coding is that all applications run in a sand box with a restricted white-list of java classes which are allowed to be used, so you may want to check the dependencies of your favorite libraries before using them!

Deploying the Application

This is the part of the process which I found really pleasantly surprising. The testing and deployment process worked (for me) straight out of the box!

Given the following constraints I really appreciate this level of detail…

1. I’m using opensolaris
2. I was using a really early version of the app engine API
3. The SDK is a single download regardless of whether you use a mac, linux or windoze

To run the app using the localhost as a testing server was as simple as issuing the command…

dev_appserver.sh [war-location]

to the shell.

Uploading the app to your app engine space is similarly easy…

appcfg.sh update [war-location]

One thing to note is that you must make sure that the settings file appengine-web.xml is in the WEB-INF folder and points to your application ID.

Once your app has uploaded, it has gone. You have no way of retrieving it back as there is no access to your application’s file space. This even includes access by the app itself (for obvious reasons).  If you do need persistence and serialisation then you are encouraged to use google’s datastore, but this comes with quota limits.

Quotas

Each application has quotas which are reset every 24 hours. These quotas include CPU time, bandwidth (both ways), Mail and deployments, among others. If you go for google’s premium service then once you reach these quota limits then anything above the quota is charged for, if you don’t then app will not function for the part which requires the quota. There are also hard limits which even premium customers can’t breach. The ten application limit per app engine account is one.

Two of the quota limits are for google datastore api usage and URLFetch usage. So even if your app is only making use of google provided services then you are still limited to amount of traffic you are allowed.

Conclusions

Things I like

• The testing and uploading process is a doddle to use
• Once you’ve set up your IDE coding and deploying is also easy
• The service is free which makes the quotas quite generous
• Using you own domain is relatively easy and can be done any time before or after the app is finished

Things I Dislike

• Why oh why can’t I delete an application once I am done with it?
• Why can I only have ten applications, especially considering the above?
• Why am I penalised by quota limits for google’s own services? Shouldn’t they encourage use of their own services rather than those of third parties?

I hope this post is of some value to some of you, please feel free to comment and if you would like me to send you my netbeans and/or jsp development templates, I am more than happy to do so!

Cheers,

Matty

Resources

http://code.google.com/appengine/ – The Google App Engine

http://www.google.com/apps/intl/en/business/index.html – Google apps for business

http://java.sun.com/j2ee/tutorial/1_3-fcs/doc/WCC3.html – The WAR packaging standard

http://ant.apache.org/ – Apache ANT

http://java.sun.com/products/servlet/ – The Java servlet standard

http://code.google.com/appengine/docs/java/jrewhitelist.html – The Java class app-engine white-list

Oxford Geek Jam

So the first Oxford Geek Jam is fast approaching with just five days to go until the event! Had loads of great feedback from folks especially following my rather poor pitch at Oxford Geek Night 12!

In many ways my low standard of my pitch served me well because it meant that people made the effort to come and speak to me in person to get more details about the event and take away a moo mini card!

So far people have kept their coding ideas and potential projects close to their chest although I have heard a few ideas that people will be bringing along! Plus I have one or two of my own.

Not sure how many people are coming yet, we have twelve confirmed, but we really need to keep this first Geek Jam as informal as possible to minimise the impact to our hosts the Jam Factory. Maybe if it’s a roaring success we can go more organised with the next one and provide a private venue/catering etc..

I’m hoping to see a good mix of people and skills for this event and I will reiterate, truly everybody is welcome.

All that remains now is to say, see you on the day!

Oxford Geek Jam, 10:30am, Sunday 7th June, Jam Factory, Oxford

Without the wind, the grass does not move. Without software, hardware is useless – Geoffrey James, The Tao of Programming

Old meets New!

Ok, I just finished an unusual project for a client today. Specs were roughly as follows…

1. collect data daily from an embedded computer
2. legacy interface of computer is DDE via a propriatory gateway app running on a windoze box
3. Take said data and visualise it on the company intranet in the form of a graph

This really is old meets new here as I have no choice but to use the legacy DDE link but then it needs visualising on the net! There are several ways to do this and I know I will get shouted at by someone, but this is the way I opted for given the time constraints!

The Daemon Process

I created a daemon process in c++ which parses an XML configuration file containing the the DDE topic and the various data to be collected (re-usable of course by changing only the XML). Once a day the data is collected via DDE via the gateway app and is dropped into a SQL database instance. The daemon process lives and breathes on the same windoze box as the gateway app.

The Webserver

This is the easy bit! A quick Java class to run in Tomcat 6 to pick up the data from the database and visualise it in a graph applet! Incidentally, if you need a simple, does-what-it-says-on-the-tin, open source Java graph library, then look no further than the JavaTao, Java Charts library!
Dont you just love hacking together systems to include legacy gear? It’s not forward thinking, but it does make you think, dde has after all been with us since windows 2.0 way back in 1987!

All in all I spent about 10 hours designing, coding and testing this solution, that doesn’t include my midnight chinese takeaway though!

Cheers,

Matt

Resources

Java Charts JavaTao Charts Library

Oxford Geek Jam 7th June 2009

Getting excited now! We’re less than a month away from the very first Oxford Geek Jam!

So what is Oxford Geek Jam? It’s an event where like minded professionals (and interested parties) get together socially and put their collaborative thinking caps on to design write or discuss new coding ideas/projects. We came up with the idea after BarcampApacheOxford as the few remaining stragglers sat, chatted and listened to the incredibly talented offbeat song smith Ben Walker!

We have a great pool of talent attending and the event is generating a lot of interest. Coming along are Oleg Lavrovsky from Cisco, Ben Werdmuller of Elgg and Curverider fame as well as Matt Westcott the unofficial king of Javascript, amongst many notable others!

We have a date (7 June) and a place (Oxford Jam Factory) all that remains now is for me to pitch the event at Oxford Geek Night 12, armed with my moo.com mini cards promoting the event! Then just sit back and see who arrives on the day!

Looking forward to seeing you there!

Matt

Resources :

Oxford Geek Jam

BarcampApacheOxford

Ben Walker Ben Walker / I hate mornings

Oxford Jam Factory

Oxford Geek Night 12

moo.com online printing company

Compiling the UNP Libs Under Opensolaris

[REPOST FROM DECEMBER 2008]

Having recently changed my changed my operating system to Opensolaris I had to re-compile the unp library from (sadly the late) Richard Stevens definitive text Unix Network Programming.
Following the instructions provided with the library source, resulted in various compilation errors rendering the library impossible to compile. I have seen many posts on this subject but no definitive how-to to fix the problem. As Stevens himself points out…

“[He does] *NOT* have time to help everyone port the code to different environments”.

I can however help when it comes to Opensolaris build 2008.11!

To compile the library take the following steps (the steps in bold are the extra bits you need for Opensolaris)…

1. Get the file unpv12e.tar.gz (which is the source distribution of the library)
2. gunzip -c unpv12e.tar.gz | tar xvf – # Unpack the archive into a location of your choice
3. cd unpv12e
4. Open the file ‘configure’ in the text editor of your choice and change line 4374 to be the same as line 4372 (overwrite the existing line)
5. ./configure # which should sort out the implementation differences
6. cd lib # build the basic library that all programs need
7. Open the file ‘unp.h’ in the text editor of your choice and comment out lines 114 to 117
8. make
9. cd ../libfree # continue building the basic library
10. make
11. cd ../libgai # the getaddrinfo() and getnameinfo() functions
12. make
13. cd ../libxti # opensolaris supports XTI
14. make # opensolaris supports XTI

You should now be good to go! Just to make sure, try to build and run the intro project from the book…

cd ../intro # build and test a basic client program
make daytimetcpcli
./daytimetcpcli 127.0.0.1 # run the program

I must say that this book is the definitive text on the subject and I can’t recommend it heartily enough.

Happy hacking,

Matt

Resources…
Read more about the author Richard Stevens.
unpv12e.tar.gz – The original file from the book
unpprecomp.tar.gz – My precompiled library based on the instructions above for Opensolaris build 2008.11 for those who cant be arsed to do it themselves!

Hello Everybody!

Hello everyone!

I’ve abandoned my old blog because it became stagnent, mostly due to my idleness! This new blog here at word press is my attempt to start afresh!

Having in said that, some of my instructional posts from my original blog will be ported over here for new readers, so there may be a flurry of activity to begin with!

I always welcome comments and suggestions.

Cheers,

Matt