Garden Robot Update 12

The long-promised CAD file.  I used LEGO Digital Designer.  This is mainly for record purposes.

Sadly, Garden Robot development might have to come to an end until next summer.  I have a lot of things planned for this machine, not the least of which is switching out the Android phone for a more capable device (Raspberry Pi) which should allow for less latency and hopefully more options when it comes to sensors.  Of course, there's a lot I would like to do for hardware as well, including some electric solenoids to control water flow, fertilizer, etc.

In any case, this project has proven itself highly valuable so far - I've been introduced to:

• C++
• Java
• Javascript
• Jquery
• HTML

And more generally:

• Basic networking (and how not to set up a server...)
• Communication protocol through byte packages
• Bluetooth MAC addresses and handshake
• SIMPLE finite state machine
• User control vs. autonomy

It's been a great project, one that I hope to return to sooner rather than later.  But for now I have a much better feel for what goes into making this sort of telepresent, connected robot, a whole folder in My Documents with code and resources, and a series of blog posts documenting my successes and failures ;)

Garden Robot Update 11

Some images.  First of all, the current state of the GUI.

Chrome:

Chrome (Android):

Safari (IOS):

And the robot itself.  Currently missing the valve and associated water tubing...

The track, nothing more than a 2X4 with a nail in each end, propped up on books:

The robot and phone plugs can be seen from behind.  Currently, I am having it "check on" each plant (designated by the blue strips of tape) at 10 minute intervals.  I was running 1 minute intervals earlier today and haven't had a problem with this version of the program...here's to hoping.

To do:
1.  Get tubing and water container together.
2.  Figure out how to get position data from the robot to the web-page (I spent hours this morning getting bluetooth messages from phone to NXT worked out, now I have to figure out how to do it in reverse)
3.  Get some soil and actually test this thing!!!

Garden Robot Update 10

It's been a full week since the last update.  Quite a few things have changed, but not a ton.  I have continued to make changes to the GUI and have implemented a rudimentary security system.  "Rudimentary" is right...it is, for now, just a prompt box that redirects the window to another page of my choosing (mwuhaha) if you don't put in the right password.

However, the most progress has been on the robot itself.  I have finally constructed revision 1.  For now, it is comprised of 3 motors (move left/right, raise/lower attachment, open/close valve).  It also has 3 sensors (ambient light, moisture, touch sensor to reset position, and light sensor to detect marks on the track and determine position).  I am currently using a 2X4 as a track, with 3 strips of electrical tape to mark out the positions of the plants.

I have also finally gotten around to coding the position control of the robot.  This one was a little bit more tricky...I wanted a system where something (either a user request or a preprogrammed routine) could ask the robot to move to any position, regardless of where it starts.  For now, I have two parts to handle these two different requests.  (1) If the bluetooth mailbox returns a value of 0, the program switches towards a preprogrammed routine.  (2) If the bluetooth mailbox returns a value that is not 0, the user is requesting a change in position.

1. After initial reset, the main task starts another task that runs simultaneously for the rest of the program.  This sub-task simply waits one second, then increments three variables - timer1, timer2 and timer3.  The main task checks to see if any of these variables has exceeded a certain threshold - if so, it calculates the amount needed to move to position 1, 2 or 3 from where it currently is, and passes the MoveRobot subroutine this calculated value as an argument.
2. The main task sets the desired position to the value read out of the bluetooth mailbox.  This value is the user's requested position.  As before, it calculates the amount needed to move to this position and calls the MoveRobot subroutine.  

Significantly, since these two branches are on either side of an if/else statement, they cannot execute at the same time and thus cannot interfere with each other.

The MoveRobot subroutine uses the downwards-facing light sensor to check for the electrical tape markings on the track.  Depending on if the argument is negative or positive, it moves backwards or forwards, incrementing a counter variable as it detects a piece of tape. Once the argument matches the counter, it breaks out of the loop.

Couple things:

1. The direct control I currently have programmed will mess up the position of the robot, which might cause it to become confused (it thinks it is in position 1 when it is actually in position 2, etc.)  To fix this I should probably get rid of direct commands and add a third branch in addition to the above two.  This way, the robot could reset itself after the user finishes controlling it.
2. I still haven't found a way to get the robot to "push" vital information to the server without a user-initiated GET request.  For example, whenever the robot changes position (ex. from 1 to 3) it should cause this data to update on the web page.  Same for valve state and attachment position.
3. Figure out port forwarding - for now the server is only accessible to devices connected to the same network.  However I don't really yet plan on using this outside of the same network...for now.

Wow, that got long.  Pictures/screenshots/code to follow.

Garden Robot Update 9

Some aesthetic updates.  Putting things in a table makes it look a bit better...I guess.  Currently I have implemented control for left and right (rotating motor A in each direction when clicked and stop when released).  The ambient light label refreshes when clicked.

I got the app running on an older Android phone today (running Android 2.2).  I initially had some difficulty with NanoHTTPD throwing a null pointer exception, but fixed that by updating to the latest version of NanoHTTPD.  

On the hardware side, the 350 milliamp converter I hacked into the NXT seems to be working fine for the single-motor rotation, but I picked up a 2.2 amp converter that needs to be placed on.  I want to put in some kind of protection, like the fuse and diode in this example.

To do:

1.  Fix the moisture level command so that it reports the proper values.

2.  Find a way to scale the ambient light command from 0 to 100%.

3.  Start on the actual design of the robot...

4.  Get the 2.2A converter together.  

5.  Figure out how (if?) the NXT should push data to the server.  For example, I think it should display its current position and valve setting.

6.  Put in new controls for open valve and close valve.

Garden Robot Update 8

Quick update.  After WAY too long struggling how to get text info (like sensor readings) from the NXT to the webpage without refreshing, I looked into using Ajax in jquery and it turned out to be an EXTREMELY simple "get" command.  No need for Canvas text, no need to mess with the HTML (all of which, unfortunately, I tried.)  Basically, this in javascript:

$("#refresh").mousedown(function() {
    $.get('lightrefresh', function(data) {
    document.getElementById("lightlabel").innerHTML = ("Ambient Light:
" + data);
    });
    $.get('moisturerefresh', function(data) {
    document.getElementById("moisturelabel").innerHTML = ("Moisture Level:
" + data);
    });
});

(the key is the second argument to the $.get request, as "function(data)" will process whatever data you are getting back)

And in the .java file:

else if(uri.equals("lightrefresh")) {
    Log.w("HelloServer", "lightrefresh");    res = new Response(Response.Status.OK, MIME_PLAINTEXT, "999");}

else if(uri.equals("moisturerefresh")) {
    Log.w("HelloServer", "moisturerefresh");    res = new Response(Response.Status.OK, MIME_PLAINTEXT, "111");}

Of course, later, the "999" and "111" will be replaced with return values from the NXT as I discussed earlier.

To do:
1.  Continue to lay out the GUI.  (It looks so ugly now, I wonder how to make it look any better...?)
2.  Actually make it work with the NXT.  Put in the direct commands to control motors and get input values.
3.  Start to figure out some form of security...I really doubt I know enough to actually encrypt any data but I want at least a form of password-protection.  Like, really annoying password-protection.
4.  Start to design the actual thing...I certainly want some sort of CAD file, but I don't (currently) have SolidWorks, so I should probably look into MLCAD or something.  Or LDD, which I have used extensively in the past.
5.  Figure out where I can actually run this thing.  I need to talk to some people.

Garden Robot Update 7

Decent breakthrough today.

Basically, I finally got the Nxtbotguard streaming solution to work with my current implementation without modifying the Nanohttpd java file.  That's the middle canvas element in the image.  Still not entirely sure how it works, as I have borrowed a fair amount of code.

Also using the Nxtbotguard solution I have implemented the bottom two arrow buttons (as images) which respond as expected when clicked (using Jquery, which seems awesome and I really need to learn more about) Nothing works with the NXT...yet.

The biggest issue is updating text however.  I want to update the ambient light value from the NXT through dynamic text, and I am a bit of a standstill how to do that...currently trying to use Canvas, like the streaming image solution, but it doesn't work.  Yet.

Still a lot to do, but might actually see a bit of progress at last...

Garden Robot Update 6

Quick update.  I know these aren't as frequent as they should be, but hopefully I can cover most of the important things I have done since the last one.

First of all, after a few days, I finally figured out a nagging issue with the existing NXTBotGuard streaming solution.  It turned out that my browser was caching the image, causing it to only update on refresh.  I forced this out by adding this line to the NanoHTTP header:

res.addHeader( "Cache-Control", "no-store");

With this out of the way, the streaming solution worked fine, but I didn't (and still don't) understand it.  Apparently the developer " modified the NanoHTTPD  server to serve files directly out of the compressed assets folder that comes with your app package (apk)."  I don't get how he is doing that, so I took the simpler solution of just reading the index.html file via AssetManager:

try {
    // Open file from Asset Manager
    InputStream inputStream = getApplicationContext().getAssets().open("index.html");    BufferedReader reader = new BufferedReader(new InputStreamReader(inputStream));    String line = "";    while ((line = reader.readLine()) != null) {
        answer += line;    }
    reader.close();
} catch(IOException ioe) {
    Log.w("Httpd", ioe.toString());}


return new NanoHTTPD.Response(answer);

Which works perfectly for html but doesn't seem to work for Javascript, as I discovered tonight, as my index.html file will call the javascript functions as expected when I access it locally, but not when I have it read using the above method.  Grr.  (For future personal reference, this index.html is in the "Code Snippets" folder)

So in general, I think that a quick and dirty method to get all this done would be to just host an ever-changing image from the camera using a simple html tag, and have that consistently update to simulate a video.  This should work using the above method.  Although that (should?) work, the dynamic canvas property of html5 has intrigued me so much that I am dying to use it, as well as learn more about html and javascript (which I would use to update the canvas element, and maybe some other things).

So...now, I am going to look online for any more solutions that I can understand about reading directly from the Asset Manager, or just work on my dirty method in pure html and see if it works.