10.31.2014

Running Latest Go on Cloud9 IDE

Tired of carrying my older mac laptop, I gave Clound9 another look so I can do Go (on the Go :) with my old chromebook.  Cloud9 has integration with GitHub and Bitbucket.  So pulling your Go project directly into your cloud9 environment is a breeze.

Cloud9 already supports the Go environment setup.  When you build your Cloud9 workspace, it will detect your project as Go project and setup a machine pre-configured with Go build tools. But, it will be an older version of Go.  Since Cloud9 gives you access to terminal access to a linux machine, that is not a problem, follow the next steps to get your Go environment up to date.

Update Cloud9 IDE with latest Go (1.3.3 as of this writing)


  1. Open you Go lang project (from Github)
  2. Open your Cloud9 terminal in workspace for your Go project.
  3. Get URL URL for Go download for linux-amd64
  4. From terminal, check location where Go binaries are installed
    1. $> which go
  5. Make note of the location where Go is installed. For instance if the previous command returns /opt/go/bin/go,   this means Go’s install directory is the /opt/ directory.
  6. Extract the archive in the into the install directory to overlay the previous install.
    1. sudo tar -C /opt/ -xvf go1.3.3.linux-amd64.tar.gz
    2. Adjust command for respective version downloaded.
  7. Test installation
    1. $> go version
  8. Ensure your GOPATH is properly setup.

After you have upgraded you will be almost there.  When you create a workspce directly from your Github project (as I did), it will not be setup like the Go workspace standard (with src/github/username, etc).  Echo $GOPATH to see where the current workspace is located (it will be pointing to your project's workspace directory or whatever you called the Cloud9 workspace).  Use the project's terminal to move the directories around into the expected structure for Go.  Done!

For a cloud-based IDE, Cloud9 seems to have many features of a modern desktop IDE.  I have not explored all, but my next stop is customization.  I want to run `go fmt` on save as I do in sublime.

3.06.2014

Hacking Golang: Accessing Specific Value from Multi-Value Function Returns

In current version of Go (1.2), a multi-value returned from a function call can only be accessed by doing a multi-variable assignment:

b, c := F()

That works great.  However, there are times, instead of using a two-step approach (assignment, then access) to get to targeted values, I want to access the value inline using an expression, not an assignment.  Right now, however, Go does not support anything like that.  For instance
the following will blow up:

type Path struct{FileName string}
...
Path{FileName:path.Split("/a/b/c.d")}

The Go compiler will complain (paraphrasing here) about multi-value in a single-value context since path.Split() returns (string, string). This is as expected.  I started thinking, with a language as capable as Go, is there a way to express target a specific return value and mute others so the expression above would work?

A Workaround
So I started to research how the multi-value returned by a function is represented inside Go?  Is it an array, a slice, container, what?  And why does fmt.Println(path.Split(...)) works without any complaint from the Go compiler?  Looking for the answer to the ladder question in Go's source code for the fmt package revealed some interesting facts about the multi-values returned from a function call:
  • Multi-values returned from one function call can be passed in as multi-variable arguments to another function.  Meaning, func Split()(string,string) can be passed as argument to   fmt.Printf(a...interface{}) .  Inside, fmt.Printf(...) receives variable a as []interface{} from the compiler.
  • The compiler only does that trick (as far as I found) for function with varargs params.  For instance f.Foo(a []interface{}) cannot be called with b.Bar()(string, string) as argument.
  • In other contexts, you get big fat ugly failures:
    • a []interface{} := path.Split(...) // illegal, fails compilation 
    • var b []interface{} = path.Split(...) // fails with multi-value in single value context error
    • var c []interface{}; c = path.Split(...) // stil fails (as above)
So, that led me to come up with a function I call the mute function (represented by Greek letter mu, single character, simple, easy to use) as defined below.
func µ(a ...interface{}) []interface{} {
    return a
}
See code at http://play.golang.org/p/IwqmoKwVm-

The mu function exploits the automatic conversion from the compiler to convert a vararg parameters in the form of "x...interface{}" to a standard []interface{}.  The implication here is that I can use mu to wrap multi-values from a function call to access specific returned value by specifying an index as shown below.

Path{FileName:µ(path.Split("/a/b/c.d"))[0].(string)}

NOTE: since µ() returns []interface{}, you will need Type Assertion to get the actual value.

You can see an example of the mu function on the Go Playground at http://play.golang.org/p/IwqmoKwVm-

Thoughts
This is a hack by all means.  Go designers take pride in keeping things simple.  Having an extra assignment step to access multi-values returned form function calls is not that big of a deal.  On the other hand, while working with Go, you will inevitably run into cases where you wish there's a way to chain function calls as expressions.  But right now, when your function call returns multiple values, you have to do an assignment first to unpack that value.  Which feels like it is robbing Go of its functional qualities.

3.05.2014

Jumping to Go

This post is about my impression of Google's language Go (or Golang).  As the holidays approached last year (2013), I decided to pour my free time into Go.  I went through the Go documentation methodically to make sure I understand the motivation behind some of the decisions regarding the language.  That is very important since I carry the Java hammer around.

The result of all of my efforts culminated into the creation of an HDFS FileSystem client written in Go - https://github.com/vladimirvivien/gowfs.  That alone is a testament of the simplicity and approachability of the language.  Within 3 weekends (in Jan/Feb), I was able to put together a working WebHDFS client.

The followings are some points I wanted to share if you are thinking about jumping into Go yourself.

Jump into Go
Getting started in go is incredibly easy.  To get a taste of the language you can take the Go Tour (http://tour.golang.org/).  The tour gives you an online editor and server-side compilation service to explore several Go examples.  The tour covers all major language constructs and APIs to get you started and keep you interested.

Go!
When you are ready to do Go locally, the steps are just as easy.  First,  download and install Go for your platform (http://golang.org/doc/install).  Next, select your favorite text editor to start slinging Go code.

The Workspace
One important point that I enjoy (as a Java developer) is the notion of Go's workspace.  It is a location where Go keeps all source and binary packages organized in a specific directory structure.  This simplifies many aspects of working with Go including building, testing, and dependency resolution.  Because of this "Convention over Configuration" approach, one can build Go code without having to specify a long list path information (as is necessary using Java tools).  For instance, the following will resolve package dependencies declared in the code, compile, and run the specified Go file:

go run hello.go

If you are from the Java camp, this may be reminiscent of what the local repository does when using the Maven tool.  However, in Go, this approach is much simplistic and the workspace is a first-class citizen with the Go tools which understand how to deal with the code's organization.

For a full explanation of Go's code organization, check out 'How to Code in Go'.

No IDE Needed (Really)
If you are from the Java camp like myself, you may not believe it's possible to create complete projects without an IDE.  Well,  yes  really, no IDE.  I wrote the entire project using Sublime Text.  How is that possible you may ask?  Well, Go is a simple language with little surprises.  In addition, the Godoc (see below) tool provides easy access to the API reference documentation whether online or offline.

Non-Ceremonious Language
I won't go into too much about the language itself as it is well documented at the http://golang.org website.  Nevertheless,  Go takes a simplistic approach to language design by foregoing object-orientation.  Go programs are collections of first-class functions, simple  types, struct types, and Go's version of interfaces.  Functions can be graphed unto any types (function receivers) to further organize code as computational components.

By removing the need to declare a class,  a constructor, and main with program argument list, a compilable, functional Go code can be as simple as the following.

package main

import "fmt"

func main() {
 fmt.Println("Hello!")
}
Run the code at http://play.golang.org/p/duRF5gXJEP
Documentation! Documentation! Documentation!
Switching sweaty Steve Balmer's chant from "Developers! Developers! Developers!.." (http://www.youtube.com/watch?v=8To-6VIJZRE) to "Documentation! Documentation! Documentation! Docu..." would be an an apt way to explain the importance of documentation for the Go folks.  Here are a few documentation resources.

Go's Website (http://golang.org)
The website has all the docs you will ever need to get started with Go including API docs, videos, best-practice articles, etc.  I think the rarity of Go books has to do with the fact that the platform is so well documented on the site.  

The Godoc Tool
The Godoc tool further demonstrates the importance of documentation within Go all the while keeping things simple.  The following command will compile the entire golang.org website along with any source-level documentation in you own code and makes them available at http://localhost:6060/.

godoc -http=:6060
 
This makes all of Go's API documentation portable even when you don't have an Internet connection.

Godoc.Org (http://godoc.org)
This website aggregate Go projects documentation from major source code hosting services including GitHub, BitBucket, Google Project, and LaunchPad.  The content is indexed and made searchable.  It is a really neat tool.

Go Talks (http://talks.golang.org/)
Another fantastic source of information are the Go Tasks.  This section of the website is a collection of deck slides and accompanying source code. 

Fast Compile and Fast Tests
Again, being from the Java camp, I got used to my project taking minutes to compile and test.  I have used that time slice (between build start and build finish) to read entire books, catch up on twitter, or even pick up a new skillset.

The story is a bit different with Go.  I can compile and test my modestly-sized HDFS client project in milliseconds.  That also includes running about two dozen tests (and counting). "So what?" you may say.  Well, Go, it turns out, is the perfect platform for TDD.  Because testing is so fast, it almost feels like there is little to no cost to integrate unit testing into your code cycle.

Solid APIs
Within its short existence, Go has garnered an impressive list of APIs (http://golang.org/pkg/) that makes its completeness rival more established languages.  All major aspect of building systems are covered in one way or the other including
  • OS-level services, 
  • Cryptography,
  • IO
  • Network communication
  • HTTP
  • HTML
  • Database
  • Regex
  • Etc
Growing Adoption
The list of companies adopting Go is phenomenal.  Again, for such a young language, many companies are betting the future of vital components on a language that is still at version 1.x.  I won't enumerate any here, but a Google search will.  As the language improves over time, I am sure the trend of adoption will continue.

OK Annoyances
Yes, the Go platform is not without its annoyances. Some annoyances are due to my background from the Java world.  Other annoyances are due to the fact that the language is young and is still evolving with plenty of room for improvements.  However, none of the annoyances I have encountered are deterrents to keep me away.  The positives far outweigh the negatives.



4.22.2012

MockHttpServer for Testing Android HTTP Client

While developing an Android project, I needed to test an HTTP client service layer running in my application.  I wanted to ensure I unit tested features such as ability to send/receive headers, request http resources, and deal with HTTP response codes properly. However, I did not want to run my JUnit tests using an external HTTP server. This would create a dependency on yet another technology that I would have to launch with Maven. Furthermore, using an external server would make it difficult to validate unit test cases from the server side.

MockHttpServer

You may already know that Android's library comes with a version of Apache's HttpComponent. While Android promotes the use of HttpComponent for mostly client side development, the provided APIs include the server-side pieces as well. So, you can create your own HTTP server running on Android.
MockHttpServer is a wrapper around the HttpComponent's HttpService API. MockHttpServer provides a simple interaction point where developers simply provide an implementation of HttpRequestHandler.
The MockHttpServer handles the setup and thread management. The following shows how to use the MockHttpServer to enumerate and test the expected headers from the client:
        private final static int SVR_PORT = 8585;
        private final static String SVR_ADDR = "http://localhost:" + SVR_PORT;
...
	public void testHeaderEnumerations () throws Exception{
		// setup mock server and handler for given path
        MockHttpServer server = new MockHttpServer(SVR_PORT);
        server.start();

        server.addRequestHandler("/test1", new HttpRequestHandler() {

            @Override
            public void handle(HttpRequest req, HttpResponse rsp, HttpContext context) throws HttpException, IOException {
            	int headerCount = 0;
                String headers = "Accept Accept-Charset";
                HeaderIterator it = req.headerIterator();
                while (it.hasNext()){
                    Header h = (Header) it.next();
                    if(headers.contains(h.getName())){
                        headerCount++;
                    }
                }
                Assert.assertEquals(3, headerCount);
            }
        });

        // setup client connection
        URL url = new URL(SVR_ADDR + "/test1");
        HttpURLConnection conn = (HttpURLConnection) url.openConnection();
        conn.addRequestProperty("Accept", "text/xml");
        conn.addRequestProperty("Accept", "Audio/mpeg");
        conn.addRequestProperty("Accept-Charset", "utf-8");

        try{
        	conn.getInputStream();
        }finally{
        	conn.disconnect();
        }

        server.stop();
	}
The top portion of the code sets up the HttpRequestHandler implementation. This is where one specifies how the server would handle the incoming HTTP request. In the implementation above, we are enumerating and validating incoming HTTP headers. The second portion of the code sets up the client call to the MockHttpServer instance. Here, for simplicity, we are using an instance of HttpURLConnection to make the HTTP request to the MockHttpServer.

4.10.2012

Jmx-Cli: A Command-Line Console to JMX

One of the reasons I created the Clamshell-Cli Framework originally was to develop a text-based console for JMX.  Well, it is here. Introducing Jmx-Cli: Jmx-Cli a command-line interface console for JMX. It was developed using the Clamshell-Cli framework (http://code.google.com/p/clamshell-cli/) to prove that the framework was flexible enough to create a useful tool.

Installing Jmx-Cli

  1. Download the current distribution from https://github.com/vladimirvivien/jmx-cli/downloads.
  2. Unzip at the location of your choosing (jmx-cli-home).
  3. Cd to jmx-cli-home
  4.  From the command prompt, type > java -jar cli.jar
If all works ok, you should see:
    /#####                                /######    /##   /##
   |__  ##                               /##__  ##  | ##  |__/
      | ##   /######/####    /##   /##  | ##  \__/  | ##   /##
      | ##  | ##_  ##_  ##  |  ## /##/  | ##        | ##  | ##
 /##  | ##  | ## \ ## \ ##   \  ####/   | ##        | ##  | ##
| ##  | ##  | ## | ## | ##    >##  ##   | ##    ##  | ##  | ##
|  ######/  | ## | ## | ##   /##/\  ##  |  ######/  | ##  | ##
 \______/   |__/ |__/ |__/  |__/  \__/   \______/   |__/  |__/

A command-line tool for JMX
Powered by Clamshell-Cli framework 
http://code.google.com/p/clamshell-cli/

Getting Started

Using Jmx-Cli is straight forward. From the prompt type 'help' to get a list of available commands:
      exit       Exits ClamShell.
      help       Displays help information for available commands.
        ps       Displays a list of running JVM processes (similar to jps tool)
   connect       Connects to local or remote JVM MBean server.
     mbean       Creates a label for identifying an MBean
      desc       Prints description for specified mbean.
      list       Lists JMX MBeans.
      exec       Execute MBean operations and getter/setter attributes.

Command Hints

You can also get command hints from the prompt by pressing on the tab key. This will list all available combinations of input. You can narrow down the hints by typing a few letters of the command then press tab.

Connecting to a JVM

To connect to a running local JVM, do the followings:
1. Use the 'ps' command to get a list of JMV processes running locally:
> ps
1444	cli.jar
1740	org.apache.catalina.startup.Bootstrap
2. use the 'connect' command to attach to a running process
> connect pid:1740
The sequence above connects to a running instance of Tomcat server.

Listing MBeans

Once connected to a JVM, you can query available MBeans that are running in that JVM. Use the 'list' command to retrieve lists of MBeans.
> list filter:"Catalina:*"
The command above will list all MBeans running in the "Catalina" domain.  You can label the list for future reference by typing:
> list filter:"Catalina:*" label:true

MBean list [Catalina:*]

     [$0] Catalina:type=Server
     [$1] Catalina:realmPath=/realm0/realm0,type=Realm
...

Describing an MBean

Once you have a list, you can select a bean to describe. The following describes bean with label 'Catalina:port=8080,type=Connector':
> desc bean:"Catalina:port=8080,type=Connector"

MBean: Catalina:port=8080,type=Connector (org.apache.catalina.mbeans.ConnectorMBean)
Implementation of a Coyote connector

Attributes:
     port : int (rw) - The port number on which we listen for requests
     useIPVHosts : boolean (rw) - Should IP-based virtual hosting be used? 
     redirectPort : int (rw) - The redirect port for non-SSL to SSL redirects
     minSpareThreads : int (rw) - The number of request processing threads that will be created
     secure : boolean (rw) - Is this a secure (SSL) Connector?
     acceptCount : int (rw) - The accept count for this Connector
     maxThreads : int (rw) - The maximum number of request processing threads to be created
     URIEncoding : java.lang.String (rw) - Character encoding used to decode the URI
     modelerType : java.lang.String (r) - Type of the modeled resource. Can be set only once
     packetSize : int (rw) - The ajp packet size.
     processorCache : int (rw) - The processor cache size.
...
     xpoweredBy : boolean (rw) - Is generation of X-Powered-By response header enabled/disabled?
     stateName : java.lang.String (r) - The name of the LifecycleState that this component is currently in
     allowTrace : boolean (rw) - Allow disabling TRACE method
     useBodyEncodingForURI : boolean (rw) - Should the body encoding be used for URI query parameters
     secret : java.lang.String (w) - Authentication secret (I guess ... not in Javadocs)

Operations:
     destroy():void
     pause():void
     stop():void
     resume():void
     start():void
     init():void
The same command can be issued using the associated label of the bean from the list (see above):
> desc bean:$28

Execute an Operation

Jmx-cli lets you execute operation on your management beans as well. The next examples shuts down the Tomcat connector for port 8080:
> exec bean:"Catalina:port=8080,type=Connector" op:"stop"

References

1. https://github.com/vladimirvivien/jmx-cli - Jmx-Cli home page
2 http://code.google.com/p/clamshell-cli/ - Clamshell-Cli Framework web site