RSS
 

Archive for the ‘Groovy’ Category

10 Cool Facts about Gradle

31 Aug

About a month ago @saylambda user group (lambda.org.il) had a “JavaScript as a Functional Language” meeting, very nicely presented by Zachary Kessin (@zkessin). You can see a summary, HTML slides and video recording of the session.

This time the subject was Groovy and Ronen Narkis (@narkisr) presented Groovy as a language, its powerful GDK, AST and MOP capabilities together with GPars examples. Ronen presentation is available in his GitHub repository.

I was invited to speak about Gradle and since my session was targeted to the audience where not everyone was familiar with either Groovy or Gradle I decided not to cover it feature-by-feature or go into any technical details (and we were short on time, as always). Instead, I picked 10 facts about Gradle which, coming from the Maven background, made me say “Hmm, that’s cool!” in the past. Hope you’ll share my experience so here it is.



My other presentations are now available in GitHub as well.

 
 

Groovy 1.8.0 – meet JsonBuilder!

17 Jun



Groovy 1.8.0 released in April brought a lot of new features to the language, one of them is native JSON support through JsonSlurper for reading JSON and JsonBuilder for writing JSON.

I recently used JsonBuilder in one of my projects and initially experienced some difficulties in understanding how it operates. My assumption was that JsonBuilder works similarly to MarkupBuilder but as I have quickly found out, it really doesn’t.

Let’s take a simple example. Assume we have a class Message that we would like to serialize to XML markup and JSON.

@groovy.transform.Canonical
class Message {
    long   id
    String sender
    String text
}

assert 'Message(23, me, some text)' ==
       new Message( 23, 'me', 'some text' ).toString()

Here I used Groovy 1.8.0 @Canonical annotation providing automatic toString(), equals() and hashCode() and a tuple (ordered) constructor.

Let’s serialize a number of messages to XML.

def messages = [ new Message( 23, 'me', 'some text'       ),
                 new Message( 24, 'me', 'some other text' ),
                 new Message( 25, 'me', 'same text'       )]

def writer = new StringWriter()
def xml    = new groovy.xml.MarkupBuilder( writer )

xml.messages() {
    messages.each { Message m -> message( id     : m.id,
                                          sender : m.sender,
                                          text   : m.text )}
}

assert writer.toString() == """
<messages>
  <message id='23' sender='me' text='some text' />
  <message id='24' sender='me' text='some other text' />
  <message id='25' sender='me' text='same text' />
</messages>""".trim()

Well, that was pretty straightforward. Let’s try to do the same with JSON.

def json = new groovy.json.JsonBuilder()

json.messages() {
    messages.each { Message m -> message( id     : m.id,
                                          sender : m.sender,
                                          text   : m.text )}
}

assert json.toString() ==
       '{"messages":{"message":{"id":25,"sender":"me","text":"same text"}}}'

Wow, where did all other messages go? Why only one last message in the list was serialized?
How about this:

json = new groovy.json.JsonBuilder()

json.messages() {
    message {
        id     23
        sender 'me'
        text   'some text'
    }
    message {
        id     24
        sender 'me'
        text   'some other text'
    }
}

assert json.toString() ==
       '{"messages":{"message":{"id":24,"sender":"me","text":"some other text"}}}'

Same story. Initially I was puzzled, but then JsonBuilder source code showed that every invocation overrides the previous content:

JsonBuilder(content = null) {
    this.content = content
}

def call(Map m) {
    this.content = m
    return content
}

def call(List l) {
    this.content = l
    return content
}

def call(Object... args) {
    this.content = args.toList()
    return this.content
}

def call(Closure c) {
    this.content = JsonDelegate.cloneDelegateAndGetContent(c)
    return content
}

As you see, one should invoke JsonBuilder exactly once, passing it a Map, List, varargs or Closure. This makes JsonBuilder very different from MarkupBuilder which can be updated as many times as needed. It could be caused by the JSON itself, whose format is stricter than free-form XML markup: something that started as a JSON map with a single Message, can not be made into array of Messages out of sudden.

The argument passed to JsonBuilder (Map, List, varargs or Closure) can also be specified in constructor so there’s no need to invoke a builder at all. You can simply initialize it with the corresponding data structure and call toString() right away. Let’s try this!

def listOfMaps = messages.collect{
                 Message m -> [ id     : m.id,
                                sender : m.sender,
                                text   : m.text ]}

assert new groovy.json.JsonBuilder( listOfMaps ).toString() ==
       '''[{"id":23,"sender":"me","text":"some text"},
           {"id":24,"sender":"me","text":"some other text"},
           {"id":25,"sender":"me","text":"same text"}]'''.
       readLines()*.trim().join()

Now it works :) After converting the list of messages to the list of Maps and sending them to the JsonBuilder in one go, the String generated contains all messages from the list. All code above is available in Groovy web console so you are welcome to try it out.

Btw, for viewing JSON online I recommend an excellent “JSON Visualization” application made by Chris Nielsen. “Online JSON Viewer” is another popular option, but I much prefer the first one. And for offline use “JSON Viewer” makes a good Fiddler plugin.

P.S.
If you need to read this JSON on the client side by sending, say, Ajax GET request, this can be easily done with jQuery.get():

<script type="text/javascript">
var j = jQuery;

j( function() {
    j.get( 'url',
           { timestamp: new Date().getTime() },
           function ( messages ){
               j.each( messages, function( index, m ) {
                   alert( "[" + m.id + "][" + m.sender + "][" + m.text + "]" );
               });
           },
           'json'
        );
});
</script>

Here I use a neat trick of a j shortcut to avoid typing jQuery too many times when using $ is not an option.

 
No Comments

Posted in Code, Groovy

 

Groovy: Picking a value from a number of options

27 Mar

Let’s say you need to define a value based on a number of conditions. If first condition holds true, then the value is X, if second condition holds true then the value is Y and so on. I used to go with a multi-level ternary operator to accomplish that:

def a = 2
def x = ( a == 1 ) ? 'aa' :
        ( a == 2 ) ? 'bb' :
        ( a == 3 ) ? 'cc' :
                     'dd'
assert x == 'bb'

Today @tim_yates has suggested a more debuggable way:

def a = 2
def x = a.with { switch( it ) {
    case 1  : return 'aa'
    case 2  : return 'bb'
    case 3  : return 'cc'
    default : return 'dd' }
}
assert x == 'bb'

Thanks, Tim! Will start using it now. This is very close to Scala version, which is even shorter:

val a = 2
val x = a match {
    case 1 => "aa"
    case 2 => "bb"
    case 3 => "cc"
    case _ => "dd"
}
assert ( x == "bb" )
 
3 Comments

Posted in Groovy, Scala

 

Maven Plugins v0.2.1 – re-written and open-sourced!

05 Mar

A "0.1" release of these Maven plugins back in November brought a lot of attention to the project, which showed me that other Maven developers find them as useful as I do. Later, people started to send me new suggestions and open YouTrack issues which I had an extreme pleasure to work on.

Version "0.2.1" was recently released and I believe this version is of paramount importance and it is much more than just an upgrade. Lots of things have changed for the project in addition to features added and bugs fixed:

  • Lots of people have helped me to work on this version by talking to me personally, over e-mail or by opening issues. A big thank you to all of you! Without your impact, ideas and bug reports this release would contain much less.
  • The code was re-written in Groovy and open-sourced. Writing Maven plugins in Groovy is such a big fun, I’ll be talking about it in “Groovy Builds” session at Gr8Conf this year. Come to see how Maven can be better!
  • A separate testing project was created with hundreds of thorough tests and examples. A GCommons library was extracted from the code into a standalone project.
  • Documentation was updated and GroovyDoc is now available as well. But if you’re short of time here’s a quick presentation. The project now has its own mailing list and you’re welcome to join in order to stay informed and take part in features discussions and prioritizing future progress.
  • The project gets more exposure: the Spring issue of Methods & Tools will come out with my article describing the plugins’ purpose and advantages for less technical people, and GroovyMag March 2011 just came out with an article about GCommons. And if all goes well, the hudson/jenkins plugin will be covered by “Jenkins: The Definitive Guide” as John Ferguson Smart kindly accepted my proposal to contribute to the book. This sounds really nice and I hope this is just the beginning.
  • JFrog and JetBrains tools keep working so well that I’m now convinced that a combination of Artifactory, TeamCity and YouTrack is an excellent option these days.


Now, what actually changed in this release? Versions "0.2" and "0.2.1" release notes are available in YouTrack and on the Wiki but here are some highlights:

  • All plugins now support a <runIf> conditional execution. It allows to invoke them conditionally, something that is otherwise impossible with Maven.
  • "maven-copy-plugin" – bullet-proof FTP download, Zip entries unpack, archives update, <runIf> per <resource>, FTP/SCP uploads.
  • "maven-hudson-plugin" – free-style jobs, CVS support, <properties>, <authToken>.
  • "maven-assert-plugin"<assertGroovy> with File.directorySize(), very handy in tests!
  • "maven-sshexec-plugin" – multiple commands support, key-based authentication, echo of the current directory and commands executed.


Now, what’s next? A lot. I also plan to polish GroovyDocs as much as possible.

 

New Maven plugins released!

14 Nov



Note, an update is available.

After a lot of work I released version "0.1" of the following Maven plugins:

  • "maven-copy-plugin" is an alternative to Maven plugins like assembly, resources, dependency, and truezip. Its purpose is to make working with archives and dependencies very easy.



    It is a Swiss Army knife if you need to copy, pack and unpack files, archives and Maven dependencies. Content replacement, network support, Groovy extension points, attaching archives created as Maven artifacts – it is possible to perform all operations in a single Maven plugin! Oh boy, just give it a try.

  • "maven-hudson-plugin" allows to generate Hudson jobs, keeping any amount of them in a single POM. Jobs can be organized hierarchically with inheritance and can invoke each other, Artifactory deployment is supported as well. Managing tens or hundreds of Hudson jobs becomes possible when they are kept in one place and inherit each other with a sensible defaults.
  • "maven-spring-batch-plugin" allows to invoke Spring Batch jobs as part of Maven build.
  • "maven-mail-plugin" allows to send mails with attachment from Maven.
  • "maven-sshexec-plugin" allows to execute commands on a remote server over ssh.
  • "maven-properties-plugin" allows to create Maven properties dynamically with Groovy snippets.
  • "maven-timestamp-plugin" allows to create a timestamp Maven property.
  • "maven-assert-plugin" allows to verify various build assertions: properties are defined, files exist and files/directories content is identical.
  • "maven-find-plugin" allows to set a Maven property to location of folder dynamically found for each module built. It helps in situations where build needs to access files in other locations than the current module.
 

Consider returning something instead of void

05 Nov

Some methods are not supposed to return anything so they usually return void:

/**
 * Verifies objects specified are not null
 *
 * @param objects objects to check
 */
static void notNull ( Object ... objects )
{
    objects.each{ assert ( it != null ), "Object specified *is* null" }
}

This short method verifies objects specified are not nulls. It is part of verifiers library I usually carry around with me:

notNull       ( someObject,   anotherObject,              .. )
notNullOrEmpty( "someString", anotherString, thirdString, .. )
notNullOrEmpty( [ a : 'b' ],  anotherMap,    thirdMap,    .. )

isFile( file )
isDir ( directory )

But .. when you have something in your hand, why throwing it away?

/**
 * Verifies objects specified are not null
 *
 * @param objects objects to check
 * @param <T> object's type
 * @return first object specified (for chaining)
 */
static <T> T notNull ( T ... objects )
{
    objects.each{ assert ( it != null ), "Object specified *is* null" }
    return objects [ 0 ]
}

Initially, all verifiers returned void but after changing them to return an object checked it became possible to wrap parameters passing or chain methods invocation:

someMethod    ( notNull( someObject ))
anotherMethod ( notNullOrEmpty( someString ))

notNullOrEmpty( someString ).replace( .. )
Process p = notNullOrEmpty( command ).execute()
isFile( file ).getParentFile()
isDir ( dir  ).listFiles().each{ .. }

The idea of returning a value, usually some kind of self, is an old one, of course. StringBuilder does it, dom4j does it but Collection does not. How many times this boolean returned is used anyway?

new StringBuilder().append( '1' ).append( '2' ).append( '3' );
Element e = root.addElement( "Tag" ).addAttribute( "attr", "puttr" );

List<String> l = new ArrayList<String>();
l.add( "Something" ); // Full stop
l.add( "Anything"  ); // Full stop!
l.add( "Nothing"   ); // Argh .. again?!

So this post is a gentle reminder to return a meaningful value instead of a useless void if you can. It will be ignored in the worst case but can be nicely chained otherwise.

 
No Comments

Posted in Code, Groovy

 

Keeping Groovy Beans in the same file

02 Sep

It is so handy to keep a bunch of Groovy Beans in the same file!

"Beans.groovy":

class Artifactory
{
    String  name
    String  repository
    boolean deployArtifacts = true
    String  user
    String  scrambledPassword
}

class Mail
{
    String  recipients
    boolean sendForUnstable   = true
    boolean sendToIndividuals = true
}

...
 
No Comments

Posted in Groovy

 

Splitting Groovy objects with “each”-like iterations

01 Sep

String.eachLine(Closure closure) allows to iterate a String line by line:

"""aaaaaaaaaaaaa
bbbbbbbbbbbbb
ccccccccccccc""".eachLine { println "[$it]" }​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​

// Produces:

[aaaaaaaaaaaaa]
[bbbbbbbbbbbbb]
[ccccccccccccc]

Similarly, File.eachFile(groovy.lang.Closure) iterates over all directory files:

new File( "." ).eachFile{ println "[${ it.canonicalPath }]" }

// Produces:

[D:\Winny\java\groovy-1.7.4\bin\grape]
[D:\Winny\java\groovy-1.7.4\bin\grape.bat]
[D:\Winny\java\groovy-1.7.4\bin\groovy]
[D:\Winny\java\groovy-1.7.4\bin\groovy.bat]
...

In fact, Object.each(groovy.lang.Closure) is the most general way to iterate over “parts” of an object:

new File( "groovy.bat" ).withReader { it.each{ println "[$it]" }}

// Produces:

[@if "%DEBUG%" == "" @echo off]
[]
[@rem ]
[@rem $Revision: 19763 $ $Date: 2010-04-05 14:33:24 +0200 (Mo, 05. Apr 2010) $]
[@rem ]
...

Here I use a fact that File.withReader(groovy.lang.Closure) passes java.io.LineNumberReader into a closure and its "each" iterates over lines. That’s how we get each line of "groovy.bat" printed.

It actually became a habit to "each { .. }" Groovy objects to see the iteration $it.
But wouldn’t it be better if "each" was splitting objects to chunks, returning a list of them ?

Object.metaClass.splitWith = { String methodName ->

    MetaMethod m = delegate.metaClass.pickMethod( methodName, Closure.class )
    assert     m

    def result = []
    m.doMethodInvoke( delegate, { result << it } )

    result
} 

def files = new File( "d:/Winny/java/groovy-1.7.4" ).splitWith( 'eachFileRecurse' )
println "Largest file is of size ${ files*.length().max() } bytes"

// Produces:

Largest file is of size 5370870 bytes

Now we can “split” any object using its iterating methods:

// There are 15458 directories in "C:/Windows"
assert 15458 ==
       new File( "C:/Windows" ).splitWith( 'eachDirRecurse' ).size()

// There are 14 non-empty lines in "groovy.bat"
assert 14    ==
       new File( "groovy.bat" ).text.splitWith( 'eachLine' ).findAll{ it }.size()

// There are 24 permutations of [ 1, 2, 3, 4 ]
assert 24    ==
       [ 1, 2, 3, 4 ].splitWith( 'eachPermutation' ).size()

// Splitting an Iterator with "each" bring back the original List
assert [ 1, 2, 3, 4 ] ==
       [ 1, 2, 3, 4 ].iterator().splitWith( 'each' )

// Cleanup file data by trim()-ing and filtering out empty lines
new File( .. ).text.splitWith( 'eachLine' )*.trim().findAll{ it }

splitWith() allows one to break an object into chunks of data without having to make a full stop and iterate, storing away each $it in a List:

println [ 1, 2, 3, 4 ].splitWith( 'eachPermutation' ).size()

// vs.

def result = []
[ 1, 2, 3, 4 ].eachPermutation{ result << it }
println result.size()



 
No Comments

Posted in Groovy

 

Neat Delicious trick – keywords combination

26 Aug

The real beauty of Delicious lies in keywords combinations.

I mentioned already that Delicious is my #2 most favorite productivity tool. Within time I developed a set of the most frequently-used keywords in the right column that can be combined with tags in the left column:

"hudson" - Mailing list - "ml"
"spring" - Issue tracker - "jira"
"ant" - API documentation - "api"
"jfrog" - Documentation - "doc"
"groovy" - Maven repository - "repo"
"maven" - Source code browsing - "code"


Now with a "d"-keyworded Delicious search I only need to type "d spring api" to get to

Lots of other useful combinations are also available:

"groovy ml" "hudson ml" "spring ml" "maven ml" "jfrog ml"
"groovy jira" "hudson jira" "spring jira" "gmaven jira" "jfrog jira"
"groovy api" "java api" "spring api" "maven api" "apache api"
"java7 doc" "java doc" "spring doc" "css doc" "html doc"
"groovy repo" "hudson repo" "apache repo" "plugins repo" "jfrog repo"
"groovy code" "hudson code" "spring code" "gmaven code" "groovypp code"


So jumping to a mailing list or API documentation for X is just a matter of "d X ml" or "d X api". Fast!

 

Ant FTP task: adding progress indicator and timeout

18 Aug

Ant FTP task provides no option to display how much the download has progressed so far, similarly to wget or curl verbose options. Setting connection timeout isn’t possible either.

But I had to implement it one day and here is what I have found out:

The basic idea is therefore to redefine an FTP task with improved version of FTPClient making use of CopyStreamListener when calling Util.copyStream(). Let’s do it!

1. New FTP task:

The biggest problem in the whole story is the following line of code: ftp = new FTPClient(). I wish there were a simple IoC-like way to “inject” an FTPClient instance here but Ant does not provide too many ways to customize its default configuration, something like RestTemplate.setRequestFactory(). Alas!

I really wish more people would embrace Spring/IoC way of thinking: everything is a Strategy and therefore can be “injected”, extended, modified, and customized.

So my first attempt was to extend Ant’s org.apache.tools.ant.taskdefs.optional.net.FTP class, override execute() method, copy its original code and replace the assignment. As much as I have no taste for copying someone else’s code in order to modify a line or two, sometimes this is the only way to go. But it didn’t work! execute() method accesses lots of private members, not accessible from a sub-class.

The next option was to copy an entire FTP class and then replace the assignment. If anyone can suggest a better way to do the same I’m all ears. So I ended up with:

public class FTP extends Task
{
    ...
    public void execute() throws BuildException {
        ...
        org.apache.commons.net.ftp.FTPClient ftp = null;
        ...
        ftp = ( verbose ? new FTPClient( getProject()) :
                          new org.apache.commons.net.ftp.FTPClient());
        ftp.setDataTimeout( 5 * 60 * 1000 ); // 5 minutes
        ...
    }
}

Line 08 sets a customized version of FTPClient or a default one, line 10 sets up FTPClient timeout.

2. New FTPClient:

public class FTPClient extends org.apache.commons.net.ftp.FTPClient
{
    private final Project project; // org.apache.tools.ant.Project

    ...

    @Override
    public boolean retrieveFile(String remote, OutputStream local)
    throws IOException
    {
        ...
        Util.copyStream( input,
                         local,
                         getBufferSize(),
                         CopyStreamEvent.UNKNOWN_STREAM_SIZE,
                         new CopyStreamListener( this.project ),
                         false );
        ...
    }
}

Again, I had to copy the original code and “patch” Util.copyStream() call to use CopyStreamListener implementation. Also, I had to copy and slightly modify an FTPConfigurator class to make it accept my new FTPClient.

3. CopyStreamListener:

public class CopyStreamListener implements org.apache.commons.net.io.CopyStreamListener
{
    private final Project project;
    private       long    mbTransferred = 0;

    ...

    @Override
    public void bytesTransferred ( long totalBytes, int bytes, long streamSize )
    {
        long mb = ( totalBytes / ( 1024 * 1024 ));
        if ( mb > mbTransferred )
        {
            mbTransferred = mb;
            this.project.log( "[" + new Date() + "]: [" + mbTransferred + "] Mb transferred" );
        }
    }
}

4. Plugging in new definition of FTP task with ClfAntBuilder:

public class ClfAntBuilder extends AntBuilder
{
    public ClfAntBuilder ()
    {
        super();
        getProject().addTaskDefinition( "ftp", FTP.class );
    }
}

5. Running it:

new ClfAntBuilder().ftp( action         : 'get',
                         server         : host,
                         userid         : username,
                         ... )
{
    fileset( dir       : '...',
             includes  : '...',
             excludes  : '...' )
}

6. The final result:

Trying to override old definition of task ftp
      [ftp] getting files
      [ftp] transferring SomeFile.xml.zip to C:\Temp\SomeFile.xml.zip
[Tue Aug 17 20:43:07 IDT 2010]: [1] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [2] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [3] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [4] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [5] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [6] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [7] Mb transferred
...

Nice!