RSS
 

Archive for the ‘Code’ Category

An importance of wrapping variables

20 Jun

Long time ago I developed a habit of wrapping every variable logged in [ .. ]:

Java:   log.error( "Class [" + c + "] not found" )
Java:   log.error( String.format( "Class [%s] not found", c ))
Groovy: log.error( "Class [$c] not found" )

The reasons were mostly to spot any trailing whitespaces in variables logged which was very helpful in error messages. As simple as it sounds, lots of messages thrown or logged today still don’t enclose variables in any special characters which can make them quite obscure.

What would you say about this error message ?

..
java.lang.IllegalArgumentException: No enum const class Something.
	at java.lang.Enum.valueOf(Enum.java:196)
..

I was surprised why failing Enum.valueOf() didn’t include the wrong value in the error message. But it did! Here’s the failing code:

..
enum Something{ A, B }
assert Something.A != Something.valueOf( '' )
..

So it was an empty String. Normally, wrong Enum.valueOf() input is printed:

..
enum Something{ A, B }
assert Something.A != Something.valueOf( 'C' )

java.lang.IllegalArgumentException: No enum const class Something.C
	at java.lang.Enum.valueOf(Enum.java:196)
..

But with empty argument "No enum const class Something." simply looked like a period at the end of the sentence! I wish it was "No enum const class [Something.]" instead.

// "java/lang/Enum.java"
public static <T extends Enum<T>> T valueOf(Class<T> enumType,
                                            String name) {
    T result = enumType.enumConstantDirectory().get(name);
    if (result != null)
        return result;
    if (name == null)
        throw new NullPointerException("Name is null");
    throw new IllegalArgumentException(
        "No enum const " + enumType +"." + name);
}
 
No Comments

Posted in Code

 

Groovy 1.8.0 – meet JsonBuilder!

17 Jun



Groovy 1.8.0 released in April brought a lot of new features to the language, one of them is native JSON support through JsonSlurper for reading JSON and JsonBuilder for writing JSON.

I recently used JsonBuilder in one of my projects and initially experienced some difficulties in understanding how it operates. My assumption was that JsonBuilder works similarly to MarkupBuilder but as I have quickly found out, it really doesn’t.

Let’s take a simple example. Assume we have a class Message that we would like to serialize to XML markup and JSON.

@groovy.transform.Canonical
class Message {
    long   id
    String sender
    String text
}

assert 'Message(23, me, some text)' ==
       new Message( 23, 'me', 'some text' ).toString()

Here I used Groovy 1.8.0 @Canonical annotation providing automatic toString(), equals() and hashCode() and a tuple (ordered) constructor.

Let’s serialize a number of messages to XML.

def messages = [ new Message( 23, 'me', 'some text'       ),
                 new Message( 24, 'me', 'some other text' ),
                 new Message( 25, 'me', 'same text'       )]

def writer = new StringWriter()
def xml    = new groovy.xml.MarkupBuilder( writer )

xml.messages() {
    messages.each { Message m -> message( id     : m.id,
                                          sender : m.sender,
                                          text   : m.text )}
}

assert writer.toString() == """
<messages>
  <message id='23' sender='me' text='some text' />
  <message id='24' sender='me' text='some other text' />
  <message id='25' sender='me' text='same text' />
</messages>""".trim()

Well, that was pretty straightforward. Let’s try to do the same with JSON.

def json = new groovy.json.JsonBuilder()

json.messages() {
    messages.each { Message m -> message( id     : m.id,
                                          sender : m.sender,
                                          text   : m.text )}
}

assert json.toString() ==
       '{"messages":{"message":{"id":25,"sender":"me","text":"same text"}}}'

Wow, where did all other messages go? Why only one last message in the list was serialized?
How about this:

json = new groovy.json.JsonBuilder()

json.messages() {
    message {
        id     23
        sender 'me'
        text   'some text'
    }
    message {
        id     24
        sender 'me'
        text   'some other text'
    }
}

assert json.toString() ==
       '{"messages":{"message":{"id":24,"sender":"me","text":"some other text"}}}'

Same story. Initially I was puzzled, but then JsonBuilder source code showed that every invocation overrides the previous content:

JsonBuilder(content = null) {
    this.content = content
}

def call(Map m) {
    this.content = m
    return content
}

def call(List l) {
    this.content = l
    return content
}

def call(Object... args) {
    this.content = args.toList()
    return this.content
}

def call(Closure c) {
    this.content = JsonDelegate.cloneDelegateAndGetContent(c)
    return content
}

As you see, one should invoke JsonBuilder exactly once, passing it a Map, List, varargs or Closure. This makes JsonBuilder very different from MarkupBuilder which can be updated as many times as needed. It could be caused by the JSON itself, whose format is stricter than free-form XML markup: something that started as a JSON map with a single Message, can not be made into array of Messages out of sudden.

The argument passed to JsonBuilder (Map, List, varargs or Closure) can also be specified in constructor so there’s no need to invoke a builder at all. You can simply initialize it with the corresponding data structure and call toString() right away. Let’s try this!

def listOfMaps = messages.collect{
                 Message m -> [ id     : m.id,
                                sender : m.sender,
                                text   : m.text ]}

assert new groovy.json.JsonBuilder( listOfMaps ).toString() ==
       '''[{"id":23,"sender":"me","text":"some text"},
           {"id":24,"sender":"me","text":"some other text"},
           {"id":25,"sender":"me","text":"same text"}]'''.
       readLines()*.trim().join()

Now it works :) After converting the list of messages to the list of Maps and sending them to the JsonBuilder in one go, the String generated contains all messages from the list. All code above is available in Groovy web console so you are welcome to try it out.

Btw, for viewing JSON online I recommend an excellent “JSON Visualization” application made by Chris Nielsen. “Online JSON Viewer” is another popular option, but I much prefer the first one. And for offline use “JSON Viewer” makes a good Fiddler plugin.

P.S.
If you need to read this JSON on the client side by sending, say, Ajax GET request, this can be easily done with jQuery.get():

<script type="text/javascript">
var j = jQuery;

j( function() {
    j.get( 'url',
           { timestamp: new Date().getTime() },
           function ( messages ){
               j.each( messages, function( index, m ) {
                   alert( "[" + m.id + "][" + m.sender + "][" + m.text + "]" );
               });
           },
           'json'
        );
});
</script>

Here I use a neat trick of a j shortcut to avoid typing jQuery too many times when using $ is not an option.

 
No Comments

Posted in Code, Groovy

 

My Git Workflow

09 Mar

When you work with Subversion you don’t talk much about workflow because there’s only one available 99% of time:

..
svn update
svn status
svn commit
..

I have an "ss" alias to "svn update && svn status" combination which I run very frequently for any project I come across. I believe most Subversion issues are caused by not running those two simple commands in time.

But it is different in Git. Since network topology, branching methodology and behavioral patterns may vary for each team (and that’s what makes Git git!) there’s a place to talk about Git workflow.

In my projects I don’t use “feature branches” much but I always work with two branches: "master" and "dev". The development only happens in "dev" which is rebased and merged with "master" when I feel it makes sense. Nightly builds build them all but only "master" artifacts are deployed to Artifactory as snapshots.

For 2 projects I have 4 branches and three different environments where I could update any of them: home, office or netbook. As I felt there’s too much time spent on bringing each project in sync with GitHub, I put together a number of scripts to update and push changes for each of them easily.

"update.bat":

@echo off

call git checkout dev
call git status
call git pull origin dev
call git checkout master
call git status
call git pull origin master
call git checkout dev

"merge.bat":

@echo off

call git checkout dev
call git status
call git rebase master
call git checkout master
call git status
call git merge dev
call git push origin
call git checkout dev

So when I approach a project I just type "update" to get it started or "merge" to wrap it up. There’s also a "merge-all.bat" script to call it a day:

@echo off

cls
e:

echo "=========> gcommons"
cd \projects\gcommons
call update
call merge

echo "=========> maven-plugins"
cd \projects\maven-plugins
call update
call merge

echo "=========> scripts"
cd \projects\scripts
call git status
call git pull origin master
call git push origin master

echo "=========> maven-plugins-test"
cd \projects\maven-plugins-test
call git status
call git pull origin master
call git push origin master

If everything is clean and no updates are left behind, I expect it to display:

"=========> gcommons"
Already on 'dev'
# On branch dev
nothing to commit (working directory clean)
From github.com:evgeny-goldin/gcommons
 * branch            dev        -> FETCH_HEAD
Already up-to-date.
Switched to branch 'master'
# On branch master
nothing to commit (working directory clean)
From github.com:evgeny-goldin/gcommons
 * branch            master     -> FETCH_HEAD
Already up-to-date.
Switched to branch 'dev'
Already on 'dev'
# On branch dev
nothing to commit (working directory clean)
Current branch dev is up to date.
Switched to branch 'master'
# On branch master
nothing to commit (working directory clean)
Already up-to-date.
Everything up-to-date
Switched to branch 'dev'
"=========> maven-plugins"
Switched to branch 'dev'
# On branch dev
nothing to commit (working directory clean)
From github.com:evgeny-goldin/maven-plugins
 * branch            dev        -> FETCH_HEAD
Already up-to-date.
Switched to branch 'master'
# On branch master
nothing to commit (working directory clean)
From github.com:evgeny-goldin/maven-plugins
 * branch            master     -> FETCH_HEAD
Already up-to-date.
Switched to branch 'dev'
Already on 'dev'
# On branch dev
nothing to commit (working directory clean)
Current branch dev is up to date.
Switched to branch 'master'
# On branch master
nothing to commit (working directory clean)
Already up-to-date.
Everything up-to-date
Switched to branch 'dev'
"=========> scripts"
# On branch master
nothing to commit (working directory clean)
From github.com:evgeny-goldin/scripts
 * branch            master     -> FETCH_HEAD
Already up-to-date.
Everything up-to-date
"=========> maven-plugins-test"
# On branch master
nothing to commit (working directory clean)
From github.com:evgeny-goldin/maven-plugins-test
 * branch            master     -> FETCH_HEAD
Already up-to-date.
Everything up-to-date
 
1 Comment

Posted in Code, Git

 

Maven Plugins v0.2.1 – re-written and open-sourced!

05 Mar

A "0.1" release of these Maven plugins back in November brought a lot of attention to the project, which showed me that other Maven developers find them as useful as I do. Later, people started to send me new suggestions and open YouTrack issues which I had an extreme pleasure to work on.

Version "0.2.1" was recently released and I believe this version is of paramount importance and it is much more than just an upgrade. Lots of things have changed for the project in addition to features added and bugs fixed:

  • Lots of people have helped me to work on this version by talking to me personally, over e-mail or by opening issues. A big thank you to all of you! Without your impact, ideas and bug reports this release would contain much less.
  • The code was re-written in Groovy and open-sourced. Writing Maven plugins in Groovy is such a big fun, I’ll be talking about it in “Groovy Builds” session at Gr8Conf this year. Come to see how Maven can be better!
  • A separate testing project was created with hundreds of thorough tests and examples. A GCommons library was extracted from the code into a standalone project.
  • Documentation was updated and GroovyDoc is now available as well. But if you’re short of time here’s a quick presentation. The project now has its own mailing list and you’re welcome to join in order to stay informed and take part in features discussions and prioritizing future progress.
  • The project gets more exposure: the Spring issue of Methods & Tools will come out with my article describing the plugins’ purpose and advantages for less technical people, and GroovyMag March 2011 just came out with an article about GCommons. And if all goes well, the hudson/jenkins plugin will be covered by “Jenkins: The Definitive Guide” as John Ferguson Smart kindly accepted my proposal to contribute to the book. This sounds really nice and I hope this is just the beginning.
  • JFrog and JetBrains tools keep working so well that I’m now convinced that a combination of Artifactory, TeamCity and YouTrack is an excellent option these days.


Now, what actually changed in this release? Versions "0.2" and "0.2.1" release notes are available in YouTrack and on the Wiki but here are some highlights:

  • All plugins now support a <runIf> conditional execution. It allows to invoke them conditionally, something that is otherwise impossible with Maven.
  • "maven-copy-plugin" – bullet-proof FTP download, Zip entries unpack, archives update, <runIf> per <resource>, FTP/SCP uploads.
  • "maven-hudson-plugin" – free-style jobs, CVS support, <properties>, <authToken>.
  • "maven-assert-plugin"<assertGroovy> with File.directorySize(), very handy in tests!
  • "maven-sshexec-plugin" – multiple commands support, key-based authentication, echo of the current directory and commands executed.


Now, what’s next? A lot. I also plan to polish GroovyDocs as much as possible.

 

MultiResourceItemReader – “No resources to read”

12 Dec

By default Spring Batch MultiResourceItemReader ignores missing resources silently:

INFO  [SimpleStepHandler] Executing step: [someStep]
WARN  [MultiResourceItemReader] No resources to read
INFO  [SimpleStepHandler] Executing step: [someOtherStep]

Sounds familiar?
Well, sometimes this behavior is wanted. But sometimes it is not and we would like to make it fail:

public class MyMultiResourceItemReader<T> extends MultiResourceItemReader<T>
{
    private static final Log logger = LogFactory.getLog( MyMultiResourceItemReader.class );

    private boolean failOnNoResources = true;
    public  boolean failOnNoResources () { return this.failOnNoResources; }
    public  void    setFailOnNoResources ( boolean failOnNoResources ) { this.failOnNoResources = failOnNoResources; }

    @Override
    public void setResources ( Resource[] resources )
    {
        boolean noResources = (( resources == null ) || ( resources.length < 1 ));

        if ( noResources && failOnNoResources())
        {
            throw new RuntimeException( "No resources to read" );
        }

        super.setResources( resources );

        if ( logger.isDebugEnabled())
        {
            logger.debug( String.format( "Resources set: %s", Arrays.toString( resources )));
        }
    }
}

As you see, MyMultiResourceItemReader now fails if you attempt to configure it with missing resources. If you need to go back to original behavior, just swap "failOnNoResources" back to "false", just do so before configuring a "resources" property:

<bean id="myReader" class="MyMultiResourceItemReader">
    <property name="failOnNoResources" value="false"/>
    <property name="resources"         value="${directory}/*.zip"/>
    ...
</bean>

Now, attempting to run Spring Batch job with missing files throws an exception. No more ignoring missing resources silently!

"BATCH-1665" will provide a built-in boolean flag to control MultiResourceItemReader strictness.

 
No Comments

Posted in Code, Spring

 

Consider returning something instead of void

05 Nov

Some methods are not supposed to return anything so they usually return void:

/**
 * Verifies objects specified are not null
 *
 * @param objects objects to check
 */
static void notNull ( Object ... objects )
{
    objects.each{ assert ( it != null ), "Object specified *is* null" }
}

This short method verifies objects specified are not nulls. It is part of verifiers library I usually carry around with me:

notNull       ( someObject,   anotherObject,              .. )
notNullOrEmpty( "someString", anotherString, thirdString, .. )
notNullOrEmpty( [ a : 'b' ],  anotherMap,    thirdMap,    .. )

isFile( file )
isDir ( directory )

But .. when you have something in your hand, why throwing it away?

/**
 * Verifies objects specified are not null
 *
 * @param objects objects to check
 * @param <T> object's type
 * @return first object specified (for chaining)
 */
static <T> T notNull ( T ... objects )
{
    objects.each{ assert ( it != null ), "Object specified *is* null" }
    return objects [ 0 ]
}

Initially, all verifiers returned void but after changing them to return an object checked it became possible to wrap parameters passing or chain methods invocation:

someMethod    ( notNull( someObject ))
anotherMethod ( notNullOrEmpty( someString ))

notNullOrEmpty( someString ).replace( .. )
Process p = notNullOrEmpty( command ).execute()
isFile( file ).getParentFile()
isDir ( dir  ).listFiles().each{ .. }

The idea of returning a value, usually some kind of self, is an old one, of course. StringBuilder does it, dom4j does it but Collection does not. How many times this boolean returned is used anyway?

new StringBuilder().append( '1' ).append( '2' ).append( '3' );
Element e = root.addElement( "Tag" ).addAttribute( "attr", "puttr" );

List<String> l = new ArrayList<String>();
l.add( "Something" ); // Full stop
l.add( "Anything"  ); // Full stop!
l.add( "Nothing"   ); // Argh .. again?!

So this post is a gentle reminder to return a meaningful value instead of a useless void if you can. It will be ignored in the worst case but can be nicely chained otherwise.

 
No Comments

Posted in Code, Groovy

 

Watch out, illegal Jetty URL

02 Nov

Is the following URL legal?


http://host?param=value

Well, for Jetty 7.x it is not. And I couldn’t understand why before a friend of mine pointed to a missing "/":


http://host/?param=value

As it appears, if you have a ContextHandler registered to service "/" requests, as we did, Jetty rejects any of them coming without a slash with 400 "Bad request" response.

If you try to use the first URL in a browser, it will add a missing slash automatically. But Apache HttpClient isn’t that smart, unfortunately.

Watch out!

 
No Comments

Posted in Code, Java, Web

 

JUnit Updates

27 Oct

After talking about TDD there comes time to bring my JUnit knowledge up-to-speed with 4.x and 4.8.x versions. Also, I see version 4.8.2 has finally made its way to central Maven repo.

“JUnit Kung Fu: Getting More Out of Your Unit Tests” (audio) teaches some new JUnit tricks.

  • Normally we call the class TestSomething and methods testFunctionalityZ(). Well, it’s kind of .. boring, isn’t it? Let’s give some respect to names! (slide 10)
    public class WhenSomethingHappens
    {
        @Test
        public void somethingShouldBeThis(){ .. }
    
        @Test
        public void somethingElseShouldBeThat(){ .. }
    
        @Test
        public void thisShouldNeverHappen(){ .. }
    }
    

    By replacing 'test' with 'should' we emphasize that we test behavior and not implementation.

  • The structure of each method should normally follow Arrange-Act-Assert structure (slide 11): prepare the environment, do what needs to be done, check results.
  • Hamcrest assertions! (slide 13) Part of JUnit already.
    assertThat( theBiscuit, is( myBiscuit ));
    assertThat( x, is( not( 4 )));
    assertThat( myList, hasItem( "3" ));
    
  • Parametrized tests (slide 27).
    @RunWith ( Parameterized.class )
    public class WhenSomethingHappens
    {
        private int x;
        private int y;
    
        public WhenSomethingHappens ( int x, int y )
        {
            this.x = x;
            this.y = y;
        }
    
        @Parameters
        public static Collection<Object[]> testData()
        {
            return Array.asList( new Object[][]{{ 1, 2 },
                                                { 3, 4 },
                                                { 5, 6 }, .. });
        }
    
        @Test
        public void thisShouldBeThat () { assert( this.x, is( this.y - 1 )) }
    }
    

    Static method marked with @Parameters provides “bulks of data” and test instance will be initialized for each element in the Collection returned. This way the test method is absolutely free from any interfering loops and iterations over various values, which can be now conveniently loaded from external resources like Excel files.

  • JUnit @Rules (slide 35) – AOP-like test interceptors, providing additional functionalities like temporary test folder, test errors collector, test timeout timer, etc.
    public class WhenSomethingHappens
    {
        @Rule
        public TemporaryFolder folder = new TemporaryFolder();
    
        @Test
        public void thisShouldBeThat()
        {
            File createdFile   = folder.newFile( "1.txt" );
            File createdFolder = folder.newFolder( "someFolder" );
            ...
        }
     }
     
  • Parallel tests (slide 48) when you run on multiply CPUs.
  • Infinitest! (slide 50) Continuous testing in Eclipse and IDEA whenever sources are saved. When some test breaks one becomes aware of that immediately and not 2 hours later.


Some more JUnit-related resources:

 
1 Comment

Posted in Code

 

Tidying up XML headers

26 Oct

What reads better?

This :

<?xml version="1.0" encoding="UTF-8"?>

<project xmlns              = "http://maven.apache.org/POM/4.0.0"
         xmlns:xsi          = "http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation = "http://maven.apache.org/POM/4.0.0

http://maven.apache.org/xsd/maven-4.0.0.xsd">
<?xml version="1.0" encoding="UTF-8"?>

<beans xmlns              = "http://www.springframework.org/schema/beans"
       xmlns:xsi          = "http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation = "http://www.springframework.org/schema/beans

http://www.s..work.org/schema/beans/spring-beans.xsd">

Or that (real examples):

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
  xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
  xsi:schemaLocation="http://maven.apache.org/POM/4.0.0

http://maven.apache.org/xsd/maven-4.0.0.xsd">
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
       xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
       xsi:schemaLocation="http://www.springframework.org/schema/beans

http://www.springframework.org/schema/beans/spring-beans-3.0.xsd">

Surprisingly, it is very uncommon to meet an aligned XML header. But it looks way better, isn’t it?

 
No Comments

Posted in Code

 

TDD for unbelievers

26 Oct

I have to admit, I was not-so-good with TDD (Test-driven development). I just didn’t see how it can help me to deliver a better software. Products and solutions I developed were pretty much rock solid and it felt good to see them running for years. What else did I need?

But it was time to give TDD another try. Obviously, it became quite impossible to ignore this subject today, especially when people talk about continuous deployment already!

“Real Developers Don’t Need Unit Tests” (video) does a good job in describing once again what TDD is good for. If you don’t see a value in TDD, like I did – please, do yourself a favor and watch this session.

Some major takeaways:

  • I guess the main point is that TDD is not about writing failing tests first, as I assumed for years. It is a way to design and implement a system, not from one’s head or some silly design specification but from the actual needs. Tests are a pleasant side-effect, but what they really do is actually show how the system works. I think it was “Groovy in Action” that initiated an excellent tradition of using asserts as a way to demonstrate what code does. What describes better the end result?
    assert "test" == str.test()
    // Or
    str.test()  // prints "test"
    

    It may sound like an obvious thing but making a mental switch from “Tests for the sake of tests” to “Design and implement through tests” was my first enlightenment as I watched the video.

  • There are three phases in doing TDD: Red, Green, Refactor. First, you provide a demonstration of a new feature. It takes a form of a failing test, right. But it is a demonstration of a new feature nevertheless. Then you implement it. And then you clean up, refactor and tidy up. The cleaning process is a critical one! From my experience it is widely and commonly omitted as people get to the “Green” phase and commit. This is exactly where products start getting a bad smell, clarity get lost, things get messy and next thing you hear is “Do not ever touch it”.
  • Test behavior, not implementation. Another common belief about unit tests is they should test class methods and state transitions, i.e., current class implementation. The problem with this approach is that tests break each time implementation changes. And instead of updating tests accordingly it is very tempting to comment them out, <exclude> them from running or simply @Ignore. If one tests functionality and behavior then it shouldn’t break! If there’s a need to test a private members then they probably do too much and should be extracted to their own class and their own tests. “Design and implement through tests” again.
  • When doing code review start from the tests. Since tests are the answer to the question “What this code does?” it makes sense to review them first to concentrate on a meaningful parts instead of everything. Especially when time is tight and only the relevant code should be reviewed.
  • Unit tests are not a substitution to integration tests, acceptance tests and QA processes. As was mentioned above, TDD doesn’t really serve the purpose of testing although it is called this way. It doesn’t test for real and doesn’t try to find bugs, it only demonstrates application’s ability to do something.
  • Treat your tests as production code. Tests should not be written as a throw-away code that is never maintained or updated. Tests are application’s user interface, its front page, if you wish. And we wouldn’t like our front page look ugly, would we?


Now, what really stops me from doing TDD starting from tomorrow? I would say a lack of commitment of everyone involved. To integrate a real TDD process into organization is very costly. It will take months until all infrastructures are set up and people stop spending a noticeable extra time on writing tests. Initially it creates a huge overhead, especially when dealing with legacy code. Eventually, an extra time spent on writing and maintaining tests pays off by producing a code that is better designed and is of much higher quality. After all, that’s how people get to continuous deployment that I mentioned above – they started with something, right?

Those willing to pay the extra price of doing TDD can get to it. Those running after new features every two weeks will have a harder time, of course.

P.S.
See a follow-up post “JUnit Updates”.

 
2 Comments

Posted in Code