It is so handy to keep a bunch of Groovy Beans in the same file!
"Beans.groovy":
class Artifactory
{
String name
String repository
boolean deployArtifacts = true
String user
String scrambledPassword
}
class Mail
{
String recipients
boolean sendForUnstable = true
boolean sendToIndividuals = true
}
...
String.eachLine(Closure closure) allows to iterate a String line by line:
"""aaaaaaaaaaaaa
bbbbbbbbbbbbb
ccccccccccccc""".eachLine { println "[$it]" }
// Produces:
[aaaaaaaaaaaaa]
[bbbbbbbbbbbbb]
[ccccccccccccc]
Similarly, File.eachFile(groovy.lang.Closure) iterates over all directory files:
new File( "." ).eachFile{ println "[${ it.canonicalPath }]" }
// Produces:
[D:\Winny\java\groovy-1.7.4\bin\grape]
[D:\Winny\java\groovy-1.7.4\bin\grape.bat]
[D:\Winny\java\groovy-1.7.4\bin\groovy]
[D:\Winny\java\groovy-1.7.4\bin\groovy.bat]
...
In fact, Object.each(groovy.lang.Closure) is the most general way to iterate over “parts” of an object:
new File( "groovy.bat" ).withReader { it.each{ println "[$it]" }}
// Produces:
[@if "%DEBUG%" == "" @echo off]
[]
[@rem ]
[@rem $Revision: 19763 $ $Date: 2010-04-05 14:33:24 +0200 (Mo, 05. Apr 2010) $]
[@rem ]
...
Here I use a fact that File.withReader(groovy.lang.Closure) passes java.io.LineNumberReader into a closure and its "each" iterates over lines. That’s how we get each line of "groovy.bat" printed.
It actually became a habit to "each { .. }" Groovy objects to see the iteration $it.
But wouldn’t it be better if "each" was splitting objects to chunks, returning a list of them ?
Object.metaClass.splitWith = { String methodName ->
MetaMethod m = delegate.metaClass.pickMethod( methodName, Closure.class )
assert m
def result = []
m.doMethodInvoke( delegate, { result << it } )
result
}
def files = new File( "d:/Winny/java/groovy-1.7.4" ).splitWith( 'eachFileRecurse' )
println "Largest file is of size ${ files*.length().max() } bytes"
// Produces:
Largest file is of size 5370870 bytes
Now we can “split” any object using its iterating methods:
// There are 15458 directories in "C:/Windows"
assert 15458 ==
new File( "C:/Windows" ).splitWith( 'eachDirRecurse' ).size()
// There are 14 non-empty lines in "groovy.bat"
assert 14 ==
new File( "groovy.bat" ).text.splitWith( 'eachLine' ).findAll{ it }.size()
// There are 24 permutations of [ 1, 2, 3, 4 ]
assert 24 ==
[ 1, 2, 3, 4 ].splitWith( 'eachPermutation' ).size()
// Splitting an Iterator with "each" bring back the original List
assert [ 1, 2, 3, 4 ] ==
[ 1, 2, 3, 4 ].iterator().splitWith( 'each' )
// Cleanup file data by trim()-ing and filtering out empty lines
new File( .. ).text.splitWith( 'eachLine' )*.trim().findAll{ it }
splitWith() allows one to break an object into chunks of data without having to make a full stop and iterate, storing away each $it in a List:
println [ 1, 2, 3, 4 ].splitWith( 'eachPermutation' ).size()
// vs.
def result = []
[ 1, 2, 3, 4 ].eachPermutation{ result << it }
println result.size()
The real beauty of Delicious lies in keywords combinations.
I mentioned already that Delicious is my #2 most favorite productivity tool. Within time I developed a set of the most frequently-used keywords in the right column that can be combined with tags in the left column:
"hudson" |
- | Mailing list | - | "ml" |
"spring" |
- | Issue tracker | - | "jira" |
"ant" |
- | API documentation | - | "api" |
"jfrog" |
- | Documentation | - | "doc" |
"groovy" |
- | Maven repository | - | "repo" |
"maven" |
- | Source code browsing | - | "code" |
"d"-keyworded Delicious search I only need to type "d spring api" to get to
Lots of other useful combinations are also available:
"d X ml" or "d X api". Fast!
Artifactory’s REST API is something I was using quite a lot recently and would like to share my experience here. Normally, there’s no need to turn to the REST API when working with Artifactory and in most cases Maven, Hudson or TeamCity plugins are the only ones “talking” to it.
But in this specific case Artifactory was used as a general storage for company’s binaries (a capability I praised a lot in the past): Java agent downloads packed Lucene indexes from Artifactory where they are uploaded to by a Hudson job running Maven and Spring Batch.
It is an interesting and very educating project which I’ll surely explore in more details later. Today, I would like to talk about the REST part of it, the way it is used with a SpringTemplate and some Artifactory-specific nuances, marked with
1. Choosing the client.
Initially I was considering Jersey as a client library knowing JFrog teams uses it to provide Artifactory’s REST support. But then I decided to stick to Spring since version 3 was already used throughout the project and one of its new features is exactly what I needed here – a RestTemplate. Why not give it a try?
2. DAO.
Accessing any external resource is normally done through a DAO layer. Accessing Artifactory shouldn’t be an exception to this rule so we have a DAO interface and an ArtifactoryDAO implementation.
public interface DAO
{
int getLatestVersion ( String serviceName, String dataType );
File download ( File dir, String serviceName, String dataType, int version );
void deleteOldVersions( String serviceName, String dataType, int oldVersions );
}
serviceName and dataType are our analogies to Maven’s groupId and artifactId. The interface allows a client to get a number of the latest version, download any version locally and delete old versions.
3. Spring time!
Java client gets a DAO implementation injected to it, which in turn gets a RestTemplate injected as well, configured with a custom message converter, more on that below.
<bean ... >
<constructor-arg name = "dao" ref = "artifactoryDAO"/>
</bean>
<bean name="artifactoryDAO" class="...">
...
<constructor-arg name = "rest">
<bean class="org.springframework.web.client.RestTemplate">
<property name="messageConverters">
<list>
<bean class="JsonMessageConverter"/>
</list>
</property>
</bean>
</constructor>
</bean>
4. Working with a RestTemplate.
As one would expect, working with a RestTemplate is pretty straightforward. From all convenient methods available I picked up an exchange() one to provide a general-usage wrapper:
<T> T request( String url, HttpMethod method, Class<T> responseType )
{
HttpHeaders headers = new HttpHeaders();
headers.set( "Accept", "application/json" );
headers.set( "Authorization", auth());
T response = rest().exchange( url,
method,
new HttpEntity<String>( headers ),
responseType ).getBody();
assert (( response != null ) &&
( responseType.isAssignableFrom( response.getClass())));
return response;
}
There is a need to specify a correct "Accept" header since RestTemplate sends "Accept: text/plain" by default which is not accepted by Artifactory.
With built-in HttpMessageConverter and its sub-classes there’s no need to worry about conversion of a REST response, JSON in our case, to an object:
Map<String, Object> m =
request( "http://artifactory/api/storage/repo/serviceName/dataType",
HttpMethod.GET,
Map.class );
Last request() argument defines a type of an object to which response will be converted, considering its "Content-Type" and converters available. I was mostly using a Map when reading Artifactory responses, like “folder info”.
There is a need to extend MappingJacksonHttpMessageConverter in order to recognize "application/vnd.org.jfrog.artifactory.storage.folderinfo+json", returned by Artifactory as JSON:
import org.springframework.http.MediaType;
import org.springframework.http.converter.json.MappingJacksonHttpMessageConverter;
public class JsonMessageConverter extends MappingJacksonHttpMessageConverter
{
@Override
public boolean canRead ( Class<?> clazz, MediaType mediaType )
{
return
( super.canRead( clazz, mediaType ) ||
(( "application".equals( mediaType.getType())) &&
( mediaType.getSubtype().startsWith( "vnd.org.jfrog.artifactory" ))));
}
}
5. Reading InputStream and downloading files.
Message converters work really nice if response is not large and fits into memory. But how about downloading files? I can not assume their content will fit into memory and would like to access a raw InputStream.
It is also possible but in a different way, using an execute() method with two callbacks:
import org.apache.commons.io.IOUtils;
...
private static final RequestCallback ACCEPT_CALLBACK =
new RequestCallback()
{
@Override
public void doWithRequest ( ClientHttpRequest request ) throws IOException
{
request.getHeaders().set( "Accept", "application/json" );
}
};
private static class FileResponseExtractor implements ResponseExtractor<Object>
{
private final File file;
private File file () { return this.file; }
private FileResponseExtractor ( File file )
{
this.file = file;
}
@Override
public Object extractData ( ClientHttpResponse response ) throws IOException
{
InputStream is = response.getBody();
OutputStream os = new BufferedOutputStream( new FileOutputStream( file()));
IOUtils.copyLarge( is, os );
IOUtils.closeQuietly( is );
IOUtils.closeQuietly( os );
return null;
}
}
...
rest().execute( downloadUrl,
HttpMethod.GET,
ACCEPT_CALLBACK,
new FileResponseExtractor( new File( downloadDir, fileName )));
Note that to download an artifact with a REST "/api/download/" method requires Artifactory Power Pack license but you can also download an artifact with a regular URL like
"http://artifactory/repo/serviceName/dataType/version/dataType-version.tar".
6. Downloading large files.
I blogged recently about ServletResponse.setContentLength() problem: it doesn’t work with files larger than 2Gb and sends a negative "Content-Length" header due to int overflow.
This problem has also appeared in Artifactory bit it is fixed already, we just need to wait for version "2.2.6" to be released.
Spring 3 uses Apache HttpClient downloading an empty file if negative "Content-Length" header is sent. So I could not download files larger than 2Gb with Spring but I could with wget!
import org.apache.commons.exec.CommandLine;
import org.apache.commons.exec.DefaultExecutor;
...
String command = String.format(
"wget -S -nv -O \"%s\" -T 120 \"%s\" --cache=off %s",
destinationFile, sourceFileUrl, credentials );
int exitCode = new DefaultExecutor().execute( CommandLine.parse( command ));
As it appears, "wget" works just fine with a negative "Content-Length" which, btw, is not that unusual and can be sent in those cases when response length is not known in advance.
7. Deleting artifacts.
Artifactory’s API doesn’t really mention it but artifacts or folders can be deleted as well, provided client has enough permissions to do so, of course:
request( "http://artifactory/repo/serviceName/dataType",
HttpMethod.DELETE,
Map.class );
8. Security.
HTTP calls can be authenticated by overriding Spring’s CommonsClientHttpRequestFactory and configuring RestTemplate to use it:
public class ArtifactoryCommonsClientHttpRequestFactory
extends CommonsClientHttpRequestFactory
{
private final String user;
private final String password;
public ArtifactoryCommonsClientHttpRequestFactory ( String user,
String password )
{
this.user = user;
this.password = password;
}
@Override
public HttpClient getHttpClient ()
{
HttpClient client = super.getHttpClient();
if ( this.user != null )
{
client.getState().setCredentials(
AuthScope.ANY,
new UsernamePasswordCredentials( this.user, this.password ));
}
return client;
}
}
<bean class="org.springframework.web.client.RestTemplate">
<property name = "requestFactory">
<bean class = "ArtifactoryCommonsClientHttpRequestFactory">
<constructor-arg name="user" value="..."/>
<constructor-arg name="password" value="..."/>
</bean>
</property>
...
</bean>
Artifactory encrypted password can be specified as a "password".
But this approach didn’t work well in all scenarios, particularly in DELETE calls which went out unauthorized so I switched to setting an "Authorization" HTTP header, as shown in “Working with a RestTemplate” section above.
9. Related links.
org.springframework.web.client.RestTemplateOverall, it took some time to implement ArtifactoryDAO and sort out all issues involved: Spring converters, request and response headers, authentication, downloading large files, etc .. But it now works extremely well, response time of each REST invocation is normally below 10 ms and Artifactory Hudson plugin eats 2.5Gb files for breakfast!
Configuration Management was always my passion.
Somehow, I have always had this thing for builds. I don’t know why, it just happens to be that way. Over the last 10 years I have accepted various development positions, but very few were actually related to CM. Occasionally, I accepted CM tasks for the joy of it. But there were other reasons: most of the time any given software project is a mess at the CM level. Secondly, most projects on which I was working had no one assigned to this job.
Somehow, builds were supposed to take care of themselves. However in practice this does not happen magically by itself: builds are usually written by some poor guy, temporarily assigned to this task with one goal in mind; to get it over with as soon as possible. And I don’t blame him.
It made me think. What’s wrong here? Why does it happen this way, that very few projects can demonstrate well written build logic, where code is reused and there’s no place to copy-paste? Why do some managers believe there’s absolutely no need to assign a dedicated resource to deal with CM issues and it can be easily taken care of by any developer? And why when these managers experience a messy environment they are still not convinced otherwise?
Lots of questions. I’m going to try and answer some of them.
Why does it happen this way that very few projects can demonstrate well written build logic, where code is reused and there’s no place to copy-paste?
"if". Lack of built-in reuse functionality made people copy-paste tons of code without ever trying to organize it.
It was Ant which made an awfully wrong assumption: “There’s no place for "if"s and loops in your builds. Builds should be straightforward and go from A to B to generate a C, no questions asked.” Ant-Contrib was a savior here but reusing someone else’s code was still not an option; <macrodef>s only brought their own issues. Eventually, people turned to “copy-paste”, the best re-usability solution around.
In Maven, conditional logic and reuse are also far from perfect. To a very limited extent conditional logic can be done with profiles and Ant/Groovy snippets while reuse is provided by <parent>-ing other POMs. I believe this is not flexible and elegant enough. In addition, Maven declarative nature makes any custom build tweaks extremely hard to implement: if there’s no attribute for what you want – you’re out of luck. The only sensible way to deal with all those limitations is to develop Maven plugins for every customized need you might have. Only then one can keep his POMs sane, not blowing to zillions of Ant tasks. But plugin development is not accepted readily by most developers so they usually turn to "maven-antrun-plugin" and .. here we have all Ant issues again.
The problem with major build tools we have had for years is that they are mostly about straightforward build steps rather than about intelligent build logic. They lack basic build language which is nicely taken care of today by Gradle which has realized that builds are programs and deserve their own build language and build model. When Hans Dockter, the creator of Gradle explained that builds always have their own logic and should be taken as seriously as a production code, I was extremely happy to hear that. Finally, there’s a tool creator who gets it.
Why do some managers believe there’s absolutely no need to assign a dedicated person to deal with CM issues?
I have no doubt if I had to measure benefits of having a good CM it will always outweigh the mere salary.
Why can’t you split CM tasks evenly between all developers? It can be done by anybody, right?
What are CM responsibilities?
As one can see, the list is not short and it is far from being a naive one-liner like “Write Ant scripts” as some managers may conceive CM duties.
What are the skills required to perform the above duties well?
In conclusion, I would say that there is no substitute for a good CM in any serious organization that prides itself and has an above average development environment. I tend to compare the CM’s position to that of a building architect who is in charge of the overall construction design, its services, function, maintenance, security, and infrastructures. Someone needs to have a bird’s-eye view of all development activities. In theory Software Architects should be fulfilling this job but in practice it is CM role.
By default, WordPress generates the following feed link:
<link type="application/rss+xml" href="http://evgeny-goldin.com/blog/feed/"/>
After registering this feed at FeedBurner I wanted "http://feeds.feedburner.com/GoldinTheJunior" to replace the link above. Since my current theme, ChocoTheme, doesn’t provide an option to configure the feed link, I had to make some changes in the code.
1. Changing RSS icon link.
"wp-content/themes/chocotheme/header.php":
2. Changing <link type="application/rss+xml" href="..."/> link.
"wp-includes/general-template.php":
This will take you to the FeedBurner if you use Google reader RSS Subscriber or any other browser tool to subscribe to the site’s RSS feed.
3. Feed redirect.
Redirect of "evgeny-goldin.com/blog/feed" to "feeds.feedburner.com/GoldinTheJunior" is possible:
RewriteRule in .htaccess file FeedBurner stopped pulling feed updates. It discovered a loop and stopped requesting the original URL. Indeed, if all feeds are redirected to a FeedBurner how does it get an original feed?
It is taken care by user agent conditions:
RewriteCond %{HTTP_USER_AGENT} !FeedBurner [NC]
RewriteCond %{HTTP_USER_AGENT} !FeedValidator [NC]
But I’ve settled with code modifications for now. It means I’ll need to take more care of WordPress upgrades as they will probably overwrite the “patches” shown above but I don’t mind re-applying them again after each upgrade. The fact that WordPress is developed in PHP makes it ridiculously easy to tweak its code where needed.
I want to specially thank Ms. Ileane Speaks and her new blog “Basic Blog Tips” for working on this issue together in “Basics of Blog Feeds and FeedBurner”. Thank you, Ileane!
Ant FTP task provides no option to display how much the download has progressed so far, similarly to wget or curl verbose options. Setting connection timeout isn’t possible either.
But I had to implement it one day and here is what I have found out:
org.apache.commons.net.ftp.FTPClientorg.apache.commons.net.io.CopyStreamListenerFTPClient doesn’t use it and passes null to org.apache.commons.net.io.Util.copyStream()FTPClient making use of CopyStreamListener when calling Util.copyStream(). Let’s do it!
1. New FTP task:
The biggest problem in the whole story is the following line of code: ftp = new FTPClient(). I wish there were a simple IoC-like way to “inject” an FTPClient instance here but Ant does not provide too many ways to customize its default configuration, something like RestTemplate.setRequestFactory(). Alas!
I really wish more people would embrace Spring/IoC way of thinking: everything is a Strategy and therefore can be “injected”, extended, modified, and customized.
So my first attempt was to extend Ant’s org.apache.tools.ant.taskdefs.optional.net.FTP class, override execute() method, copy its original code and replace the assignment. As much as I have no taste for copying someone else’s code in order to modify a line or two, sometimes this is the only way to go. But it didn’t work! execute() method accesses lots of private members, not accessible from a sub-class.
The next option was to copy an entire FTP class and then replace the assignment. If anyone can suggest a better way to do the same I’m all ears. So I ended up with:
public class FTP extends Task
{
...
public void execute() throws BuildException {
...
org.apache.commons.net.ftp.FTPClient ftp = null;
...
ftp = ( verbose ? new FTPClient( getProject()) :
new org.apache.commons.net.ftp.FTPClient());
ftp.setDataTimeout( 5 * 60 * 1000 ); // 5 minutes
...
}
}
Line 08 sets a customized version of FTPClient or a default one, line 10 sets up FTPClient timeout.
2. New FTPClient:
public class FTPClient extends org.apache.commons.net.ftp.FTPClient
{
private final Project project; // org.apache.tools.ant.Project
...
@Override
public boolean retrieveFile(String remote, OutputStream local)
throws IOException
{
...
Util.copyStream( input,
local,
getBufferSize(),
CopyStreamEvent.UNKNOWN_STREAM_SIZE,
new CopyStreamListener( this.project ),
false );
...
}
}
Again, I had to copy the original code and “patch” Util.copyStream() call to use CopyStreamListener implementation. Also, I had to copy and slightly modify an FTPConfigurator class to make it accept my new FTPClient.
3. CopyStreamListener:
public class CopyStreamListener implements org.apache.commons.net.io.CopyStreamListener
{
private final Project project;
private long mbTransferred = 0;
...
@Override
public void bytesTransferred ( long totalBytes, int bytes, long streamSize )
{
long mb = ( totalBytes / ( 1024 * 1024 ));
if ( mb > mbTransferred )
{
mbTransferred = mb;
this.project.log( "[" + new Date() + "]: [" + mbTransferred + "] Mb transferred" );
}
}
}
4. Plugging in new definition of FTP task with ClfAntBuilder:
public class ClfAntBuilder extends AntBuilder
{
public ClfAntBuilder ()
{
super();
getProject().addTaskDefinition( "ftp", FTP.class );
}
}
5. Running it:
new ClfAntBuilder().ftp( action : 'get',
server : host,
userid : username,
... )
{
fileset( dir : '...',
includes : '...',
excludes : '...' )
}
6. The final result:
Trying to override old definition of task ftp
[ftp] getting files
[ftp] transferring SomeFile.xml.zip to C:\Temp\SomeFile.xml.zip
[Tue Aug 17 20:43:07 IDT 2010]: [1] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [2] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [3] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [4] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [5] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [6] Mb transferred
[Tue Aug 17 20:43:07 IDT 2010]: [7] Mb transferred
...
Nice!
javax.servlet.ServletResponse.setContentLength(int) is a new kind of Y2k bug. It accepts int assuming no files larger than 2Gb will ever be sent as an HTTP response.
If you pass a long larger than 231-1 which is exactly 2Gb, it overflows and negative header is sent in response.
Free Download Manager fails to download a file and Apache Commons HttpClient v3.1 downloads an empty one (need to check version 4)
Too bad Java SE did the same mistake and java.net.URLConnection.getContentLength() returns int as well, it means the API will never be changed.
Right now the solution is to use HttpServletResponse.setHeader(String,String):
long length = file.length(); .. response.setHeader( "Content-Length", String.valueOf( length ))
Other APIs have taken the right route though:
org.springframework.http.HttpHeaders.getContentLength() – returns longorg.apache.http.HttpEntity.getContentLength() – returns long
JFrog’s “To Build or Not to Be” seminar was an exceptional one. Usually, there are very few events fully devoted to the subject of builds and build tools. Lucky us we had this one with so many key people of today’s build arena:
I allowed myself to ask participants some questions.
Sergey Anchipolevsky, Max Feldman – TeamCity:
“What does "Git (JetBrains)" Git integration of TeamCity mean?”
It’s just a version of Git plugin developed by JetBrains, to differentiate it from previous Git plugins.
“Is it possible to "tag" a YouTrack issue?”
That’s a YouTrack question but yes, you can tag an issue.
“Is it possible to backup YouTrack issues?”
Sure, you can schedule YouTrack database backups with a cron expression or run it manually.
“Is it possible to re-order YouTrack issues by drag-n-dropping them, similarly to how it is done in Nozbe?”
No, it can’t be done. YouTrack isn’t really a task management tool, like Nozbe. It is an issue tracker which is something different. The order between issues is specified by a "sort by" criteria so they can’t be arbitrarily re-ordered with drag-n-drop. But you should probably take a look at checkvist.com, a simpler task management tool developed by JetBrains developers.
“Ok, TeamCity question. What do you think of cloud-based TeamCity version, similar to Artifactory Online. Can it be done?”
Oh, that’s a good one. First of all, let’s get something clear. You can’t provide a SaaS solution for a build server. Normally, SaaS solutions, like Artifactory Online that you have mentioned, are based on a shared resources, hosted by provider. But something like a build server is too resource-hungry and can’t be shared with other users.
My comment: I agree. Experience shows how unshareable build servers are. You really want a dedicated machine for each one of them, trying to run too many builds on the same machine may quickly bring the server to its knees and overall response time drops dramatically.
But something else can be done. First of all, you can run build agents on EC2 today. Second, we can provide TeamCity images ready to be run on EC2 as well. But it’s always better if you run main TeamCity server inside your organization, on your own IT infrastructures. We haven’s seen too many customers willing to out-source the build server completely: networking issues, server configurations, additional setups, security .. it’s too much of a pain and, again, can hardly be shared between different users.
“I see, thank you. Is it possible to develop plugins for TeamCity?”
Sure!
Yoav Landman – Artifactory:
“After hearing your session about module development options in Java – what do you think of JPF? Is it somehow a player in today’s Java module-space?”
I looked at JPF several times before as an application level module system, but I don’t think it amounts to a Java language module system. You can incorporate it into an application but you can’t build or run a full application based strictly on JPF modules (last time I checked).
“You mentioned Java moduling system is not united and fragmented, we have lot’s of approaches and various repos available. Can “repo1″ be *the one*?”
It kind of is at the moment and there are no much alternatives available. I wish it would be more developer friendly, though.
“As people are running more and more Artifactory Online instances, does it contribute to overall module-space fragmentation? Don’t get me wrong, I love it and had a great success in providing a Maven support for the Groovy++ project with a help of groovypp.artifactoryonline.com. I have my own repo set up at evgeny-goldin.org/artifactory that I’m about to use for my personal projects. So it’s cool, by all means. Still, the fragmentation issue you have just mentioned during the session just made me wonder .. “
Having a central repository and a central URL to get things from is a great thing. However, Maven central brings some long-lasting issues with it: the existence of module developers is not obvious and artifacts cannot be linked back to them; publishing is not straightforward – rather than getting an account and a simple Web UI/REST deploy, you need to set up rsync or learn a commercial tool to do that – in both cases the final published artifact looses relation to its creator account; repositories provide no REST API, there’s no access to statistics, and mostly – no searches: rather than sending a query to Central you have to download index file and search on the client side. I think these are the main concerns that keep people managing their public repositories on apache/sf/google-code/svn/artifactory/nexus etc.
Hans Dockter – Gradle:
“Gradle is a truly awesome build tool. Are you planning to provide a way to invoke Maven plugins from Gradle?”
Yes, it is of very high priority, we will provide a way to “import” and run an existing "pom.xml"
“In fact, I was thinking more of invoking a plugin directly, given its <groupid>, <artifactid>, and <version>. Similarly to how it is done by Mojo Executor .. “
Ah, right, right, this one. Sure, it should be done as well, although it is not simple, you know. To invoke a Maven plugin you need to “bind” it with a Maven container but we may probably trick into thinking there’s a Maven container while it will be talking to a Gradle run-time environment. Anyway, it’s definitely on our TODO list.
“The reason I ask is to ease Maven to Gradle migration or simply re-use an existing Maven plugins, some of which are really good! Now, I believe Gradle is a great tool but it requires a serious understanding of what you’re doing. In some aspects, it reminds me of Git where some people consider it to be too complex sometimes”
Yes, there’s a need to provide a really simple guidance for a people to switch between the tools or adopt a new one, like Gradle. But you need to understand your tool, there’s no doubt about it.
…
It was time to say good bye. But I hope to see you all again!
A big thanks to Guy Nir and Shlomi Ben Haim for generously allowing me to use some of the photos they have made.
Spring Batch job definition:
<bean>
<property name="resources" value="file:/${path}/some.*.pattern.zip"/>
...
</bean>
"${path}/some.*.pattern.zip" is right, I can guarantee that. Double-checked!
So why Spring Batch complaints "No resources to read" and does nothing?
Argh .. probably, has something to do with files pattern matching.
All right, I was curious about Ant-like files pattern matching in non-Ant environments for a long time already.
How one does that?
For "maven-new-assembly-plugin" I’ve managed with "file-management" module:
<dependency>
<groupid>org.apache.maven.shared</groupid>
<artifactid>file-management</artifactid>
<version>1.2.1</version>
<scope>compile</scope>
</dependency>
import org.apache.maven.shared.model.fileset.FileSet; import org.apache.maven.shared.model.fileset.util.FileSetManager; FileSet fs = new FileSet(); fs.setDirectory( .. ); fs.setIncludes( .. ); fs.setExcludes( .. ); String[] files = new FileSetManager().getIncludedFiles( fs )
but that’s Maven. How is it done in Spring?
Well, debugging is always fun!
AbstractApplicationContext.getResources()PathMatchingResourcePatternResolver.getResources()PathMatchingResourcePatternResolver.findPathMatchingResources()PathMatchingResourcePatternResolver.doFindPathMatchingFileResources()PathMatchingResourcePatternResolver.retrieveMatchingFiles()PathMatchingResourcePatternResolver.doRetrieveMatchingFiles()AntPathMatcher.match()AntPathMatcher.doMatch() – a FishEye link. As I expected, the matching process is quite involved .."org.springframework.util.AntPathMatcher". Doesn’t surprise me, "org.springframework.util" package was always full of nice utilities. One more to a toolbox!
P.S.
"No resources to read" ?
"file:/" was not required:
<bean>
<property name="resources" value="${path}/some.*.pattern.zip"/>
...
</bean>
Previous version was converted to something like "file:c:\file:/", but this one worked just fine.
I guess this version would also work, according to PathMatchingResourcePatternResolver Javadoc:
<bean>
<property name="resources" value="file:${path}/some.*.pattern.zip"/>
...
</bean>
One more example where a single "/" is of crucial importance. I love this job!
