We use maven 2 here for everything. I love maven – I hate maven – but – it’s the best we have and it makes sense for open source projects. Once you know maven – you know it. Everyone I know who has switched resisted it at first because Maven 1.x stank but they are all glad they made the shift to Maven2. This is another area where we can seek de facto based standards and reduce the barrier to entry for Alfresco.
That said… my gripes with maven are (Maven has some flaws that have nothing to do with Maven per se.):
First is the issue of Sun Jars which are not allowed to be housed in the Maven repository or any other repository because of some silliness on the part of Sun. Second are all the other jars that are not in the maven repository because vendors (cough-fresco-cough <GRIN>) have not made them available.
The second issue is that the central maven repository is a superfund site where people have managed to place the same libraries at 30 different groups and names – this is mavens fault because they don’t seem to manage the issue.
The third issue is that Eclipse plug-ins for maven are Terrible (yes – with a capital ‘T’) but they are just as good as any support for home cooked ant builds so I am not sure it matters – we just don’t gain much on the eclipse front. Netbeans however seems to have nice tools (and a profiler) so in some ways we are gaining there for people who use netbeans.
The last issue I can think of is that working offline with maven has sometimes bitten me because I can't reach a dependency – this is not much different that just not having the Jar on hand except that with Maven you don't ship with the jars the way the Alfresco source is currently shipped – good for the download, it's small if you just want to sniff around the source but it does mean you must be on-line for your first build.
Other than that – my concerns are with logistics:
We need a strong plan with guidelines for defining groups, artifact ids, dependencies and other practices.
We need a plan for the overall source configuration – to be honest Alfresco is pretty well suited for a port: we already have things broken down in to sub projects.
I think we need a proof of concept that demonstates
* nightly builds will continue to work
* junit tests will continue to run
* the sun will continue to rise and so fourth
The actual porting of the code to a Maven2 build is simple and we could do it very quickly with the help of a few – that isnt the hard part. The hard part would be actually making the switch.
Alfresco's engineering team has to be on board and has to participate in moving and the community has to be on board and know whats happening and when its going to happen and then have support after it has happened.
That is the tough part – making the change where it touches the people
If we do decide to make the move (aka there is strong support and the POC works well) we would need to move with the quickness (kung-fu sword noises here) and make this happen. We can't sit on the fence with two builds, it is too far down the stack and way too important to mess around. It is also essentially a fork until it is one build again … so once we decide to go (should we decide to go) we need to execute the act quickly.
That said… my gripes with maven are (Maven has some flaws that have nothing to do with Maven per se.):
First is the issue of Sun Jars which are not allowed to be housed in the Maven repository or any other repository because of some silliness on the part of Sun. Second are all the other jars that are not in the maven repository because vendors (cough-fresco-cough <GRIN>) have not made them available.
The second issue is that the central maven repository is a superfund site where people have managed to place the same libraries at 30 different groups and names – this is mavens fault because they don’t seem to manage the issue.
The third issue is that Eclipse plug-ins for maven are Terrible (yes – with a capital ‘T’) but they are just as good as any support for home cooked ant builds so I am not sure it matters – we just don’t gain much on the eclipse front. Netbeans however seems to have nice tools (and a profiler) so in some ways we are gaining there for people who use netbeans.
The last issue I can think of is that working offline with maven has sometimes bitten me because I can't reach a dependency – this is not much different that just not having the Jar on hand except that with Maven you don't ship with the jars the way the Alfresco source is currently shipped – good for the download, it's small if you just want to sniff around the source but it does mean you must be on-line for your first build.
Other than that – my concerns are with logistics:
We need a strong plan with guidelines for defining groups, artifact ids, dependencies and other practices.
We need a plan for the overall source configuration – to be honest Alfresco is pretty well suited for a port: we already have things broken down in to sub projects.
I think we need a proof of concept that demonstates
* nightly builds will continue to work
* junit tests will continue to run
* the sun will continue to rise and so fourth
The actual porting of the code to a Maven2 build is simple and we could do it very quickly with the help of a few – that isnt the hard part. The hard part would be actually making the switch.
Alfresco's engineering team has to be on board and has to participate in moving and the community has to be on board and know whats happening and when its going to happen and then have support after it has happened.
That is the tough part – making the change where it touches the people
If we do decide to make the move (aka there is strong support and the POC works well) we would need to move with the quickness (kung-fu sword noises here) and make this happen. We can't sit on the fence with two builds, it is too far down the stack and way too important to mess around. It is also essentially a fork until it is one build again … so once we decide to go (should we decide to go) we need to execute the act quickly.
The inter-project dependencies are a needless pain in the neck
for developers. It ends up forcing certain things to live in
places because of when they're built, rather than what would
be the ideal logical organization. In at few cases, it forces
us to do lazy-init within spring…etc. Cruft like this only gets worse
over time, so I'd like us to fix now while it's still relatively easy.
This is a developer's perspective, not a user's… but that matters too!
I'd like to learn a lot more about Maven II. If it can meet all our
requirements, and most folks prefer it, then that's great. Ant is ok,
but I don't love it. That said, I think we could be using Ant a lot
better than we are currently.
javac can see everything at once means perfect build dependencies
with 0 effort forever. Otherwise we're right back to:
Ugh. The picture I'm painting isn't that bad yet, but it's always there,
gnawing, gnawing, gnawing….
Regarding your concern about many jar files, you can actually
compose jars, so there can actually be *fewer* of them in the
final product regardless of the intermediate jars you choose to
create (or not). If we go the many-smaller-jars route in the
final product, then the way of handling compatability is via
a major/minor numbering scheme. Tried and true. The existence
proof: UNIX libs (kudos to Sun for getting it right).
Check out One-JAR:
http://www.ibm.com/developerworks/library/j-onejar/index.html?ca=drs-j4904
As far as I'm concerned, everything is on the table.
For example, the way we're using Ant currently, source code
that belongs to the same package cannot live in the same directory,
due to build dependency issues. That's not a fundamental problem
with Ant, but rather with our current use of Ant. It's my understanding
that Maven II lets you do things at a higher level than Ant, but sometimes
that can mean loss of lower-level control.
For example, some tools force you to build things a certain way,
and if your goals and their model are badly matched, the tool gets
in the way). I'm not saying that's the case with Maven II,
but I don't know for sure that it's not either… yet.
Being able to build from a read-only source is great.
It adds a lot of flexibility to the system in terms of
permissions, media types, etc. It's also just cleaner
(e.g.: we'll be able to get rid of a lot of svn:ignore tags).
As for build speed, one javac is faster than n, and only ever builds
exactly what's out of date. When you have n projects, you've got
to be a lot more careful, and even then people tend to throw in a
"clean" operation because they get scared. As soon as you do that,
you're rebuilding way more than you really need to; if you fail to
do that when things are slightly wrong in the dependencies
(which is easy to do), then you waste time in a different way…
which is no fun either.
Essentially, as soon as you break things up into multiple projects,
you take ownership of hand-crafting dependencies rather than just
letting the compiler do its job. It's bad for exactly the same reasons
recursive make is bad.
How these compiled classes are *packaged* however, is a totally
orthogonal issue, so long as the build framework supports multiple
artifacts per project. In Maven I, that was an issue, but I don't
know what the deal is with Maven II. Some stuff needs to go
into server/lib while other stuff needs to be in common/lib,
so this actually is a hard requirement. I don't want to be forced
into a different project / javac invocation just because the build
framework thinks it knows better and refuses to generate multiple
artifacts in one project. If that's the case with Maven II, it's a
non-starter in my view.
Again, this isn't something we've had a chance to discuss a lot
internally, so I'm not summarizing a collectively held viewpoint,
nor am I even expressing misgivings regarding Maven II.
For all I know right now, I'd love it. That said, this is at least
part of the checklist I'll use when I get around to forming my
own opinion on the topic. I'm sure there will be other stuff
too, such as learning curve, interop with tools, how others
in the community & engineering feel about it all, and so forth.
Well, you asked for my opinion, and all you got was my meta-opinion!
I hope to do better than that soon.
Cheers,
-Jon
for developers. It ends up forcing certain things to live in
places because of when they're built, rather than what would
be the ideal logical organization. In at few cases, it forces
us to do lazy-init within spring…etc. Cruft like this only gets worse
over time, so I'd like us to fix now while it's still relatively easy.
This is a developer's perspective, not a user's… but that matters too!
I'd like to learn a lot more about Maven II. If it can meet all our
requirements, and most folks prefer it, then that's great. Ant is ok,
but I don't love it. That said, I think we could be using Ant a lot
better than we are currently.
This doesn't seem to be a build harness question per se, but rather a packaging question.It's not just about the packaging. Structuring things so that one big
javac can see everything at once means perfect build dependencies
with 0 effort forever. Otherwise we're right back to:
class X needs to be built before Y, but it needs class P
which must be put in location Q but then that generates a circular
dependency that Ant can handle but Eclipse can't… unless you
sprinkle lazy inits here and there in various spring config files…
and then muck around with .classpath files used by the continuous
builds done by IDEs such as Eclipse… and… and….
Ugh. The picture I'm painting isn't that bad yet, but it's always there,
gnawing, gnawing, gnawing….
Regarding your concern about many jar files, you can actually
compose jars, so there can actually be *fewer* of them in the
final product regardless of the intermediate jars you choose to
create (or not). If we go the many-smaller-jars route in the
final product, then the way of handling compatability is via
a major/minor numbering scheme. Tried and true. The existence
proof: UNIX libs (kudos to Sun for getting it right).
Check out One-JAR:
http://www.ibm.com/developerworks/library/j-onejar/index.html?ca=drs-j4904
As far as I'm concerned, everything is on the table.
> Make the source tree reflect the java package organizationIt depends on the underlying capabilities of that harness.
This is independent of the build harness correct?
For example, the way we're using Ant currently, source code
that belongs to the same package cannot live in the same directory,
due to build dependency issues. That's not a fundamental problem
with Ant, but rather with our current use of Ant. It's my understanding
that Maven II lets you do things at a higher level than Ant, but sometimes
that can mean loss of lower-level control.
For example, some tools force you to build things a certain way,
and if your goals and their model are badly matched, the tool gets
in the way). I'm not saying that's the case with Maven II,
but I don't know for sure that it's not either… yet.
Being able to build from a read-only source is great.
It adds a lot of flexibility to the system in terms of
permissions, media types, etc. It's also just cleaner
(e.g.: we'll be able to get rid of a lot of svn:ignore tags).
As for build speed, one javac is faster than n, and only ever builds
exactly what's out of date. When you have n projects, you've got
to be a lot more careful, and even then people tend to throw in a
"clean" operation because they get scared. As soon as you do that,
you're rebuilding way more than you really need to; if you fail to
do that when things are slightly wrong in the dependencies
(which is easy to do), then you waste time in a different way…
which is no fun either.
Essentially, as soon as you break things up into multiple projects,
you take ownership of hand-crafting dependencies rather than just
letting the compiler do its job. It's bad for exactly the same reasons
recursive make is bad.
How these compiled classes are *packaged* however, is a totally
orthogonal issue, so long as the build framework supports multiple
artifacts per project. In Maven I, that was an issue, but I don't
know what the deal is with Maven II. Some stuff needs to go
into server/lib while other stuff needs to be in common/lib,
so this actually is a hard requirement. I don't want to be forced
into a different project / javac invocation just because the build
framework thinks it knows better and refuses to generate multiple
artifacts in one project. If that's the case with Maven II, it's a
non-starter in my view.
Again, this isn't something we've had a chance to discuss a lot
internally, so I'm not summarizing a collectively held viewpoint,
nor am I even expressing misgivings regarding Maven II.
For all I know right now, I'd love it. That said, this is at least
part of the checklist I'll use when I get around to forming my
own opinion on the topic. I'm sure there will be other stuff
too, such as learning curve, interop with tools, how others
in the community & engineering feel about it all, and so forth.
Well, you asked for my opinion, and all you got was my meta-opinion!
I hope to do better than that soon.
Cheers,
-Jon
The inter-project dependencies are a needless pain in the neck
for developers. It ends up forcing certain things to live in
places because of when they're built, rather than what would
be the ideal logical organization. In at few cases, it forces
us to do lazy-init within spring…etc. Cruft like this only gets worse
over time, so I'd like us to fix now while it's still relatively easy.
This is a developer's perspective, not a user's… but that matters too!
The comments on the build dependencies are really interesting for me and I am going to give them a lot of thought.
I've always liked the idea of keeping things in separate projects and using maven to include what needs to be included. I tend to think of it as a virtue that each piece only gets compiled with what it needs and what it needs is either declared or missing (in the POM file) in which case [need not declared] an explosion will occur. The pom file is a sort of documentation on the cohesion of the code. I also like the fact that you can enforce certain layering and strict use of APIs across libraries (check for cohesion with implementation) by breaking things in to sub projects and declaring dependencies. I haven't fretted much that I am taking some of the dependency management off the shoulders of javac but then, I've never worried too much about build time – which can become an issue if you have to rebuild and you freak and run a clean – and some of the symptoms you mention (like clean) are actualities.
Really good stuff Jon. I have a lot to think about here because what you are saying is very much different then what I am currently doing. We have a lot of small projects linked together by dummy pom files and Maven is responsible for determining the build order. I always like having to come from a completely different direction and examine how we are doing things. One of the reasons I though Maven would be a natural fit for Alfresco was the fact that Alfresco is currently split up in a similar fashion.
So if I get what you are saying: you would like packages to be atomic so that they can be individually built and deployed as a patch, but that the whole project can also be built in one javac.
It’s interesting because really never thought of Alfresco as a single project. I think of the core and repository as very different from the web client for example and I’ve always thought those lines would deepen with time. Now, again that is a bit of deployment versus source tree talk, and they are not the same thing so again, I may have to go rethink. I am stuck in that mind set because I do break my libraries up as separate projects and I build my applications as yet separate projects. In most cases the applications have only spring configuration, web configuration and a few domain classes and maven pulls in all the dependencies.
The fact that the Alfresco approach was similar (not the same but not too far off) re-enforced my thought process around that kind of organization. I’ve been aware of at least some of the basic CONS for some time but I have always felt the PROS outweighed them. I’ll be interested to hear what others throw in on this subject but I really appreciate the thoughts you have put here especially because they are very different from my own on the subject.
