[Geotools-devel] Geotidy plan

[Martin Desruisseaux]

Hello all

Following on the JAI operation discussion, I mentioned that I'm working on
Operation2D / OperationJAI right now. Actually I'm porting them to geotidy. In
the process I rewrote yesterday the way the result of GridCoverage2D operations
can be cached (I realised that the implementation in GeoTools 2 is broken). But
as far as Operation2D / OperationJAI are concerned, I do not envision API change
at this time.

This bring the question about when geotidy can be offered for a merge with
GeoTools. Currently the referencing module is done, except for Oracle and HSQL
plugins of EPSG (I'm doing my test on PostgreSQL for now) and the builder
package, maybe to move in an other module.

I'm porting key classes from the coverage module right now (not all of them - I
will delay some of them to a later time). GridCoverage2D is alreay ported; the
main class left are Operation2D and OperationJAI, which I'm doing today.

We already have a Mercurial clone of GeoTools SVN:
http://hg.geomatys.com/geotools/trunk/. Later this week or at the beging of next
week, we will create a clone of this clone in which we will delete the following
modules:

* metadata
* referencing
* referencing3D
* epsg-*
* coverage
* coverageio
* widgets-swing
* widgets-swing-pending
* openoffice

and replace them by dependencies toward geotidy modules. At this stage the goal
will be to get GeoTools to compile and pass its tests, without consideration for
Java 5. Only after that point, we will downgrate geotidy to Java 5 on a "geotidy
for geotools" clone (or something like that). At this point the work will be
ready for offering to the community. If accepted, we can merge to SVN the
changes applied on the GeoTools Mercurial clone. The "geotidy for geotools" code
can be merged to the SVN, or we can left it on Mercurial. I would recommand the
later (Mercurial commands look like a lot SVN commands); anyone in the community
can take the initiative to declare his Mercurial clone as the "official" one for
GeoTools, with agreement of the others.

The space for the Mercurial clones is already in place:

http://hg.geomatys.com/

Just a note in order to avoid diplomatic incident: the name "geomessy" applies
to *our* (at geomatys) code, not to GeoTools code. The comment said "Modules
*for* GeoTools", not "Modules *from* GeoTools".

Regards,

Martin

[Jody Garnett]

Hi Martin it looks like you are getting close - I confess I have not
been paying attention until you you are ready to merge.

You already know the collaboration drill - as a module maintainer you
can make some decisions about the modules you are working on; for any
api changes there is the proposal process.

For my own sanity you may want to start putting together the proposal
covering API change a bit before you are ready to merge.

Jody

On Tue, Mar 24, 2009 at 8:37 PM, Martin Desruisseaux

Post by Martin Desruisseaux
Hello all
Following on the JAI operation discussion, I mentioned that I'm working on
Operation2D / OperationJAI right now. Actually I'm porting them to geotidy. In
the process I rewrote yesterday the way the result of GridCoverage2D operations
can be cached (I realised that the implementation in GeoTools 2 is broken). But
as far as Operation2D / OperationJAI are concerned, I do not envision API change
at this time.
This bring the question about when geotidy can be offered for a merge with
GeoTools. Currently the referencing module is done, except for Oracle and HSQL
plugins of EPSG (I'm doing my test on PostgreSQL for now) and the builder
package, maybe to move in an other module.
I'm porting key classes from the coverage module right now (not all of them - I
will delay some of them to a later time). GridCoverage2D is alreay ported; the
main class left are Operation2D and OperationJAI, which I'm doing today.
http://hg.geomatys.com/geotools/trunk/. Later this week or at the beging of next
week, we will create a clone of this clone in which we will delete the following
  * metadata
  * referencing
  * referencing3D
  * epsg-*
  * coverage
  * coverageio
  * widgets-swing
  * widgets-swing-pending
  * openoffice
and replace them by dependencies toward geotidy modules. At this stage the goal
will be to get GeoTools to compile and pass its tests, without consideration for
Java 5. Only after that point, we will downgrate geotidy to Java 5 on a "geotidy
for geotools" clone (or something like that). At this point the work will be
ready for offering to the community. If accepted, we can merge to SVN the
changes applied on the GeoTools Mercurial clone. The "geotidy for geotools" code
can be merged to the SVN, or we can left it on Mercurial. I would recommand the
later (Mercurial commands look like a lot SVN commands); anyone in the community
can take the initiative to declare his Mercurial clone as the "official" one for
GeoTools, with agreement of the others.
    http://hg.geomatys.com/
Just a note in order to avoid diplomatic incident: the name "geomessy" applies
to *our* (at geomatys) code, not to GeoTools code. The comment said "Modules
*for* GeoTools", not "Modules *from* GeoTools".
       Regards,
               Martin
------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
Geotools-devel mailing list
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[Martin Desruisseaux]

Hello Jody

Post by Jody Garnett
For my own sanity you may want to start putting together the proposal
covering API change a bit before you are ready to merge.

For now, I have put together two level of informations:

* An overview level, trying to give an overview of the
main changes without going in the details:

http://geotidy.geomatys.fr/build/tools/gt-migrate/changes.html

* For the classes that were renamed, I listed all classname
changes in a module that can be run from the command line
for updating automatically existing code:

http://geotidy.geomatys.fr/build/tools/gt-migrate/index.html

Martin

[Simone Giannecchini]

Ciao Martin,
now that you have reached the stage of playing with gridcoverages I
would like to ask you again what the plan is and how we can
collaborate, since we are about to start some more work on the same
topic (actually more on IO then on the core modules for the time
being) and I would not like to end up in duplicating the efforts or,
worse, having two different implementations inside geotools for
similar capabilities like it happened in the past already.

Simone.

On Tue, Mar 24, 2009 at 1:15 PM, Martin Desruisseaux

Post by Martin Desruisseaux
Hello Jody

Post by Jody Garnett
For my own sanity you may want to start putting together the proposal
covering API change a bit before you are ready to merge.

* An overview level, trying to give an overview of the
http://geotidy.geomatys.fr/build/tools/gt-migrate/changes.html
* For the classes that were renamed, I listed all classname
  changes in a module that can be run from the command line
http://geotidy.geomatys.fr/build/tools/gt-migrate/index.html
    Martin
------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
Geotools-devel mailing list
https://lists.sourceforge.net/lists/listinfo/geotools-devel

--
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Owner - Software Engineer
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: [phone]
fax: [phone]
mob: [phone]


http://www.geo-solutions.it
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------

[Martin Desruisseaux]

Hello Simone

For now I'm porting the GridCoverage classes with few changes (much less work
than I did for referencing), except replacing some deprecated methods by their
replacement. Once this part is done, we will merge geotidy with geotools as
described in my previous email, and we will test that on our server. The merge
will be public on the Mercurial repositories mentioned in my previous email.

Only after the merge with geotools, I plan to work on coverage I/O and postgrid.
They are related work since postgrid has its own, quite bad, I/O framework which
need to be replaced by a better one. The first thing I wish to do is IIOMetadata
for coverage, wich I have been unable to complete at this time.

I have not given any though about what a "CoverageReader" (or "CoverageStore",
or whatever) interface may look like, so the room is all open here. Postgrid may
be seen as a kind of CoverageStore backed by a PostgreSQL database, so I have a
little bit of experience here, but I absolutly don't consider the postgrid API
as a model. It has never been commited to geotools because I have never been
happy with its API, so again the room it pretty much open in this area.

In parallel to this work, I would like to make Operation2D/OperationJAI evolve
as we get suggestions from you, Mickael Bedward, Andrea Antonnelo or others. I
may be slow on this task, so it may be worth to consider really seriously
distributed versionning systems as a way to organize our work.

Martin

[Simone Giannecchini]

Ciao Martin,
- about distributed versioning, I have not had time yet to play with
it but It is something that I am pretty sure we should investigate
- about coverage IO and postgrid, I'll try to get back with some ideas
in the next weeks, for the moment I have to do some maintenance work
on the imagemosaic. Anyway, it would be great to reduce the number of
mosaic-like stuff and bring in postgrid somehow.


Simone.

On Tue, Mar 24, 2009 at 2:44 PM, Martin Desruisseaux

Post by Martin Desruisseaux
Hello Simone
For now I'm porting the GridCoverage classes with few changes (much less
work than I did for referencing), except replacing some deprecated methods
by their replacement. Once this part is done, we will merge geotidy with
geotools as described in my previous email, and we will test that on our
server. The merge will be public on the Mercurial repositories mentioned in
my previous email.
Only after the merge with geotools, I plan to work on coverage I/O and
postgrid. They are related work since postgrid has its own, quite bad, I/O
framework which need to be replaced by a better one. The first thing I wish
to do is IIOMetadata for coverage, wich I have been unable to complete at
this time.
I have not given any though about what a "CoverageReader" (or
"CoverageStore", or whatever) interface may look like, so the room is all
open here. Postgrid may be seen as a kind of CoverageStore backed by a
PostgreSQL database, so I have a little bit of experience here, but I
absolutly don't consider the postgrid API as a model. It has never been
commited to geotools because I have never been happy with its API, so again
the room it pretty much open in this area.
In parallel to this work, I would like to make Operation2D/OperationJAI
evolve as we get suggestions from you, Mickael Bedward, Andrea Antonnelo or
others. I may be slow on this task, so it may be worth to consider really
seriously distributed versionning systems as a way to organize our work.
       Martin

--
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Owner - Software Engineer
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: [phone]
fax: [phone]
mob: [phone]


http://www.geo-solutions.it
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------

[Justin Deoliveira]

Hi Martin,

Sounds like you are making good progress. That said I would like to see
a formal proposal with voting before any changes come back to geotools.
Unfortunately the forking nature of how this work has been done
represents a lot of risk to projects that depend critically on the
geotools referencing code.

I also think we need it to be easy to revert back to the current
referencing code. While I realize you have worked hard to maintain most
of the api and backward compatibility there will undoubtedly be
regressions. Those regressions could be "deal breakers" so depending
projects need a way to fall back onto the old code easily.

-Justin

Post by Martin Desruisseaux
Hello all
Following on the JAI operation discussion, I mentioned that I'm working on
Operation2D / OperationJAI right now. Actually I'm porting them to geotidy. In
the process I rewrote yesterday the way the result of GridCoverage2D operations
can be cached (I realised that the implementation in GeoTools 2 is broken). But
as far as Operation2D / OperationJAI are concerned, I do not envision API change
at this time.
This bring the question about when geotidy can be offered for a merge with
GeoTools. Currently the referencing module is done, except for Oracle and HSQL
plugins of EPSG (I'm doing my test on PostgreSQL for now) and the builder
package, maybe to move in an other module.
I'm porting key classes from the coverage module right now (not all of them - I
will delay some of them to a later time). GridCoverage2D is alreay ported; the
main class left are Operation2D and OperationJAI, which I'm doing today.
http://hg.geomatys.com/geotools/trunk/. Later this week or at the beging of next
week, we will create a clone of this clone in which we will delete the following
* metadata
* referencing
* referencing3D
* epsg-*
* coverage
* coverageio
* widgets-swing
* widgets-swing-pending
* openoffice
and replace them by dependencies toward geotidy modules. At this stage the goal
will be to get GeoTools to compile and pass its tests, without consideration for
Java 5. Only after that point, we will downgrate geotidy to Java 5 on a "geotidy
for geotools" clone (or something like that). At this point the work will be
ready for offering to the community. If accepted, we can merge to SVN the
changes applied on the GeoTools Mercurial clone. The "geotidy for geotools" code
can be merged to the SVN, or we can left it on Mercurial. I would recommand the
later (Mercurial commands look like a lot SVN commands); anyone in the community
can take the initiative to declare his Mercurial clone as the "official" one for
GeoTools, with agreement of the others.
http://hg.geomatys.com/
Just a note in order to avoid diplomatic incident: the name "geomessy" applies
to *our* (at geomatys) code, not to GeoTools code. The comment said "Modules
*for* GeoTools", not "Modules *from* GeoTools".
Regards,
Martin
------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
Geotools-devel mailing list
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[Martin Desruisseaux]

Hello Justin

Post by Justin Deoliveira
Sounds like you are making good progress. That said I would like to see
a formal proposal with voting before any changes come back to geotools.
Unfortunately the forking nature of how this work has been done
represents a lot of risk to projects that depend critically on the
geotools referencing code.

Sure, it sound normal. But it is not a "take it or leave it" deal; the merge
with GeoTools will exists publicly on Mercurial anyway, so anyone can create a
clone and applies his own changes before an eventual commit to the SVN. The
Mercurial repository to be merged to GeoTools SVN doesn't need to be our; it can
be your clone if the community prefer that way. And the merge can be applied on
a SVN branch before the SVN trunk.

Post by Justin Deoliveira
I also think we need it to be easy to revert back to the current
referencing code. While I realize you have worked hard to maintain most
of the api and backward compatibility there will undoubtedly be
regressions. Those regressions could be "deal breakers" so depending
projects need a way to fall back onto the old code easily.

Maybe by performing the merge on a SVN branch before the trunk? Or yet before
that, maybe by trying to checkout GeoTools from the Mercurial clone where the
merge will have been applied?

Martin

[Justin Deoliveira]

Post by Martin Desruisseaux
Hello Justin

Post by Justin Deoliveira
Sounds like you are making good progress. That said I would like to
see a formal proposal with voting before any changes come back to
geotools. Unfortunately the forking nature of how this work has been
done represents a lot of risk to projects that depend critically on
the geotools referencing code.

Sure, it sound normal. But it is not a "take it or leave it" deal; the
merge with GeoTools will exists publicly on Mercurial anyway, so anyone
can create a clone and applies his own changes before an eventual commit
to the SVN.

Then this repository should should not be called "GeoTools" in any way
until the PSC/community votes it the official repository, or until the
PSC/community accepts the merge of geotools + geotidy as "geotools 3.0".

There is already too much confusion around the relationship between
GeoTidy and GeoTools. People have been told that GeoTidy == GeoTools
3.0, which is completely false and misleading imo.

The Mercurial repository to be merged to GeoTools SVN

Post by Martin Desruisseaux
doesn't need to be our; it can be your clone if the community prefer
that way. And the merge can be applied on a SVN branch before the SVN
trunk.

Post by Justin Deoliveira
I also think we need it to be easy to revert back to the current
referencing code. While I realize you have worked hard to maintain
most of the api and backward compatibility there will undoubtedly be
regressions. Those regressions could be "deal breakers" so depending
projects need a way to fall back onto the old code easily.

Maybe by performing the merge on a SVN branch before the trunk? Or yet
before that, maybe by trying to checkout GeoTools from the Mercurial
clone where the merge will have been applied?

Yeah, or perhaps just commenting out the referencing module in
library/pom.xml, and keep the artifact from geotidy with the same group
id and artifact id.

Post by Martin Desruisseaux
Martin

[Simone Giannecchini]

Post by Justin Deoliveira

Post by Martin Desruisseaux
Hello Justin

Post by Justin Deoliveira
Sounds like you are making good progress. That said I would like to
see a formal proposal with voting before any changes come back to
geotools. Unfortunately the forking nature of how this work has been
done represents a lot of risk to projects that depend critically on
the geotools referencing code.

Sure, it sound normal. But it is not a "take it or leave it" deal; the
merge with GeoTools will exists publicly on Mercurial anyway, so anyone
can create a clone and applies his own changes before an eventual commit
to the SVN.

Then this repository should should not be called "GeoTools" in any way
until the PSC/community votes it the official repository, or until the
PSC/community accepts the merge of geotools + geotidy as "geotools 3.0".
There is already too much confusion around the relationship between
GeoTidy and GeoTools. People have been told that GeoTidy == GeoTools
3.0, which is completely false and misleading imo.

I agree with Justin, I don't think I am discovering a new planet by
saying that how this whole geotidy vs geotools thing has been
presented is, at least, confusing if not deceiving (as usual no intent
to start a fight). Many people have asked me what was going and I
found it quite difficult to explain that this was not a fork, even
because that is what appears to me as well, at least so far.
Nevertheless, my intention is to try and make as many people as
possible happy (including myself of course). Therefore, yeah, let's
keep things clearly separated, but let's try to work in a way that we
can merge them as soon as we can. Of course, IMHO, this needs a rather
different approach with respect to the "this is what I have done"
approach that has been used so far, otherwise I am pretty sure the
following code will loop forever.

while(true){
sleep(2_weeks);




Simone.

Post by Justin Deoliveira
The Mercurial repository to be merged to GeoTools SVN

Post by Martin Desruisseaux
doesn't need to be our; it can be your clone if the community prefer
that way. And the merge can be applied on a SVN branch before the SVN
trunk.

Post by Justin Deoliveira
I also think we need it to be easy to revert back to the current
referencing code. While I realize you have worked hard to maintain
most of the api and backward compatibility there will undoubtedly be
regressions. Those regressions could be "deal breakers" so depending
projects need a way to fall back onto the old code easily.

Maybe by performing the merge on a SVN branch before the trunk? Or yet
before that, maybe by trying to checkout GeoTools from the Mercurial
clone where the merge will have been applied?

Yeah, or perhaps just commenting out the referencing module in
library/pom.xml, and keep the artifact from geotidy with the same group
id and artifact id.

Post by Martin Desruisseaux
    Martin

--
Justin Deoliveira
OpenGeo - http://opengeo.org
Enterprise support for open source geospatial.
------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
Geotools-devel mailing list
https://lists.sourceforge.net/lists/listinfo/geotools-devel

--
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Owner - Software Engineer
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: [phone]
fax: [phone]
mob: [phone]


http://www.geo-solutions.it
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------

[Simone Giannecchini]

Post by Justin Deoliveira

Post by Martin Desruisseaux
Hello Justin

Post by Justin Deoliveira
Sounds like you are making good progress. That said I would like to
see a formal proposal with voting before any changes come back to
geotools. Unfortunately the forking nature of how this work has been
done represents a lot of risk to projects that depend critically on
the geotools referencing code.

Sure, it sound normal. But it is not a "take it or leave it" deal; the
merge with GeoTools will exists publicly on Mercurial anyway, so anyone
can create a clone and applies his own changes before an eventual commit
to the SVN.

Then this repository should should not be called "GeoTools" in any way
until the PSC/community votes it the official repository, or until the
PSC/community accepts the merge of geotools + geotidy as "geotools 3.0".
There is already too much confusion around the relationship between
GeoTidy and GeoTools. People have been told that GeoTidy == GeoTools
3.0, which is completely false and misleading imo.

I agree with Justin, I don't think I am discovering a new planet by
saying that how this whole geotidy vs geotools thing has been
presented is, at least, confusing if not deceiving (as usual no intent
to start a fight). Many people have asked me what was going and I
found it quite difficult to explain that this was not a fork, even
because that is what appears to me as well, at least so far.
Nevertheless, my intention is to try and make as many people as
possible happy (including myself of course). Therefore, yeah, let's
keep things clearly separated, but let's try to work in a way that we
can merge them as soon as we can. Of course, IMHO, this needs a rather
different approach with respect to the "this is what I have done"
approach that has been used so far, otherwise I am pretty sure the
following code will loop forever.

while(true){
sleep(2_weeks);
if(geotidy == geotools-3.0)
break;
}



Ciao,
Simone.

[Andrea Aime]

Post by Martin Desruisseaux
This bring the question about when geotidy can be offered for a merge with
GeoTools. Currently the referencing module is done, except for Oracle and HSQL
plugins of EPSG (I'm doing my test on PostgreSQL for now) and the builder
package, maybe to move in an other module.

GeoTidy has been source of many headaches so far, at least for me.
I appreciate the need for cleaning up the code, and I'm _very_
interested in the list of improvements and fixes you've implemented
there (I really mean it).

On the other side, GeoTidy has been created on a separate repository,
and managed by a single person, which makes it look a lot like an
hostile fork.
The list of changes
(http://geotidy.geomatys.fr/build/tools/gt-migrate/changes.html)
also says in some places GeoTidy is GeoTools 3, but there is no PSC
agreement on it, it's a single handed decision.
Also, to my knowledge no-one outside of Geomatys has right now the
resources or the need to create a GeoTools 3 branch.

I thought the plan was to make GeoTidy a replacement of the referencing
modules that could be used by GeoTools trunk, yet the number of
changes, removed and renamed classes seems to preclude a solution
in which the GeoTidy changes are simply merged into the GeoTools SVN
(that is, the place where the official GeoTools library is stored).

An alternative path that seems promising would be to build GeoTidy
a GeoTools compatible version that GeoTools trunk can depend onto,
just like we depend on many other external libraries.
This would also clarify once and for all that GeoTidy is its
own library, separated from GeoTools, on which GeoTools depend
just like it depends on JAI, Jakarta commons, Batik and many others.
It would also fit very well with your intention to be the benevolent
dictator of GeoTidy, preventing anyone else to directly commit on it.

It also seems, and this is new to me, that GeoTidy contains also
coverage handling classes. Mixed with your intention of being a
benevolent dictator of your modules I find this very disturbing,
as it would prevent Simone from contributing to it (after all
you said at FOSS4G 2008 that you don't want anything to do
with Simone, with various other PSC members present that can confirm
what I've just wrote).
This is very problematic, as from a GeoServer point of view:
Simone work is what made the difference between unusable raster
support to something that can rival with MapServer on standard imagery.
Also, we have a very strong need for a more serious coverage I/O
support that the "coverage store" proposal seems to match pretty
effectively.
I appreciate that you're open to work on that, but past experience
show that does not mean we'll get anything usable in the
year to come. After all, Jody's threading issues have been officially
considered and merged into GeoTidy 1.5 years after he proposed
them, no? I don't think we can wait that much, plus I don't
want a situation in which you're the sole judge of what's
good and wrong in a proposal, this is a community, and the
PSC is the only valid judge on proposals.

You've completely bypassed the PSC and the proposals
mechanism for all the GeoTidy code changes, which makes me believe
you're not interested in collaborating with GeoTools by its
rules. In your own library instead you're free to do pretty much
what you want, without asking other people and without compromising,
but that's no more a community effort, that's no more GeoTools.

So, whilst I see a viable future for a GeoTools dependency on
GeoTidy as an external referencing library, I have reservations
that the same could be made work for coverages.

In order to bring GeoTidy back home it is my personal opinion that
two things would be needed:
- make all the API changes occurred in GeoTidy go through
a standard GeoTools proposal process and merge back on svn
- accept the fact that the proposal mechanism in GeoTools
might overrule your judgement on coverage handling.
In particular, when the coverage store api reaches the
proposal stage, it may well happen that you have the only
-1 on the proposal. In that case after two subsequent votes
the proposed changes would go in even if you're the maintainer
of the module
As I said, this is only my point of view, a PSC vote is needed
to deal with this matter, but first we have to sort out a
list of possible action plans, discuss, and vote on the best one.

Btw, a shared and distributed community is one of the requirements for
OSGEO incubation, DeeGree is having troubles incubating because
there is a single company behind the project. They only lately
went past that issue:
http://wiki.osgeo.org/wiki/Deegree_Incubation_Status
See the list of OSGEO principles:
http://www.osgeo.org/incubator/process/principles.html
A benevolent dictator model is nor open nor collaborative.
See for example GDAL, where Frank W. had to give up the benevolent
dictator model for a PSC made of people coming from different
organizations.

One last hard question.
You're a PSC memember, yet the whole GeoTidy thing seems
to either suggest you're either not recognizing the PSC anymore
(given you've bypassed the proposal mechanism whilst still
using the org.geotools package names)
or that you already decided that GeoTidy is a separate library.
Which one is it?

Cheers
Andrea

[Jody Garnett]

Hi Andrea;

How to deal with distributed version control (so far I would like to
treat them as branches?) is going to be something we need solved this
year; indeed I mentioned that in my status report to OSGeo.

We actually have a smaller branch to try this out on in the form of
adding OSGi manifest information.

As indicated earlier I am mostly ignoring geotidy as a branch since I
have no say in its direction or development. When there is a proposal
I have a roll to play and will pay attention. Ideally I like the idea
of figuring out tough problems on a branch where there is less risk of
tripping over each other; however it makes the merge process more
expensive for everyone involved.

Jody

PS. I agree with your points about being careful about branding (and
really want to avoid a situtation like with complex features where a
branch was run for years). I actually made the same point on the udig
list last week; also about the OSGi manifest repo/branch.

Post by Andrea Aime

Post by Martin Desruisseaux
This bring the question about when geotidy can be offered for a merge with
GeoTools. Currently the referencing module is done, except for Oracle and HSQL
plugins of EPSG (I'm doing my test on PostgreSQL for now) and the builder
package, maybe to move in an other module.

GeoTidy has been source of many headaches so far, at least for me.
I appreciate the need for cleaning up the code, and I'm _very_
interested in the list of improvements and fixes you've implemented
there (I really mean it).
On the other side, GeoTidy has been created on a separate repository,
and managed by a single person, which makes it look a lot like an
hostile fork.
The list of changes
(http://geotidy.geomatys.fr/build/tools/gt-migrate/changes.html)
also says in some places GeoTidy is GeoTools 3, but there is no PSC
agreement on it, it's a single handed decision.
Also, to my knowledge no-one outside of Geomatys has right now the
resources or the need to create a GeoTools 3 branch.
I thought the plan was to make GeoTidy a replacement of the referencing
modules that could be used by GeoTools trunk, yet the number of
changes, removed and renamed classes seems to preclude a solution
in which the GeoTidy changes are simply merged into the GeoTools SVN
(that is, the place where the official GeoTools library is stored).
An alternative path that seems promising would be to build GeoTidy
a GeoTools compatible version that GeoTools trunk can depend onto,
just like we depend on many other external libraries.
This would also clarify once and for all that GeoTidy is its
own library, separated from GeoTools, on which GeoTools depend
just like it depends on JAI, Jakarta commons, Batik and many others.
It would also fit very well with your intention to be the benevolent
dictator of GeoTidy, preventing anyone else to directly commit on it.
It also seems, and this is new to me, that GeoTidy contains also
coverage handling classes. Mixed with your intention of being a
benevolent dictator of your modules I find this very disturbing,
as it would prevent Simone from contributing to it (after all
you said at FOSS4G 2008 that you don't want anything to do
with Simone, with various other PSC members present that can confirm
what I've just wrote).
Simone work is what made the difference between unusable raster
support to something that can rival with MapServer on standard imagery.
Also, we have a very strong need for a more serious coverage I/O
support that the "coverage store" proposal seems to match pretty
effectively.
I appreciate that you're open to work on that, but past experience
show that does not mean we'll get anything usable in the
year to come. After all, Jody's threading issues have been officially
considered and merged into GeoTidy 1.5 years after he proposed
them, no? I don't think we can wait that much, plus I don't
want a situation in which you're the sole judge of what's
good and wrong in a proposal, this is a community, and the
PSC is the only valid judge on proposals.
You've completely bypassed the PSC and the proposals
mechanism for all the GeoTidy code changes, which makes me believe
you're not interested in collaborating with GeoTools by its
rules. In your own library instead you're free to do pretty much
what you want, without asking other people and without compromising,
but that's no more a community effort, that's no more GeoTools.
So, whilst I see a viable future for a GeoTools dependency on
GeoTidy as an external referencing library, I have reservations
that the same could be made work for coverages.
In order to bring GeoTidy back home it is my personal opinion that
- make all the API changes occurred in GeoTidy go through
  a standard GeoTools proposal process and merge back on svn
- accept the fact that the proposal mechanism in GeoTools
  might overrule your judgement on coverage handling.
  In particular, when the coverage store api reaches the
  proposal stage, it may well happen that you have the only
  -1 on the proposal. In that case after two subsequent votes
  the proposed changes would go in even if you're the maintainer
  of the module
As I said, this is only my point of view, a PSC vote is needed
to deal with this matter, but first we have to sort out a
list of possible action plans, discuss, and vote on the best one.
Btw, a shared and distributed community is one of the requirements for
OSGEO incubation, DeeGree is having troubles incubating because
there is a single company behind the project. They only lately
http://wiki.osgeo.org/wiki/Deegree_Incubation_Status
http://www.osgeo.org/incubator/process/principles.html
A benevolent dictator model is nor open nor collaborative.
See for example GDAL, where Frank W. had to give up the benevolent
dictator model for a PSC made of people coming from different
organizations.
One last hard question.
You're a PSC memember, yet the whole GeoTidy thing seems
to either suggest you're either not recognizing the PSC anymore
(given you've bypassed the proposal mechanism whilst still
using the org.geotools package names)
or that you already decided that GeoTidy is a separate library.
Which one is it?
Cheers
Andrea
--
Andrea Aime
OpenGeo - http://opengeo.org
Expert service straight from the developers.
------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
Geotools-devel mailing list
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[Martin Desruisseaux]

Hello Andrea

Putting "geotidy" under a GeoTools umbrella (either as a separated Maven project
or merged to SVN) is an offering that we make to the community, but as far as we
(at geomatys) are concerned we have no obligation to do so. This offer has been
made repeatedly since the begining. If the PSC don't want it, we are completly
fine with that.

Creating geotidy on a separated repository may have been a source of critisism,
but sincerly it would have been impossible for me to do otherwise. GeoTools SVN
has many thousands of javac and javadoc warnings. I can't fix everyone warnings
and I want to work on the warnings in the modules that I maintain without being
bured under everyone warnings. Furthermore geotidy has been a public repository
since its begining.

You said that I'm aiming to be a "dictator". However every modules in Geotidy,
including Coverage, are mine. More important, I'm the benevolent dictator on my
Mercurial clone of geotidy. Anyone can be the dictator of their own clone of the
repository, can commit whatever changes he wants on his clone and can share his
changes with who he wants. Simone or anyone else can not be prevented by myself
from commiting, and nothing can prevent Geoserver from using Simone's work. The
usage of a Distributed Versionning System is at the heart of this Geotidy move,
you really need to understand its principle.

About the "Threaded EPSG" story: Try to imagine the opposite scenario. Imagine
that a big compagny (said ESRI - I'm just inventing) paids Geomatys for doing
major work on the JTS library. Geomatys had never touched JTS code before.
Geomatys sign the contract then said to Martin Davis (the JTS author and
maintainer) "Hey! we are going to change big parts of JTS. You are not going to
work with us (unless for free); please let just agree on the plan and let us do
the code". Geomatys duplicates lot of code, then leaves the library in a messy
state and wait for Martin Davis to clean this code for free. A few JTS releases
are made in that state before Martin Davis finally have a chance to work 3 weeks
- for free - cleaning Geomatys work, fixing bugs and logical inconsistencies.
Then someone complains that it tooks years before he did this job, suggesting
that it is all Martin's fault.

This kind of story is part of the reasons why we created geotidy.


Martin

[Simone Giannecchini]

On Wed, Mar 25, 2009 at 2:58 PM, Martin Desruisseaux

Post by Martin Desruisseaux
Hello Andrea
Putting "geotidy" under a GeoTools umbrella (either as a separated Maven project
or merged to SVN) is an offering that we make to the community, but as far as we
(at geomatys) are concerned we have no obligation to do so. This offer has been
made repeatedly since the begining. If the PSC don't want it, we are completly
fine with that.
Creating geotidy on a separated repository may have been a source of critisism,
but sincerly it would have been impossible for me to do otherwise. GeoTools SVN
has many thousands of javac and javadoc warnings. I can't fix everyone warnings
and I want to work on the warnings in the modules that I maintain without being
bured under everyone warnings. Furthermore geotidy has been a public repository
since its begining.
You said that I'm aiming to be a "dictator". However every modules in Geotidy,
including Coverage, are mine

This is -THE- problem Martin, the modules are not YOURS (maintainer
!= owner), the modules are part of the project and being so someone
else may change them providing that is follows the rules, which means
proposal and everything. If you don't have time to review changes,
even roughly, you cannot stop them. You are the original author but
this should give you only a limited amount of special rights on the
modules, if you like it or not.
This is how open source usually works, what you describe, is what I
call closed-open source, you want people to use your work not to
participate in it.

Post by Martin Desruisseaux
More important, I'm the benevolent dictator on my
Mercurial clone of geotidy. Anyone can be the dictator of their own clone of the
repository, can commit whatever changes he wants on his clone and can share his
changes with who he wants. Simone or anyone else can not be prevented by myself
from commiting, and nothing can prevent Geoserver from using Simone's work. The
usage of a Distributed Versionning System is at the heart of this Geotidy move,
you really need to understand its principle.
About the "Threaded EPSG" story: Try to imagine the opposite scenario. Imagine
that a big compagny (said ESRI - I'm just inventing) paids Geomatys for doing
major work on the JTS library. Geomatys had never touched JTS code before.
Geomatys sign the contract then said to Martin Davis (the JTS author and
maintainer) "Hey! we are going to change big parts of JTS. You are not going to
work with us (unless for free); please let just agree on the plan and let us do
the code". Geomatys duplicates lot of code, then leaves the library in a messy
state and wait for Martin Davis to clean this code for free. A few JTS releases
are made in that state before Martin Davis finally have a chance to work 3 weeks
- for free - cleaning Geomatys work, fixing bugs and logical inconsistencies.
Then someone complains that it tooks years before he did this job, suggesting
that it is all Martin's fault.
This kind of story is part of the reasons why we created geotidy.

Sorry Martin, but this example is somehow irrelevant. There are
processes to handle this:

- proposal
- discussion
- work

If the work has been done without a proposal, then the people cannot
expect us to accept it, but on the other side, you cannot even expect
people to stop working because you want to always have the last word
on *your* modules or review every single line of code.
You have to realize that this will never happen, even inside a single
company. Therefore when the proposal procedure is in place there
should not be any problems of accepting contributions. Of course we
may want to discuss about improving the proposal stuff and about
setting coding standards and everything as well as about ways to
enforce them.

About what aaime reported, I am going to pretend that you were somehow
misunderstood, since I believe that anyone whose age is >= 12 years
would not behave like it has been reported.
On my side, I will repeat what I usually say, I have always found your
work useful and interesting, I have always promoted/used it myself, I
will always try to collaborate if possible (see my email about
coverage IO), but if this does not happen, I don't think I will cut my
wrists :-).

Btw, for me this discussion is closed, I don't have any more time to
spend on analyzing this geotidy-thing at least until I see plans,
proposals and a collaborative approach. Whatever I will propose on
coverages will be proposed for GeoTools and will follow its
procedures.


Simone.

Post by Martin Desruisseaux
       Martin
------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
Geotools-devel mailing list
https://lists.sourceforge.net/lists/listinfo/geotools-devel

--
-------------------------------------------------------
Ing. Simone Giannecchini
GeoSolutions S.A.S.
Owner - Software Engineer
Via Carignoni 51
55041 Camaiore (LU)
Italy

phone: [phone]
fax: [phone]
mob: [phone]


http://www.geo-solutions.it
http://simboss.blogspot.com/
http://www.linkedin.com/in/simonegiannecchini

-------------------------------------------------------

[Martin Desruisseaux]

Post by Simone Giannecchini
This is -THE- problem Martin, the modules are not YOURS (maintainer
!= owner), the modules are part of the project and being so someone
else may change them providing that is follows the rules, which means
proposal and everything. If you don't have time to review changes,
even roughly, you cannot stop them. You are the original author but
this should give you only a limited amount of special rights on the
modules, if you like it or not.
This is how open source usually works, what you describe, is what I
call closed-open source, you want people to use your work not to
participate in it.

This is your vision, but I disagree that it is the way OpenSource work. Linux
and Gnumeric for instance don't work that way. They do have module maintainers
who have the final word on what goes in. Those projects don't let anyone commit
any code. Commiters must proof that they really master the module, and for some
projects like BSD it takes years of contributions in the form of patches.

I remember someone whose very first contribution in the referencing module was
to break the Object.equals(Object) contract in GeneralEnvelope, because he was
not aware of the relationship between equals and hashCode. I had to revert his
commit and write an email about this aspect of Java. Then he made changes based
on an erroneous understanding of what WeakReference are for - I had to revert
again. Then he replaced some calculations on AffineTransform by calculations on
Envelope without asking me, because Envelope are easier to understand. He did
not realize that Envelopes are ambiguous (they said nothing about axis order,
axis reversal, rotations...). Breaking a module with changes based on wrong Java
and mathematic understanding commited without asking the maintainer is not my
vision of how OpenSource works.

I'm all for collaboration. But given the above, do you understand that I really
want to review every line of code commited in the modules that I maintain? (at
least until I find someone I trust enough). It should not be such a big problem.
You don't need my agreement; do whatever you want on your own Mercurial clone.
The clone that I maintain will have every lines reviewed by myself or by someone
that I trust, but no one is forced to use that clone as his primary source. If
it can be done under a GeoTools umbrela, cool!! If it can not, thats fine. We
will leave GeoTools, thats all.

Martin

[Andrea Aime]

Post by Martin Desruisseaux
This is your vision, but I disagree that it is the way OpenSource
work. Linux and Gnumeric for instance don't work that way. They do
have module maintainers who have the final word on what goes in.
Those projects don't let anyone commit any code. Commiters must proof
that they really master the module, and for some projects like BSD it
takes years of contributions in the form of patches.

There are also successful projects where the model is even more
drastic than ours, as there is no module maintainer, worse, there
is not even an author on each source code file. I'm not talking
about some strange niche project, I'm talking about Subversion
itself, and I have to say, I like this model very much:

http://subversion.tigris.org/
http://video.google.com/videoplay?docid=-4216011961522818645

Plus, all OSGEO projects have a PSC that can overrule eventual
decisions of a module maintainer, provided they have a concept
of module maintainer at all (most don't). And much of the OSGeo
precedents were derived from Apache, which has similar governance rules.
Even in the projects that requires most of the review, like OpenLayers,
where each patch gets reviewed by two other people before being
committed to trunk, there is no single person that can just
decide to block the code contributions, it takes two reviews,
from any PSC member, not from specific ones.

Post by Martin Desruisseaux
I remember someone whose very first contribution in the referencing
module was to break the Object.equals(Object) contract in
GeneralEnvelope, because he was not aware of the relationship between
equals and hashCode. I had to revert his commit and write an email
about this aspect of Java. Then he made changes based on an erroneous
understanding of what WeakReference are for - I had to revert again.
Then he replaced some calculations on AffineTransform by
calculations on Envelope without asking me, because Envelope are
easier to understand. He did not realize that Envelopes are ambiguous
(they said nothing about axis order, axis reversal, rotations...).
Breaking a module with changes based on wrong Java and mathematic
understanding commited without asking the maintainer is not my vision
of how OpenSource works.

Even if Simone didn't quite get things right early on he then turned in
to a very valuable contributor to the library. We all have areas that
we might not know as well as others, and if we don't have the space to
correct people and teach them better then we don't have much at all. The
flip side is something like georeferenced images, which you for many
years refused to accept as a valid GIS use case. Having mechanisms
for the project to learn from the collective intelligence of all
involved _is_ how Open Source works. There may be differences in the
specifics of people's preference for governance models, how that
collective intelligence gets tapped and used with out being a burden.
And it sounds like there are different visions going on here.

Post by Martin Desruisseaux
I'm all for collaboration. But given the above, do you understand
that I really want to review every line of code commited in the
modules that I maintain? (at least until I find someone I trust
enough). It should not be such a big problem. You don't need my
agreement; do whatever you want on your own Mercurial clone. The
clone that I maintain will have every lines reviewed by myself or by
someone that I trust, but no one is forced to use that clone as his
primary source. If it can be done under a GeoTools umbrela, cool!! If
it can not, thats fine. We will leave GeoTools, thats all.

Martin, from where I stand you left GeoTools when you decided to make
your own fork and bypass all the GeoTools governance rules.
Being part of GeoTools is about respecting its rules, we have a PSC,
we have process to perform API changes, and GeoTidy did not respect
either. Which is absolutely fine if you're making your own
fork, not as much if you allow people to see it as part of a GeoTools
"umbrella".
An "umbrella" in which each subproject does what it wants
sounds like an empty label, has no other practical
meaning and does not seem to offer any advantage to either
party over what we already do use "reuse" other projects
jars, that is, go on the Internet, add a maven dependency,
and use what is provided in the manner we see best fit.

GeoTools has traditionally not been a benevolent dictator model, and
joining OSGeo only reinforces that. If GeoTidy chooses a benevolent
dictator model that's great - people working together should have the
same preference in governance models. At some point, maybe when
GeoTools upgrades to Java 6, or when there's a solid Java 5 jar from
GeoTidy that doesn't break anything, then GeoTools
will rely directly on GeoTidy.

Cheers
Andrea

[Justin Deoliveira]

While I have nothing but the utmost respect for your work Martin, Andrea
is right. The community and PSC were totally bypassed with this entire
situation. GeoTools is a community project which has always been run by
its module maintainers. And that as always been its strength imo.
Whenever there are controversial topics brought up we all come to a
consensus, and the best possible product usually emerges.

I can't think of anything more detrimental to an open source project
than allowing people to fork to get around having to collaborate with
others about decisions. You can dress it up all you want and call it
"distributed versioning control" but it is what it is.

The mess that resulted from the threaded EPSG work is unfortunate. But
Simone is correct that there are processes in place to protect against
this sort of thing. It is your job as the maintainer to demand a
proposal, ask for patches, have people work on a branch, etc...

There also the fact that being a community project means doing the best
to accept contributions when they arise, and accepting the fact that
they might not always be aesthetically perfect.

Post by Martin Desruisseaux
Hello Andrea
Putting "geotidy" under a GeoTools umbrella (either as a separated Maven project
or merged to SVN) is an offering that we make to the community, but as far as we
(at geomatys) are concerned we have no obligation to do so. This offer has been
made repeatedly since the begining. If the PSC don't want it, we are completly
fine with that.
Creating geotidy on a separated repository may have been a source of critisism,
but sincerly it would have been impossible for me to do otherwise. GeoTools SVN
has many thousands of javac and javadoc warnings. I can't fix everyone warnings
and I want to work on the warnings in the modules that I maintain without being
bured under everyone warnings. Furthermore geotidy has been a public repository
since its begining.
You said that I'm aiming to be a "dictator". However every modules in Geotidy,
including Coverage, are mine. More important, I'm the benevolent dictator on my
Mercurial clone of geotidy. Anyone can be the dictator of their own clone of the
repository, can commit whatever changes he wants on his clone and can share his
changes with who he wants. Simone or anyone else can not be prevented by myself
from commiting, and nothing can prevent Geoserver from using Simone's work. The
usage of a Distributed Versionning System is at the heart of this Geotidy move,
you really need to understand its principle.
About the "Threaded EPSG" story: Try to imagine the opposite scenario. Imagine
that a big compagny (said ESRI - I'm just inventing) paids Geomatys for doing
major work on the JTS library. Geomatys had never touched JTS code before.
Geomatys sign the contract then said to Martin Davis (the JTS author and
maintainer) "Hey! we are going to change big parts of JTS. You are not going to
work with us (unless for free); please let just agree on the plan and let us do
the code". Geomatys duplicates lot of code, then leaves the library in a messy
state and wait for Martin Davis to clean this code for free. A few JTS releases
are made in that state before Martin Davis finally have a chance to work 3 weeks
- for free - cleaning Geomatys work, fixing bugs and logical inconsistencies.
Then someone complains that it tooks years before he did this job, suggesting
that it is all Martin's fault.
This kind of story is part of the reasons why we created geotidy.
Martin
------------------------------------------------------------------------------
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
_______________________________________________
Geotools-devel mailing list
https://lists.sourceforge.net/lists/listinfo/geotools-devel

[Martin Desruisseaux]

Post by Justin Deoliveira
The mess that resulted from the threaded EPSG work is unfortunate. But
Simone is correct that there are processes in place to protect against
this sort of thing. It is your job as the maintainer to demand a
proposal, ask for patches, have people work on a branch, etc...

The EPSG work has been done on a branch, sleeped there one or two years because
I didn't had the time to review, then I got pressure for accepting the merge on
the trunk without review. I understand the wish to get this work pushed in a
more visible space. DVS, while not the solution to everything neither a fix for
lack of concensus, could nevertheless have been of help.

I think that the GeoTools governance model is broken. Linux, Gnumeric, BSD, Java
don't work the way we are trying to work. We vote on topic about modules that
most of us don't understand enough. We contradict ourself about the maintainer
role. Recents email expose different views about maintenainer roles. And the
following links:

http://docs.codehaus.org/display/GEOT/3+Module+Maintainers
http://docs.codehaus.org/display/GEOT/4+Project+Management+Committee

contradict:

http://docs.codehaus.org/display/GEOT/GeoTools+change+proposal


However a valid point of recent discussion is that "geotidy" should never had a
"geotools 3" label. At the time geotidy was created, it was aimed to be short
lived (I was suppose to finish in December). I'm more than 3 months late and
confusion increase. I'm going to search-and-replace every occurences of
"GeoTools 3" by "<whatever> 3" in the next 48 hours.

Martin