Discussion:
Re: irc logs
(too old to reply)
Jody Garnett
2003-10-14 10:21:18 UTC
Permalink
Sorry I missed you Jody, I ran out for food as soon as the session was
wrapping up. Thought I'd email the logs to you guys, as the
procrastination factor of both James and I has made the posting of the
logs a little less than timely lately.
No worries - hard to go through these ircs after the fact...
<cholmes> I don't have much experience with coordinate systems, but does FeatureSource seem like a reasonable place to set them?
I have the *best* suggestion for how to handle coordinate system hints:
as a hint in the Query object.
It seems that we need to provide this service, and make it explicit that
it is only a hint. The Query object
is our only place in data-exp which currently has hints.

I couple more thoughts in reading through this:
If we do move to DataStore.getFeatureReader( Query, Transaction ) I
would like to keep the transaction
separated from the Query request. The two notions are orthogonal and it
would be nice not to tangle them.

I did update DataStore to have a a deprecated collection() method that
returned a FeatureCollection. Although it seems that IanS
and myself were convinced that we could not plan on keeping an In memory
data structure.
<cholmes> So you'll have to construct a FeatureCollection from a FeatureReader, to say that you explicitly want it in memory.
<cholmes> (Correct me if I'm wrong here IanS, or Jody and Sean over email when reading the logs)
<IanS> sounds right
That is right - although FeatureCollection may be retired once rendering
code adjusts to FeatureReader..
<aaime> Oh, hope that there will be a convenience constructor to build a FeatureCollection directly from a FeatureReader...
FeatureResults.collection() is the method
<aaime> Oh, ok. It would be nice to have an updated UML class diagram ;-)
<cholmes> But yes, I don't see any reason not to provide public access to the datasource creating the FeatureResult, for postgis's getBounds method it will need it anyways.
<cholmes> Sorry about no UML, it's in CVS though, which is slightly better than before when they were just abstract ideas over email ;)
<cholmes> Sounds good. Email me the method signatures that you want with some good javadoc comments, or just add them yourself to FeatureSource on data-exp.
Thanks chris - this was going to be my response to the thread :-)
Cheers
Jody
Martin Desruisseaux
2003-10-14 12:25:03 UTC
Permalink
Post by Jody Garnett
as a hint in the Query object.
It seems that we need to provide this service, and make it explicit that
it is only a hint.
I like pretty much this proposal. Thank for making it :)

+1

Martin.
Martin Desruisseaux
2003-10-14 12:40:16 UTC
Permalink
Post by Martin Desruisseaux
Post by Jody Garnett
I have the *best* suggestion for how to handle coordinate system
hints: as a hint in the Query object.
I like pretty much this proposal. Thank for making it :)
+1
Well, looking at the UML, it bring a questions: Since Query is an
argument of 'getFeatures', a user can make different Query on the same
FeatureSource. Consequently, he would have to respecify the
CoordinateSystem for each new Query. Is it right?

Since the CoordinateSystem is often linked to a stream (e.g. a shape
file), I wonder if it should be set close to the place where the
DataStore is created (e.g. close to the place where the file is opened)?

We will need a 'getCoordinateSystem()' method in some place, but it
doesn't need to be in the same class than 'setCoordinateSystemHint'. A
renderer doesn't want to know what was the CS hint; it want to know what
the CS actually is.

Martin.
Jody Garnett
2003-10-14 14:01:18 UTC
Permalink
Post by Martin Desruisseaux
Well, looking at the UML, it bring a questions: Since Query is an
argument of 'getFeatures', a user can make different Query on the same
FeatureSource. Consequently, he would have to respecify the
CoordinateSystem for each new Query. Is it right?
That is correct - I am suggesting this on the grounds that this need is
not required often (most FeatureStores should have an idea of their
CoordinateSystem).
Post by Martin Desruisseaux
Since the CoordinateSystem is often linked to a stream (e.g. a shape
file), I wonder if it should be set close to the place where the
DataStore is created (e.g. close to the place where the file is opened)?
We do have a couple of other options:
DataStore.getFeatureStore( typeName:String) could be repalced with
DataStore.getFeatureStore( featureType:FeatureType ) with featureType
supplying the required CoordianteSystem.
Or Catalog.getDataStore( namespace: String ): DataStore could be pressed
into service by using a method to associate namespaces with a
CoordinateSystem - Catalog.setCoordianteSystem( namespace:String, cs:
CoordinateSystem )

I am afraid I am a bit out of the loop, someone with a better knowledge
of the problem will need to choose...
Post by Martin Desruisseaux
We will need a 'getCoordinateSystem()' method in some place, but it
doesn't need to be in the same class than 'setCoordinateSystemHint'. A
renderer doesn't want to know what was the CS hint; it want to know
what the CS actually is.
Martin.
Chris Holmes
2003-10-14 14:29:13 UTC
Permalink
Post by Jody Garnett
DataStore.getFeatureStore( typeName:String) could be repalced with
DataStore.getFeatureStore( featureType:FeatureType ) with featureType
supplying the required CoordianteSystem.
Or Catalog.getDataStore( namespace: String ): DataStore could be pressed
into service by using a method to associate namespaces with a
CoordinateSystem )
What are your thoughts on namespaces? I'd like to have our featureTypes
return with proper namespaces. I was thinking of a
FeatureSource.setNamespace(String) method, but you mention namespace here
in Catalog.getDataStore. Should each DataStore have its own namespace?
Where do we set the namespace? The FeatureSource gives me more
flexibility with GeoServer, as I currently let users define their
namespaces however they like, and then just hack them in at the end of GML
production chain. But we're getting away from that with the gml reader,
and using FeatureTypes namespace is the place to do that. For that we
need FeatureTypes to be created with the proper namespaces, and
FeatureType creation happens in data.

I've also got a more generic ResultSetAttributeReader and
WKTAttributeReader working, and am making one for each attribute and
passing to a JoiningFeatureReader, which is working. One little thing
that tripped me up was that I had it so each calls ResultSet.next(), which
when they were all split up with the same result set ran through the
results way to fast. Now I have an index of the row, and each calls
ResultSet.absolute(index). It seemed to work, from my small test cases,
but I'd like it tested a lot more. If there's a better way to do that let
me know, I'm a little afraid of everyone using the same resultSet object
and calling absolute on it.

I can commit these right now if anyone wants to check them out, if not
I'll commit right before I leave.

The next thing I'm working on is getting the FeatureType to return right,
with the correct typeName and namespace. And I'm also going to work on
semantics for a FeatureID reader, so we can create features with the
correct IDs.

(sorry for not addressing the CS issue. I'll think about it in a bit)

Chris
Jody Garnett
2003-10-14 14:53:15 UTC
Permalink
Post by Chris Holmes
What are your thoughts on namespaces? I'd like to have our featureTypes
return with proper namespaces. I was thinking of a
FeatureSource.setNamespace(String) method, but you mention namespace here
in Catalog.getDataStore. Should each DataStore have its own namespace?
The idea of a Catalog was mentioned in email (I initially had it work in
terms of typeNames
resulting in an api rather similar to DataStore), after these
corrections Catalog is where you see it operating
with namespaces.

Today I am focusing on Locks in the main branch - I am afraid I do not
know much about our intent with
namespaces.
Post by Chris Holmes
that tripped me up was that I had it so each calls ResultSet.next(), which
when they were all split up with the same result set ran through the
results way to fast. Now I have an index of the row, and each calls
ResultSet.absolute(index). It seemed to work, from my small test cases,
but I'd like it tested a lot more. If there's a better way to do that let
me know, I'm a little afraid of everyone using the same resultSet object
and calling absolute on it.
I am not quite sure how AttributeReaders fit into ResultSet processing
here is my guess:
In the shapefile reader they read the next little bit of the stream
until they have read in an entire Feature.
A direct interpretation of this would be to have an offset into the
current ResultSet which each AttributeReader advances.
In this manner a single AttributeReader could use more than one Column
to store its results.

The FeatureReader would be responsible for calling ResultSet.next() and
setting the offset to 0.
The AttributeReaders would be in charge of calling getObject( offset )
and advancing the offset by the number of columns used.

Cheers,
Jody
Sean Geoghegan
2003-10-14 16:34:04 UTC
Permalink
Chris,

If you can commit this stuff today Ill have a look at it. I plan to
make some progress on the Oracle and the Abstract JDBC data-exp stuff today.

Sean
Post by Chris Holmes
I can commit these right now if anyone wants to check them out, if not
I'll commit right before I leave.
The next thing I'm working on is getting the FeatureType to return right,
with the correct typeName and namespace. And I'm also going to work on
semantics for a FeatureID reader, so we can create features with the
correct IDs.
(sorry for not addressing the CS issue. I'll think about it in a bit)
Chris
-------------------------------------------------------
This SF.net email is sponsored by: SF.net Giveback Program.
SourceForge.net hosts over 70,000 Open Source Projects.
Click here: http://sourceforge.net/supporters.php
_______________________________________________
Geotools-devel mailing list
https://lists.sourceforge.net/lists/listinfo/geotools-devel
Andrea Aime
2003-10-14 22:13:08 UTC
Permalink
Post by Jody Garnett
Sorry I missed you Jody, I ran out for food as soon as the session was
wrapping up. Thought I'd email the logs to you guys, as the
procrastination factor of both James and I has made the posting of the
logs a little less than timely lately.
No worries - hard to go through these ircs after the fact...
<cholmes> I don't have much experience with coordinate systems, but does
FeatureSource seem like a reasonable place to set them?
as a hint in the Query object.
It seems that we need to provide this service, and make it explicit that
it is only a hint. The Query object
is our only place in data-exp which currently has hints.
May be nice, but then we'll have to requery the data source in order to
impose a different CS (case in which the user knows that an eventually
already declared CS in the datasource is wrong). Besides, in this case it
would not be an hint, it would be an order.
Post by Jody Garnett
I did update DataStore to have a a deprecated collection() method that
returned a FeatureCollection. Although it seems that IanS
and myself were convinced that we could not plan on keeping an In memory
data structure.
<cholmes> So you'll have to construct a FeatureCollection from a
FeatureReader, to say that you explicitly want it in memory. <cholmes>
(Correct me if I'm wrong here IanS, or Jody and Sean over email when
reading the logs) <IanS> sounds right
That is right - although FeatureCollection may be retired once rendering
code adjusts to FeatureReader..
Wait wait... the renderer code could try to work without keeping all the data in
memory, but I hope you'll consider how much time it takes to read the data
every time you have to:
* redraw, if the renderer doesn't create itw own caching system
* change style (that would make also Martin's renderer to read back data in order
to recompute the styles).
Just to remind people how things are working, for the moment lite renderer
reads all the data at each redraw and computes styles on the fly, Martin's
thru the RenderedLayerFactory reads everything once to compute styles and
the keeps in memory the styles and the feature geometries (so, if you have to
draw interactively lite-renderer back by FeatureResults is a good choice only
if you have to render a really huge data set once, so big that it's not possible
to make the geometries fit in the heap).
Reading data from the source may invole a lot of expensive operations, such
as disk access, network access, parsing (think about the shapefile dbf file)
and so on: streaming allows us to cope with huge data sets, but a caching
system is badly needed to get consistent performace.
Post by Jody Garnett
<aaime> Oh, hope that there will be a convenience constructor to build a
FeatureCollection directly from a FeatureReader...
And please consider that rendering is not the only concern. For example,
let's say you want to overlay data coming from two different data sources.
The trivial way is to:
* load them into memory
* spatially index one with strtree
* loop onto the other one and for each geometry find the overlapping ones,
compute the overlay for each geometry couple, compute the attributes of
the newly obtained feature, store it
I have this on my disk since months, but I haven't published anywhere because
due to jts limitations (having to recompute the topology at each step, using
doubles instead of singles, and so on) it's embarassingly slow (at least in my opinion).
Now, how slow it could become if you have to do it on a streaming a couple
data source without random access and without native spatial indexes? You need
at the very least efficient random access on the one that's indexed (which means
that you may have to pickle it before doing the overlay, and then perform computation).

Andrea

Loading...