[Flightgear-scenery] Proposal for scenery creation with jenkins

[Geoff McLane]

Hi Yves,

Sorry, it was meant for the list... which is
where I will (hopefully) send this ;=))

As far as I understand terrafitting is optional

Hmmm, yes, I remember reading somewhere that if
you do NOT run terrafit, then fg-construct, and
maybe genapts, use an 'internal' terrafit service,
but this is far from optimized... nor particularly
good a doing a 'substitute' job...

But like you, I hope other can shed some CLEAR
light on this ;=))

While you suggest terrafit helped in filling in
voids, and thus was less needed once the data is
'cleaned', but I understood this is not its sole
purpose...

What I do know, from my scenery building so far,
is that when building a pieces of scenery I
could get very sharp edged ridges, which look
ugly, and certainly not very 'real'.

If I then re-ran terrafit, pushing the default
parameters - increasing the --maxnodes by a
factor of 10 X or even 100 X, and reduced the
--maxerror by a factor of 10, down to say 4,
these very sharp ridges would be 'smoothed' out,
at the cost of an ever increasing 'triangle'
count.

At the time it was possible to push terrafit like
--maxnodes 100000 --maxerror 1 such that the triangle
count went above the 64 K index barrier existing at
the time in the BTG, and thus produced 'spaghetti'
scenery, when the fg renderer index overflowed.

With the addition of the next version of the BTG
file, this 64 K barrier went away, and thus very
'dense' - read many many triangles - was possible.

And while such dense scenery took a LOT longer to
build - terrafit ran for hours - and took extra
time to load from disk, it did not particularly
impact the frame rate, once loaded, and produced
beautiful detailed scenery... will hillside
'curved' rather than flat and angular...

I do not exactly know the 'optimal' parameters
for terrafit, except as explained above, but the
defaults seem far too course...

also support a programmed download,

Do you write me a nice script for that.
http://download.fgx.ch/proposals/mapserver.zip

I will try to find the time to look at this...
will revert after having had a look...

Now note you seem to have moved onto airport
generation in fgx jenkins... still running
last time I looked ;=))

Regards,
Geoff.

-------- Forwarded Message --------

Subject: Re: [Flightgear-scenery] Proposal for scenery creation with
jenkins
Date: Mon, 21 May 2012 00:48:41 +0200
Hi Geoff, this mail reached me private, but maybe it was intended to
send to the list ? Anyway, in case you want to see your questions at
list, my answers included, just feel free to reply to this again and
change the address to scenerylist. I think there are some important
points we start to discuss which might be interesting for others too ...
Thanks, Cheers, Yves
--------------------------------

Hi all
We recently installed a jenkins at fgx.ch for FGx distribution. The idea
came up to test a scenery creation jenkins system. One idea/proposal is
Elevation-chop slaves (world coverage, creating SRTM-3 arrays ready for
use for all scenery devs)
Genapts slaves (world coverage, generating Airport files all over the
world, automated with pulling monthly data cycle of xplane 850 data)
Individual (but powerful) ogr-decode and fgfs-construct slaves,
generating "some degree" x "some degree" tiles, based on updated custom
shapefile repositories
What do you think about this idea ? Any comments welcome.
Cheers, Yves

Hi Yves,
Great idea, and will help where I can... this
extends the elevation information available
from say mapserver...
Some questions, as usual ;=)) Just about Level
1: initially... elevation preparation...
Assume you are chopping those SRTM that you have
'cleaned'?

Yes, and it’s already chopped. Took 24 hours for the all the current
SRTM-3 data. Now it can be used in a "work/SRTM-3" directory.

Are these originally from dds.cr.usgs.gov/srtm
or some other source?
Are these 3-arcsec 'version1' or 'version2_1?

Yes it is this source and it’s version 2_1

And do you intend to also do the 1-arcsec that
are available for USA? And maybe some other places,
given the 'right' license?

I tested SRTM-1 once, and it seems to be to detailed and scenery
stutters a lot. But maybe this excellent data can be reduced to 60 meter
resolution, that’s also what is coming from Alaska and Canada.

After the initial elevation chop, which creates the
'thousands' of<index>.arr.gz files, do you intend
to also then run 'terrafit' to generate the
matching<index>.fit.gz files?
If yes, what parameter will you be using on
'terrafit'? The default --minnodes 50, --maxnodes
1000, and --maxerror 40, or will you 'push'
terrafit higher? If so, to what? Or maybe a
choice?

As far as I understand terrafitting is optional, and I do not see any
reason to run terrafit in case there are no holes anymore, but nobody
knows ;-) Maybe someone could elaborate a bit and make a proposal what
terrafit parameters makes sense with no-data-filled SRTM-3 (90 meters)
resolution.

Finally, with all this beautiful chopped,
and perhaps 'fitted' data available, have you given
any thought to how we can access this?

There are different options for this. All SRTM-3 arr.gz files gives 14
GB of data at the moment, depends on improvements the next months. I can
make this data available trough svn or git. Server and space is not the

Perhaps something like a web interface where I
can select or set a rectangular region, and the
server with say zip all the arr.gz (and fix.gz),
with their full bucket folder information so I
can simply unzip it into one of the folders used
by say genapts, or genapts850, like say SRTM-3,
or SRTM-1, etc in my 'work' folder.
And hopefully it will also support a programmed
download, like say -
wget http://elevations.fgx.ch?minlat=-34&maxlat=-30&\
minlon=146&maxlon=151
to get all the elevation information within
about 100 nm of YGIL ;=))

Do you write me a nice script for that. Much appreciated. Could be perl,
yes ;-)
The webinterface for seleting boxes with openlayers is already prepared
but needs some further improvements of course. (I forgot where I pushed
http://download.fgx.ch/proposals/mapserver.zip )
Unzip and open download.html in a browser, works also locally. Drag on
the map while shift key pressed. This was a proposal for a download page
on the mapserver once.
Thanks a lot, Cheers, Yves

[HB-GRAL]

Post by Geoff McLane
And while such dense scenery took a LOT longer to
build - terrafit ran for hours - and took extra
time to load from disk, it did not particularly
impact the frame rate, once loaded, and produced
beautiful detailed scenery... will hillside
'curved' rather than flat and angular...
I do not exactly know the 'optimal' parameters
for terrafit, except as explained above, but the
defaults seem far too course...

Hi Geoff

Found this
http://www.cullam.com/flightgear.htm

"Terrafit reduces the number of elevation points used to described a
surface. It uses a max error threshold and max number of points to
determine which points are necessary, and which points make little
enough difference to be safely ignored. This works using two point
lists, the "available" list, which starts with all points from the DEM,
and a "final" list, which starts empty. The four corner points are moved
from the available list to the final list, creating two triangular
planes. It then scans the entire available list to find the single point
that is farthest from the a plane, and adds it to the final list. This
will devide one of the triangles in two. This algorithm is applied
recursively, until all remaining available points are within the error
threshold, or the specified maximum number of points is reached."

I can also read that the values of min/max is count of nodes, and errors
means "meter" (?). Also the terragear tools readme doesn’t give more
light here.

I used min=20 max=2000 and error=3 once for mountain areas and this was
not that bad. But maybe someone can bring some more light into this how
good values should/could be set to get good results "all over the world"
with SRTM-3 data ;-)

Or does this need some tests and looing to wireframed results in
flightgear, counting nodes, having a look to the framerate with all
shaders activated ?

Cheers, Yves